diff options
Diffstat (limited to 'doc')
-rw-r--r-- | doc/README.md | 6 | ||||
-rw-r--r-- | doc/REST-interface.md | 2 | ||||
-rw-r--r-- | doc/build-freebsd.md | 19 | ||||
-rw-r--r-- | doc/build-netbsd.md | 14 | ||||
-rw-r--r-- | doc/build-openbsd.md | 12 | ||||
-rw-r--r-- | doc/build-windows.md | 2 | ||||
-rw-r--r-- | doc/dependencies.md | 2 | ||||
-rw-r--r-- | doc/developer-notes.md | 171 | ||||
-rw-r--r-- | doc/fuzzing.md | 259 | ||||
-rw-r--r-- | doc/init.md | 6 | ||||
-rw-r--r-- | doc/multiprocess.md | 35 | ||||
-rw-r--r-- | doc/productivity.md | 10 | ||||
-rw-r--r-- | doc/rapidcheck.md | 84 | ||||
-rw-r--r-- | doc/reduce-memory.md | 5 | ||||
-rw-r--r-- | doc/reduce-traffic.md | 18 | ||||
-rw-r--r-- | doc/release-notes-16528.md | 0 | ||||
-rw-r--r-- | doc/release-notes-18918.md | 0 | ||||
-rw-r--r-- | doc/release-notes.md | 287 | ||||
-rw-r--r-- | doc/release-notes/release-notes-0.20.0.md | 987 | ||||
-rw-r--r-- | doc/release-process.md | 2 | ||||
-rw-r--r-- | doc/translation_strings_policy.md | 3 | ||||
-rw-r--r-- | doc/zmq.md | 2 |
22 files changed, 1548 insertions, 378 deletions
diff --git a/doc/README.md b/doc/README.md index 4dd21c4620..19d8204d83 100644 --- a/doc/README.md +++ b/doc/README.md @@ -30,7 +30,7 @@ Drag Bitcoin Core to your applications folder, and then run Bitcoin Core. * See the documentation at the [Bitcoin Wiki](https://en.bitcoin.it/wiki/Main_Page) for help and more information. -* Ask for help on [#bitcoin](http://webchat.freenode.net?channels=bitcoin) on Freenode. If you don't have an IRC client, use [webchat here](http://webchat.freenode.net?channels=bitcoin). +* Ask for help on [#bitcoin](https://webchat.freenode.net/#bitcoin) on Freenode. If you don't have an IRC client, use [webchat here](https://webchat.freenode.net/#bitcoin). * Ask for help on the [BitcoinTalk](https://bitcointalk.org/) forums, in the [Technical Support board](https://bitcointalk.org/index.php?board=4.0). Building @@ -66,8 +66,8 @@ The Bitcoin repo's [root README](/README.md) contains relevant information on th ### Resources * Discuss on the [BitcoinTalk](https://bitcointalk.org/) forums, in the [Development & Technical Discussion board](https://bitcointalk.org/index.php?board=6.0). -* Discuss project-specific development on #bitcoin-core-dev on Freenode. If you don't have an IRC client, use [webchat here](http://webchat.freenode.net/?channels=bitcoin-core-dev). -* Discuss general Bitcoin development on #bitcoin-dev on Freenode. If you don't have an IRC client, use [webchat here](http://webchat.freenode.net/?channels=bitcoin-dev). +* Discuss project-specific development on #bitcoin-core-dev on Freenode. If you don't have an IRC client, use [webchat here](https://webchat.freenode.net/#bitcoin-core-dev). +* Discuss general Bitcoin development on #bitcoin-dev on Freenode. If you don't have an IRC client, use [webchat here](https://webchat.freenode.net/#bitcoin-dev). ### Miscellaneous - [Assets Attribution](assets-attribution.md) diff --git a/doc/REST-interface.md b/doc/REST-interface.md index a63391e01a..11e9c90f49 100644 --- a/doc/REST-interface.md +++ b/doc/REST-interface.md @@ -30,7 +30,7 @@ To query for a confirmed transaction, enable the transaction index via "txindex= Given a block hash: returns a block, in binary, hex-encoded binary or JSON formats. Responds with 404 if the block doesn't exist. -The HTTP request and response are both handled entirely in-memory, thus making maximum memory usage at least 2.66MB (1 MB max block, plus hex encoding) per request. +The HTTP request and response are both handled entirely in-memory. With the /notxdetails/ option JSON response will only contain the transaction hash instead of the complete transaction details. The option only affects the JSON response. diff --git a/doc/build-freebsd.md b/doc/build-freebsd.md index 4831623504..f48855a344 100644 --- a/doc/build-freebsd.md +++ b/doc/build-freebsd.md @@ -10,7 +10,7 @@ This guide does not contain instructions for building the GUI. You will need the following dependencies, which can be installed as root via pkg: -```shell +```bash pkg install autoconf automake boost-libs git gmake libevent libtool pkgconf git clone https://github.com/bitcoin/bitcoin.git @@ -18,7 +18,7 @@ git clone https://github.com/bitcoin/bitcoin.git In order to run the test suite (recommended), you will need to have Python 3 installed: -```shell +```bash pkg install python3 ``` @@ -29,32 +29,33 @@ See [dependencies.md](dependencies.md) for a complete overview. BerkeleyDB is only necessary for the wallet functionality. To skip this, pass `--disable-wallet` to `./configure` and skip to the next section. -```shell +```bash ./contrib/install_db4.sh `pwd` export BDB_PREFIX="$PWD/db4" ``` ## Building Bitcoin Core -**Important**: Use `gmake` (the non-GNU `make` will exit with an error): +**Important**: Use `gmake` (the non-GNU `make` will exit with an error). With wallet: -```shell +```bash ./autogen.sh ./configure --with-gui=no \ BDB_LIBS="-L${BDB_PREFIX}/lib -ldb_cxx-4.8" \ - BDB_CFLAGS="-I${BDB_PREFIX}/include" + BDB_CFLAGS="-I${BDB_PREFIX}/include" \ + MAKE=gmake ``` Without wallet: -```shell +```bash ./autogen.sh -./configure --with-gui=no --disable-wallet +./configure --with-gui=no --disable-wallet MAKE=gmake ``` followed by: -```shell +```bash gmake # use -jX here for parallelism gmake check # Run tests if Python 3 is available ``` diff --git a/doc/build-netbsd.md b/doc/build-netbsd.md index ab422f6aa7..47049a780e 100644 --- a/doc/build-netbsd.md +++ b/doc/build-netbsd.md @@ -37,13 +37,13 @@ from ports, for the same reason as boost above (g++/libstd++ incompatibility). If you have to build it yourself, you can use [the installation script included in contrib/](/contrib/install_db4.sh) like so: -```shell +```bash ./contrib/install_db4.sh `pwd` ``` from the root of the repository. Then set `BDB_PREFIX` for the next section: -```shell +```bash export BDB_PREFIX="$PWD/db4" ``` @@ -52,24 +52,26 @@ export BDB_PREFIX="$PWD/db4" **Important**: Use `gmake` (the non-GNU `make` will exit with an error). With wallet: -``` +```bash ./autogen.sh ./configure --with-gui=no CPPFLAGS="-I/usr/pkg/include" \ LDFLAGS="-L/usr/pkg/lib" \ BOOST_CPPFLAGS="-I/usr/pkg/include" \ BOOST_LDFLAGS="-L/usr/pkg/lib" \ BDB_LIBS="-L${BDB_PREFIX}/lib -ldb_cxx-4.8" \ - BDB_CFLAGS="-I${BDB_PREFIX}/include" + BDB_CFLAGS="-I${BDB_PREFIX}/include" \ + MAKE=gmake ``` Without wallet: -``` +```bash ./autogen.sh ./configure --with-gui=no --disable-wallet \ CPPFLAGS="-I/usr/pkg/include" \ LDFLAGS="-L/usr/pkg/lib" \ BOOST_CPPFLAGS="-I/usr/pkg/include" \ - BOOST_LDFLAGS="-L/usr/pkg/lib" + BOOST_LDFLAGS="-L/usr/pkg/lib" \ + MAKE=gmake ``` Build and run the tests: diff --git a/doc/build-openbsd.md b/doc/build-openbsd.md index dad2566a6c..53c647ae34 100644 --- a/doc/build-openbsd.md +++ b/doc/build-openbsd.md @@ -38,19 +38,19 @@ from ports, for the same reason as boost above (g++/libstd++ incompatibility). If you have to build it yourself, you can use [the installation script included in contrib/](/contrib/install_db4.sh) like so: -```shell +```bash ./contrib/install_db4.sh `pwd` CC=cc CXX=c++ ``` from the root of the repository. Then set `BDB_PREFIX` for the next section: -```shell +```bash export BDB_PREFIX="$PWD/db4" ``` ### Building Bitcoin Core -**Important**: use `gmake`, not `make`. The non-GNU `make` will exit with a horrible error. +**Important**: Use `gmake` (the non-GNU `make` will exit with an error). Preparation: ```bash @@ -70,12 +70,14 @@ Make sure `BDB_PREFIX` is set to the appropriate path from the above steps. To configure with wallet: ```bash ./configure --with-gui=no CC=cc CXX=c++ \ - BDB_LIBS="-L${BDB_PREFIX}/lib -ldb_cxx-4.8" BDB_CFLAGS="-I${BDB_PREFIX}/include" + BDB_LIBS="-L${BDB_PREFIX}/lib -ldb_cxx-4.8" \ + BDB_CFLAGS="-I${BDB_PREFIX}/include" \ + MAKE=gmake ``` To configure without wallet: ```bash -./configure --disable-wallet --with-gui=no CC=cc CXX=c++ +./configure --disable-wallet --with-gui=no CC=cc CXX=c++ MAKE=gmake ``` Build and run the tests: diff --git a/doc/build-windows.md b/doc/build-windows.md index bbff638b90..d3dc467f19 100644 --- a/doc/build-windows.md +++ b/doc/build-windows.md @@ -9,7 +9,7 @@ The options known to work for building Bitcoin Core on Windows are: and is the platform used to build the Bitcoin Core Windows release binaries. * On Windows, using [Windows Subsystem for Linux (WSL)](https://docs.microsoft.com/windows/wsl/about) and the Mingw-w64 cross compiler tool chain. -* On Windows, using a native compiler tool chain such as [Visual Studio](https://www.visualstudio.com). +* On Windows, using a native compiler tool chain such as [Visual Studio](https://www.visualstudio.com). See [README.md](/build_msvc/README.md). Other options which may work, but which have not been extensively tested are (please contribute instructions): diff --git a/doc/dependencies.md b/doc/dependencies.md index 51a2240107..0cb5311e8b 100644 --- a/doc/dependencies.md +++ b/doc/dependencies.md @@ -13,7 +13,7 @@ These are the dependencies currently used by Bitcoin Core. You can find instruct | FreeType | [2.7.1](https://download.savannah.gnu.org/releases/freetype) | | No | | [Yes](https://github.com/bitcoin/bitcoin/blob/master/depends/packages/qt.mk) (Android only) | | GCC | | [4.8+](https://gcc.gnu.org/) (C++11 support) | | | | | HarfBuzz-NG | | | | | [Yes](https://github.com/bitcoin/bitcoin/blob/master/depends/packages/qt.mk) | -| libevent | [2.1.11-stable](https://github.com/libevent/libevent/releases) | 2.0.22 | No | | | +| libevent | [2.1.11-stable](https://github.com/libevent/libevent/releases) | [2.0.21](https://github.com/bitcoin/bitcoin/pull/18676) | No | | | | libpng | | | | | [Yes](https://github.com/bitcoin/bitcoin/blob/master/depends/packages/qt.mk) | | librsvg | | | | | | | MiniUPnPc | [2.0.20180203](https://miniupnp.tuxfamily.org/files) | | No | | | diff --git a/doc/developer-notes.md b/doc/developer-notes.md index 6e8bde47f8..b33b3ad18a 100644 --- a/doc/developer-notes.md +++ b/doc/developer-notes.md @@ -43,6 +43,7 @@ Developer Notes - [Suggestions and examples](#suggestions-and-examples) - [Release notes](#release-notes) - [RPC interface guidelines](#rpc-interface-guidelines) + - [Internal interface guidelines](#internal-interface-guidelines) <!-- markdown-toc end --> @@ -422,27 +423,52 @@ and its `cs_KeyStore` lock for example). Threads ------- -- ThreadScriptCheck : Verifies block scripts. +- [Main thread (`bitcoind`)](https://doxygen.bitcoincore.org/bitcoind_8cpp.html#a0ddf1224851353fc92bfbff6f499fa97) + : Started from `main()` in `bitcoind.cpp`. Responsible for starting up and + shutting down the application. -- ThreadImport : Loads blocks from `blk*.dat` files or `-loadblock=<file>`. +- [ThreadImport (`b-loadblk`)](https://doxygen.bitcoincore.org/init_8cpp.html#ae9e290a0e829ec0198518de2eda579d1) + : Loads blocks from `blk*.dat` files or `-loadblock=<file>` on startup. -- ThreadDNSAddressSeed : Loads addresses of peers from the DNS. +- [ThreadScriptCheck (`b-scriptch.x`)](https://doxygen.bitcoincore.org/validation_8cpp.html#a925a33e7952a157922b0bbb8dab29a20) + : Parallel script validation threads for transactions in blocks. -- ThreadMapPort : Universal plug-and-play startup/shutdown. +- [ThreadHTTP (`b-http`)](https://doxygen.bitcoincore.org/httpserver_8cpp.html#abb9f6ea8819672bd9a62d3695070709c) + : Libevent thread to listen for RPC and REST connections. -- ThreadSocketHandler : Sends/Receives data from peers on port 8333. +- [HTTP worker threads(`b-httpworker.x`)](https://doxygen.bitcoincore.org/httpserver_8cpp.html#aa6a7bc27265043bc0193220c5ae3a55f) + : Threads to service RPC and REST requests. -- ThreadOpenAddedConnections : Opens network connections to added nodes. +- [Indexer threads (`b-txindex`, etc)](https://doxygen.bitcoincore.org/class_base_index.html#a96a7407421fbf877509248bbe64f8d87) + : One thread per indexer. -- ThreadOpenConnections : Initiates new connections to peers. +- [SchedulerThread (`b-scheduler`)](https://doxygen.bitcoincore.org/class_c_scheduler.html#a14d2800815da93577858ea078aed1fba) + : Does asynchronous background tasks like dumping wallet contents, dumping + addrman and running asynchronous validationinterface callbacks. -- ThreadMessageHandler : Higher-level message handling (sending and receiving). +- [TorControlThread (`b-torcontrol`)](https://doxygen.bitcoincore.org/torcontrol_8cpp.html#a4faed3692d57a0d7bdbecf3b37f72de0) + : Libevent thread for tor connections. -- DumpAddresses : Dumps IP addresses of nodes to `peers.dat`. +- Net threads: -- ThreadRPCServer : Remote procedure call handler, listens on port 8332 for connections and services them. + - [ThreadMessageHandler (`b-msghand`)](https://doxygen.bitcoincore.org/class_c_connman.html#aacdbb7148575a31bb33bc345e2bf22a9) + : Application level message handling (sending and receiving). Almost + all net_processing and validation logic runs on this thread. -- Shutdown : Does an orderly shutdown of everything. + - [ThreadDNSAddressSeed (`b-dnsseed`)](https://doxygen.bitcoincore.org/class_c_connman.html#aa7c6970ed98a4a7bafbc071d24897d13) + : Loads addresses of peers from the DNS. + + - [ThreadMapPort (`b-upnp`)](https://doxygen.bitcoincore.org/net_8cpp.html#a63f82a71c4169290c2db1651a9bbe249) + : Universal plug-and-play startup/shutdown. + + - [ThreadSocketHandler (`b-net`)](https://doxygen.bitcoincore.org/class_c_connman.html#a765597cbfe99c083d8fa3d61bb464e34) + : Sends/Receives data from peers on port 8333. + + - [ThreadOpenAddedConnections (`b-addcon`)](https://doxygen.bitcoincore.org/class_c_connman.html#a0b787caf95e52a346a2b31a580d60a62) + : Opens network connections to added nodes. + + - [ThreadOpenConnections (`b-opencon`)](https://doxygen.bitcoincore.org/class_c_connman.html#a55e9feafc3bab78e5c9d408c207faa45) + : Initiates new connections to peers. Ignoring IDE/editor files -------------------------- @@ -964,7 +990,7 @@ Some good examples of scripted-diff: - [scripted-diff: Rename InitInterfaces to NodeContext](https://github.com/bitcoin/bitcoin/commit/301bd41a2e6765b185bd55f4c541f9e27aeea29d) uses an elegant script to replace occurrences of multiple terms in all source files. -- [scripted-diff: Remove g_connman, g_banman globals](https://github.com/bitcoin/bitcoin/commit/301bd41a2e6765b185bd55f4c541f9e27aeea29d) +- [scripted-diff: Remove g_connman, g_banman globals](https://github.com/bitcoin/bitcoin/commit/8922d7f6b751a3e6b3b9f6fb7961c442877fb65a) replaces specific terms in a list of specific source files. - [scripted-diff: Replace fprintf with tfm::format](https://github.com/bitcoin/bitcoin/commit/fac03ec43a15ad547161e37e53ea82482cc508f9) @@ -1100,3 +1126,124 @@ A few guidelines for introducing and reviewing new RPC interfaces: timestamps in the documentation. - *Rationale*: User-facing consistency. + +Internal interface guidelines +----------------------------- + +Internal interfaces between parts of the codebase that are meant to be +independent (node, wallet, GUI), are defined in +[`src/interfaces/`](../src/interfaces/). The main interface classes defined +there are [`interfaces::Chain`](../src/interfaces/chain.h), used by wallet to +access the node's latest chain state, +[`interfaces::Node`](../src/interfaces/node.h), used by the GUI to control the +node, and [`interfaces::Wallet`](../src/interfaces/wallet.h), used by the GUI +to control an individual wallet. There are also more specialized interface +types like [`interfaces::Handler`](../src/interfaces/handler.h) +[`interfaces::ChainClient`](../src/interfaces/chain.h) passed to and from +various interface methods. + +Interface classes are written in a particular style so node, wallet, and GUI +code doesn't need to run in the same process, and so the class declarations +work more easily with tools and libraries supporting interprocess +communication: + +- Interface classes should be abstract and have methods that are [pure + virtual](https://en.cppreference.com/w/cpp/language/abstract_class). This + allows multiple implementations to inherit from the same interface class, + particularly so one implementation can execute functionality in the local + process, and other implementations can forward calls to remote processes. + +- Interface method definitions should wrap existing functionality instead of + implementing new functionality. Any substantial new node or wallet + functionality should be implemented in [`src/node/`](../src/node/) or + [`src/wallet/`](../src/wallet/) and just exposed in + [`src/interfaces/`](../src/interfaces/) instead of being implemented there, + so it can be more modular and accessible to unit tests. + +- Interface method parameter and return types should either be serializable or + be other interface classes. Interface methods shouldn't pass references to + objects that can't be serialized or accessed from another process. + + Examples: + + ```c++ + // Good: takes string argument and returns interface class pointer + virtual unique_ptr<interfaces::Wallet> loadWallet(std::string filename) = 0; + + // Bad: returns CWallet reference that can't be used from another process + virtual CWallet& loadWallet(std::string filename) = 0; + ``` + + ```c++ + // Good: accepts and returns primitive types + virtual bool findBlock(const uint256& hash, int& out_height, int64_t& out_time) = 0; + + // Bad: returns pointer to internal node in a linked list inaccessible to + // other processes + virtual const CBlockIndex* findBlock(const uint256& hash) = 0; + ``` + + ```c++ + // Good: takes plain callback type and returns interface pointer + using TipChangedFn = std::function<void(int block_height, int64_t block_time)>; + virtual std::unique_ptr<interfaces::Handler> handleTipChanged(TipChangedFn fn) = 0; + + // Bad: returns boost connection specific to local process + using TipChangedFn = std::function<void(int block_height, int64_t block_time)>; + virtual boost::signals2::scoped_connection connectTipChanged(TipChangedFn fn) = 0; + ``` + +- For consistency and friendliness to code generation tools, interface method + input and inout parameters should be ordered first and output parameters + should come last. + + Example: + + ```c++ + // Good: error output param is last + virtual bool broadcastTransaction(const CTransactionRef& tx, CAmount max_fee, std::string& error) = 0; + + // Bad: error output param is between input params + virtual bool broadcastTransaction(const CTransactionRef& tx, std::string& error, CAmount max_fee) = 0; + ``` + +- For friendliness to code generation tools, interface methods should not be + overloaded: + + Example: + + ```c++ + // Good: method names are unique + virtual bool disconnectByAddress(const CNetAddr& net_addr) = 0; + virtual bool disconnectById(NodeId id) = 0; + + // Bad: methods are overloaded by type + virtual bool disconnect(const CNetAddr& net_addr) = 0; + virtual bool disconnect(NodeId id) = 0; + ``` + +- For consistency and friendliness to code generation tools, interface method + names should be `lowerCamelCase` and standalone function names should be + `UpperCamelCase`. + + Examples: + + ```c++ + // Good: lowerCamelCase method name + virtual void blockConnected(const CBlock& block, int height) = 0; + + // Bad: uppercase class method + virtual void BlockConnected(const CBlock& block, int height) = 0; + ``` + + ```c++ + // Good: UpperCamelCase standalone function name + std::unique_ptr<Node> MakeNode(LocalInit& init); + + // Bad: lowercase standalone function + std::unique_ptr<Node> makeNode(LocalInit& init); + ``` + + Note: This last convention isn't generally followed outside of + [`src/interfaces/`](../src/interfaces/), though it did come up for discussion + before in [#14635](https://github.com/bitcoin/bitcoin/pull/14635). diff --git a/doc/fuzzing.md b/doc/fuzzing.md index c34ca4cb59..419b1db44e 100644 --- a/doc/fuzzing.md +++ b/doc/fuzzing.md @@ -1,138 +1,157 @@ -Fuzz-testing Bitcoin Core -========================== - -A special test harness in `src/test/fuzz/` is provided for each fuzz target to -provide an easy entry point for fuzzers and the like. In this document we'll -describe how to use it with AFL and libFuzzer. - -## Preparing fuzzing - -The fuzzer needs some inputs to work on, but the inputs or seeds can be used -interchangeably between libFuzzer and AFL. - -Extract the example seeds (or other starting inputs) into the inputs -directory before starting fuzzing. - -``` -git clone https://github.com/bitcoin-core/qa-assets -export DIR_FUZZ_IN=$PWD/qa-assets/fuzz_seed_corpus -``` - -AFL needs an input directory with examples, and an output directory where it -will place examples that it found. These can be anywhere in the file system, -we'll define environment variables to make it easy to reference them. - -So, only for AFL you need to configure the outputs path: - -``` -mkdir outputs -export AFLOUT=$PWD/outputs -``` - -libFuzzer will use the input directory as output directory. - -## AFL - -### Building AFL - -It is recommended to always use the latest version of afl: -``` -wget http://lcamtuf.coredump.cx/afl/releases/afl-latest.tgz -tar -zxvf afl-latest.tgz -cd afl-<version> -make -export AFLPATH=$PWD -``` +# Fuzzing Bitcoin Core using libFuzzer + +## Quickstart guide + +To quickly get started fuzzing Bitcoin Core using [libFuzzer](https://llvm.org/docs/LibFuzzer.html): + +```sh +$ git clone https://github.com/bitcoin/bitcoin +$ cd bitcoin/ +$ ./autogen.sh +$ CC=clang CXX=clang++ ./configure --enable-fuzz --with-sanitizers=address,fuzzer,undefined +# macOS users: If you have problem with this step then make sure to read "macOS hints for +# libFuzzer" on https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md#macos-hints-for-libfuzzer +$ make +$ src/test/fuzz/process_message +# abort fuzzing using ctrl-c +``` + +## Fuzzing harnesses, fuzzing output and fuzzing corpora + +[`process_message`](https://github.com/bitcoin/bitcoin/blob/master/src/test/fuzz/process_message.cpp) is a fuzzing harness for the [`ProcessMessage(...)` function (`net_processing`)](https://github.com/bitcoin/bitcoin/blob/master/src/net_processing.cpp). The available fuzzing harnesses are found in [`src/test/fuzz/`](https://github.com/bitcoin/bitcoin/tree/master/src/test/fuzz). + +The fuzzer will output `NEW` every time it has created a test input that covers new areas of the code under test. For more information on how to interpret the fuzzer output, see the [libFuzzer documentation](https://llvm.org/docs/LibFuzzer.html). + +If you specify a corpus directory then any new coverage increasing inputs will be saved there: + +```sh +$ mkdir -p process_message-seeded-from-thin-air/ +$ src/test/fuzz/process_message process_message-seeded-from-thin-air/ +INFO: Seed: 840522292 +INFO: Loaded 1 modules (424174 inline 8-bit counters): 424174 [0x55e121ef9ab8, 0x55e121f613a6), +INFO: Loaded 1 PC tables (424174 PCs): 424174 [0x55e121f613a8,0x55e1225da288), +INFO: 0 files found in process_message-seeded-from-thin-air/ +INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes +INFO: A corpus is not provided, starting from an empty corpus +#2 INITED cov: 94 ft: 95 corp: 1/1b exec/s: 0 rss: 150Mb +#3 NEW cov: 95 ft: 96 corp: 2/3b lim: 4 exec/s: 0 rss: 150Mb L: 2/2 MS: 1 InsertByte- +#4 NEW cov: 96 ft: 98 corp: 3/7b lim: 4 exec/s: 0 rss: 150Mb L: 4/4 MS: 1 CrossOver- +#21 NEW cov: 96 ft: 100 corp: 4/11b lim: 4 exec/s: 0 rss: 150Mb L: 4/4 MS: 2 ChangeBit-CrossOver- +#324 NEW cov: 101 ft: 105 corp: 5/12b lim: 6 exec/s: 0 rss: 150Mb L: 6/6 MS: 5 CrossOver-ChangeBit-CopyPart-ChangeBit-ChangeBinInt- +#1239 REDUCE cov: 102 ft: 106 corp: 6/24b lim: 14 exec/s: 0 rss: 150Mb L: 13/13 MS: 5 ChangeBit-CrossOver-EraseBytes-ChangeBit-InsertRepeatedBytes- +#1272 REDUCE cov: 102 ft: 106 corp: 6/23b lim: 14 exec/s: 0 rss: 150Mb L: 12/12 MS: 3 ChangeBinInt-ChangeBit-EraseBytes- + NEW_FUNC[1/677]: 0x55e11f456690 in std::_Function_base::~_Function_base() /usr/lib/gcc/x86_64-linux-gnu/8/../../../../include/c++/8/bits/std_function.h:255 + NEW_FUNC[2/677]: 0x55e11f465800 in CDataStream::CDataStream(std::vector<unsigned char, std::allocator<unsigned char> > const&, int, int) src/./streams.h:248 +#2125 REDUCE cov: 4820 ft: 4867 corp: 7/29b lim: 21 exec/s: 0 rss: 155Mb L: 6/12 MS: 2 CopyPart-CMP- DE: "block"- + NEW_FUNC[1/9]: 0x55e11f64d790 in std::_Rb_tree<uint256, std::pair<uint256 const, std::chrono::duration<long, std::ratio<1l, 1000000l> > >, std::_Select1st<std::pair<uint256 const, std::chrono::duration<long, std::ratio<1l, 1000000l> > > >, std::less<uint256>, std::allocator<std::pair<uint256 const, std::chrono::duration<long, std::ratio<1l, 1000000l> > > > >::~_Rb_tree() /usr/lib/gcc/x86_64-linux-gnu/8/../../../../include/c++/8/bits/stl_tree.h:972 + NEW_FUNC[2/9]: 0x55e11f64d870 in std::_Rb_tree<uint256, std::pair<uint256 const, std::chrono::duration<long, std::ratio<1l, 1000000l> > >, std::_Select1st<std::pair<uint256 const, std::chrono::duration<long, std::ratio<1l, 1000000l> > > >, std::less<uint256>, std::allocator<std::pair<uint256 const, std::chrono::duration<long, std::ratio<1l, 1000000l> > > > >::_M_erase(std::_Rb_tree_node<std::pair<uint256 const, std::chrono::duration<long, std::ratio<1l, 1000000l> > > >*) /usr/lib/gcc/x86_64-linux-gnu/8/../../../../include/c++/8/bits/stl_tree.h:1875 +#2228 NEW cov: 4898 ft: 4971 corp: 8/35b lim: 21 exec/s: 0 rss: 156Mb L: 6/12 MS: 3 EraseBytes-CopyPart-PersAutoDict- DE: "block"- + NEW_FUNC[1/5]: 0x55e11f46df70 in std::enable_if<__and_<std::allocator_traits<zero_after_free_allocator<char> >::__construct_helper<char, unsigned char const&>::type>::value, void>::type std::allocator_traits<zero_after_free_allocator<char> >::_S_construct<char, unsigned char const&>(zero_after_free_allocator<char>&, char*, unsigned char const&) /usr/lib/gcc/x86_64-linux-gnu/8/../../../../include/c++/8/bits/alloc_traits.h:243 + NEW_FUNC[2/5]: 0x55e11f477390 in std::vector<unsigned char, std::allocator<unsigned char> >::data() /usr/lib/gcc/x86_64-linux-gnu/8/../../../../include/c++/8/bits/stl_vector.h:1056 +#2456 NEW cov: 4933 ft: 5042 corp: 9/55b lim: 21 exec/s: 0 rss: 160Mb L: 20/20 MS: 3 ChangeByte-InsertRepeatedBytes-PersAutoDict- DE: "block"- +#2467 NEW cov: 4933 ft: 5043 corp: 10/76b lim: 21 exec/s: 0 rss: 161Mb L: 21/21 MS: 1 InsertByte- +#4215 NEW cov: 4941 ft: 5129 corp: 17/205b lim: 29 exec/s: 4215 rss: 350Mb L: 29/29 MS: 5 InsertByte-ChangeBit-CopyPart-InsertRepeatedBytes-CrossOver- +#4567 REDUCE cov: 4941 ft: 5129 corp: 17/204b lim: 29 exec/s: 4567 rss: 404Mb L: 24/29 MS: 2 ChangeByte-EraseBytes- +#6642 NEW cov: 4941 ft: 5138 corp: 18/244b lim: 43 exec/s: 2214 rss: 450Mb L: 43/43 MS: 3 CopyPart-CMP-CrossOver- DE: "verack"- +# abort fuzzing using ctrl-c +$ ls process_message-seeded-from-thin-air/ +349ac589fc66a09abc0b72bb4ae445a7a19e2cd8 4df479f1f421f2ea64b383cd4919a272604087a7 +a640312c98dcc55d6744730c33e41c5168c55f09 b135de16e4709558c0797c15f86046d31c5d86d7 +c000f7b41b05139de8b63f4cbf7d1ad4c6e2aa7f fc52cc00ec1eb1c08470e69f809ae4993fa70082 +$ cat --show-nonprinting process_message-seeded-from-thin-air/349ac589fc66a09abc0b72bb4ae445a7a19e2cd8 +block^@M-^?M-^?M-^?M-^?M-^?nM-^?M-^? +``` + +In this case the fuzzer managed to create a `block` message which when passed to `ProcessMessage(...)` increased coverage. + +The project's collection of seed corpora is found in the [`bitcoin-core/qa-assets`](https://github.com/bitcoin-core/qa-assets) repo. + +To fuzz `process_message` using the [`bitcoin-core/qa-assets`](https://github.com/bitcoin-core/qa-assets) seed corpus: + +```sh +$ git clone https://github.com/bitcoin-core/qa-assets +$ src/test/fuzz/process_message qa-assets/fuzz_seed_corpus/process_message/ +INFO: Seed: 1346407872 +INFO: Loaded 1 modules (424174 inline 8-bit counters): 424174 [0x55d8a9004ab8, 0x55d8a906c3a6), +INFO: Loaded 1 PC tables (424174 PCs): 424174 [0x55d8a906c3a8,0x55d8a96e5288), +INFO: 991 files found in qa-assets/fuzz_seed_corpus/process_message/ +INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes +INFO: seed corpus: files: 991 min: 1b max: 1858b total: 288291b rss: 150Mb +#993 INITED cov: 7063 ft: 8236 corp: 25/3821b exec/s: 0 rss: 181Mb +… +``` + +If you find coverage increasing inputs when fuzzing you are highly encouraged to submit them for inclusion in the [`bitcoin-core/qa-assets`](https://github.com/bitcoin-core/qa-assets) repo. + +Every single pull request submitted against the Bitcoin Core repo is automatically tested against all inputs in the [`bitcoin-core/qa-assets`](https://github.com/bitcoin-core/qa-assets) repo. Contributing new coverage increasing inputs is an easy way to help make Bitcoin Core more robust. + +## macOS hints for libFuzzer + +The default Clang/LLVM version supplied by Apple on macOS does not include +fuzzing libraries, so macOS users will need to install a full version, for +example using `brew install llvm`. -For macOS you may need to ignore x86 compilation checks when running `make`: -`AFL_NO_X86=1 make`. +Should you run into problems with the address sanitizer, it is possible you +may need to run `./configure` with `--disable-asm` to avoid errors +with certain assembly code from Bitcoin Core's code. See [developer notes on sanitizers](https://github.com/bitcoin/bitcoin/blob/master/doc/developer-notes.md#sanitizers) +for more information. -### Instrumentation +You may also need to take care of giving the correct path for `clang` and +`clang++`, like `CC=/path/to/clang CXX=/path/to/clang++` if the non-systems +`clang` does not come first in your path. -To build Bitcoin Core using AFL instrumentation (this assumes that the -`AFLPATH` was set as above): -``` -./configure --disable-ccache --disable-shared --enable-tests --enable-fuzz CC=${AFLPATH}/afl-gcc CXX=${AFLPATH}/afl-g++ -export AFL_HARDEN=1 -make -``` +Full configure that was tested on macOS Catalina with `brew` installed `llvm`: -If you are using clang you will need to substitute `afl-gcc` with `afl-clang` -and `afl-g++` with `afl-clang++`, so the first line above becomes: -``` -./configure --disable-ccache --disable-shared --enable-tests --enable-fuzz CC=${AFLPATH}/afl-clang CXX=${AFLPATH}/afl-clang++ +```sh +./configure --enable-fuzz --with-sanitizers=fuzzer,address,undefined CC=/usr/local/opt/llvm/bin/clang CXX=/usr/local/opt/llvm/bin/clang++ --disable-asm ``` -We disable ccache because we don't want to pollute the ccache with instrumented -objects, and similarly don't want to use non-instrumented cached objects linked -in. +Read the [libFuzzer documentation](https://llvm.org/docs/LibFuzzer.html) for more information. This [libFuzzer tutorial](https://github.com/google/fuzzing/blob/master/tutorial/libFuzzerTutorial.md) might also be of interest. -The fuzzing can be sped up significantly (~200x) by using `afl-clang-fast` and -`afl-clang-fast++` in place of `afl-gcc` and `afl-g++` when compiling. When -compiling using `afl-clang-fast`/`afl-clang-fast++` the resulting -binary will be instrumented in such a way that the AFL -features "persistent mode" and "deferred forkserver" can be used. See -https://github.com/google/AFL/tree/master/llvm_mode for details. +# Fuzzing Bitcoin Core using american fuzzy lop (`afl-fuzz`) -### Fuzzing +## Quickstart guide -To start the actual fuzzing use: +To quickly get started fuzzing Bitcoin Core using [`afl-fuzz`](https://github.com/google/afl): -``` -export FUZZ_TARGET=bech32 # Pick a fuzz_target -mkdir ${AFLOUT}/${FUZZ_TARGET} -$AFLPATH/afl-fuzz -i ${DIR_FUZZ_IN}/${FUZZ_TARGET} -o ${AFLOUT}/${FUZZ_TARGET} -m52 -- src/test/fuzz/${FUZZ_TARGET} +```sh +$ git clone https://github.com/bitcoin/bitcoin +$ cd bitcoin/ +$ git clone https://github.com/google/afl +$ make -C afl/ +$ make -C afl/llvm_mode/ +$ ./autogen.sh +$ CC=$(pwd)/afl/afl-clang-fast CXX=$(pwd)/afl/afl-clang-fast++ ./configure --enable-fuzz +$ make +# For macOS you may need to ignore x86 compilation checks when running "make". If so, +# try compiling using: AFL_NO_X86=1 make +$ mkdir -p inputs/ outputs/ +$ echo A > inputs/thin-air-input +$ afl/afl-fuzz -i inputs/ -o outputs/ -- src/test/fuzz/bech32 +# You may have to change a few kernel parameters to test optimally - afl-fuzz +# will print an error and suggestion if so. ``` -You may have to change a few kernel parameters to test optimally - `afl-fuzz` -will print an error and suggestion if so. +Read the [`afl-fuzz` documentation](https://github.com/google/afl) for more information. -On macOS you may need to set `AFL_NO_FORKSRV=1` to get the target to run. -``` -export FUZZ_TARGET=bech32 # Pick a fuzz_target -mkdir ${AFLOUT}/${FUZZ_TARGET} -AFL_NO_FORKSRV=1 $AFLPATH/afl-fuzz -i ${DIR_FUZZ_IN}/${FUZZ_TARGET} -o ${AFLOUT}/${FUZZ_TARGET} -m52 -- src/test/fuzz/${FUZZ_TARGET} -``` - -## libFuzzer +# Fuzzing Bitcoin Core using Honggfuzz -A recent version of `clang`, the address/undefined sanitizers (ASan/UBSan) and -libFuzzer is needed (all found in the `compiler-rt` runtime libraries package). +## Quickstart guide -To build all fuzz targets with libFuzzer, run +To quickly get started fuzzing Bitcoin Core using [Honggfuzz](https://github.com/google/honggfuzz): +```sh +$ git clone https://github.com/bitcoin/bitcoin +$ cd bitcoin/ +$ ./autogen.sh +$ git clone https://github.com/google/honggfuzz +$ cd honggfuzz/ +$ make +$ cd .. +$ CC=$(pwd)/honggfuzz/hfuzz_cc/hfuzz-clang CXX=$(pwd)/honggfuzz/hfuzz_cc/hfuzz-clang++ ./configure --enable-fuzz --with-sanitizers=address,undefined +$ make +$ mkdir -p inputs/ +$ honggfuzz/honggfuzz -i inputs/ -- src/test/fuzz/process_message ``` -./configure --disable-ccache --enable-fuzz --with-sanitizers=fuzzer,address,undefined CC=clang CXX=clang++ -make -``` - -See https://llvm.org/docs/LibFuzzer.html#running on how to run the libFuzzer -instrumented executable. - -Alternatively, you can run the script through the fuzzing test harness (only -libFuzzer supported so far). You need to pass it the inputs directory and -the specific test target you want to run. - -``` -./test/fuzz/test_runner.py ${DIR_FUZZ_IN} bech32 -``` - -### macOS hints for libFuzzer -The default clang/llvm version supplied by Apple on macOS does not include -fuzzing libraries, so macOS users will need to install a full version, for -example using `brew install llvm`. - -Should you run into problems with the address sanitizer, it is possible you -may need to run `./configure` with `--disable-asm` to avoid errors -with certain assembly code from Bitcoin Core's code. See [developer notes on sanitizers](https://github.com/bitcoin/bitcoin/blob/master/doc/developer-notes.md#sanitizers) -for more information. - -You may also need to take care of giving the correct path for clang and -clang++, like `CC=/path/to/clang CXX=/path/to/clang++` if the non-systems -clang does not come first in your path. - -Full configure that was tested on macOS Catalina with `brew` installed `llvm`: -``` -./configure --disable-ccache --enable-fuzz --with-sanitizers=fuzzer,address,undefined CC=/usr/local/opt/llvm/bin/clang CXX=/usr/local/opt/llvm/bin/clang++ --disable-asm -``` +Read the [Honggfuzz documentation](https://github.com/google/honggfuzz/blob/master/docs/USAGE.md) for more information. diff --git a/doc/init.md b/doc/init.md index 0b327aab58..99aa0a0def 100644 --- a/doc/init.md +++ b/doc/init.md @@ -20,9 +20,9 @@ The macOS configuration assumes bitcoind will be set up for the current user. Configuration --------------------------------- -At a bare minimum, bitcoind requires that the rpcpassword setting be set -when running as a daemon. If the configuration file does not exist or this -setting is not set, bitcoind will shut down promptly after startup. +Running bitcoind as a daemon does not require any manual configuration. You may +set the `rpcauth` setting in the `bitcoin.conf` configuration file to override +the default behaviour of using a special cookie for authentication. This password does not have to be remembered or typed as it is mostly used as a fixed token that bitcoind and client programs read from the configuration diff --git a/doc/multiprocess.md b/doc/multiprocess.md new file mode 100644 index 0000000000..471d8561f7 --- /dev/null +++ b/doc/multiprocess.md @@ -0,0 +1,35 @@ +# Multiprocess Bitcoin + +On unix systems, the `--enable-multiprocess` build option can be passed to `./configure` to build new `bitcoin-node`, `bitcoin-wallet`, and `bitcoin-gui` executables alongside existing `bitcoind` and `bitcoin-qt` executables. + +`bitcoin-node` is a drop-in replacement for `bitcoind`, and `bitcoin-gui` is a drop-in replacement for `bitcoin-qt`, and there are no differences in use or external behavior between the new and old executables. But internally (after [#10102](https://github.com/bitcoin/bitcoin/pull/10102)), `bitcoin-gui` will spawn a `bitcoin-node` process to run P2P and RPC code, communicating with it across a socket pair, and `bitcoin-node` will spawn `bitcoin-wallet` to run wallet code, also communicating over a socket pair. This will let node, wallet, and GUI code run in separate address spaces for better isolation, and allow future improvements like being able to start and stop components independently on different machines and environments. + +## Next steps + +Specific next steps after [#10102](https://github.com/bitcoin/bitcoin/pull/10102) will be: + +- [ ] Adding `-ipcbind` and `-ipcconnect` options to `bitcoin-node`, `bitcoin-wallet`, and `bitcoin-gui` executables so they can listen and connect to TCP ports and unix socket paths. This will allow separate processes to be started and stopped any time and connect to each other. +- [ ] Adding `-server` and `-rpcbind` options to the `bitcoin-wallet` executable so wallet processes can handle RPC requests directly without going through the node. +- [ ] Supporting windows, not just unix systems. The existing socket code is already cross-platform, so the only windows-specific code that needs to be written is code spawning a process and passing a socket descriptor. This can be implemented with `CreateProcess` and `WSADuplicateSocket`. Example: https://memset.wordpress.com/2010/10/13/win32-api-passing-socket-with-ipc-method/. +- [ ] Adding sandbox features, restricting subprocess access to resources and data. See [https://eklitzke.org/multiprocess-bitcoin](https://eklitzke.org/multiprocess-bitcoin). + +## Debugging + +After [#10102](https://github.com/bitcoin/bitcoin/pull/10102), the `-debug=ipc` command line option can be used to see requests and responses between processes. + +## Installation + +The multiprocess feature requires [Cap'n Proto](https://capnproto.org/) and [libmultiprocess](https://github.com/chaincodelabs/libmultiprocess) as dependencies. A simple way to get starting using it without installing these dependencies manually is to use the [depends system](../depends) with the `MULTIPROCESS=1` [dependency option](../depends#dependency-options) passed to make: + +``` +cd <BITCOIN_SOURCE_DIRECTORY> +make -C depends NO_QT=1 MULTIPROCESS=1 +./configure --prefix=$PWD/depends/x86_64-pc-linux-gnu +make +src/bitcoin-node -regtest -printtoconsole -debug=ipc +BITCOIND=bitcoin-node test/functional/test_runner.py +``` + +The configure script will pick up settings and library locations from the depends directory, so there is no need to pass `--enable-multiprocess` as a separate flag when using the depends system (it's controlled by the `MULTIPROCESS=1` option). + +Alternately, you can install [Cap'n Proto](https://capnproto.org/) and [libmultiprocess](https://github.com/chaincodelabs/libmultiprocess) packages on your system, and just run `./configure --enable-multiprocess` without using the depends system. The configure script will be able to locate the installed packages via [pkg-config](https://www.freedesktop.org/wiki/Software/pkg-config/). See [Installation](https://github.com/chaincodelabs/libmultiprocess#installation) section of the libmultiprocess readme for install steps. See [build-unix.md](build-unix.md) and [build-osx.md](build-osx.md) for information about installing dependencies in general. diff --git a/doc/productivity.md b/doc/productivity.md index b25ddc94e5..555f0afe3c 100644 --- a/doc/productivity.md +++ b/doc/productivity.md @@ -12,7 +12,7 @@ Table of Contents * [Multiple working directories with `git worktrees`](#multiple-working-directories-with-git-worktrees) * [Interactive "dummy rebases" for fixups and execs with `git merge-base`](#interactive-dummy-rebases-for-fixups-and-execs-with-git-merge-base) * [Writing code](#writing-code) - * [Format C/C++/Protobuf diffs with `clang-format-diff.py`](#format-ccprotobuf-diffs-with-clang-format-diffpy) + * [Format C/C++ diffs with `clang-format-diff.py`](#format-cc-diffs-with-clang-format-diffpy) * [Format Python diffs with `yapf-diff.py`](#format-python-diffs-with-yapf-diffpy) * [Rebasing/Merging code](#rebasingmerging-code) * [More conflict context with `merge.conflictstyle diff3`](#more-conflict-context-with-mergeconflictstyle-diff3) @@ -118,13 +118,13 @@ You can also set up [upstream refspecs](#reference-prs-easily-with-refspecs) to Writing code ------------ -### Format C/C++/Protobuf diffs with `clang-format-diff.py` +### Format C/C++ diffs with `clang-format-diff.py` See [contrib/devtools/README.md](/contrib/devtools/README.md#clang-format-diff.py). ### Format Python diffs with `yapf-diff.py` -Usage is exactly the same as [`clang-format-diff.py`](#format-ccprotobuf-diffs-with-clang-format-diffpy). You can get it [here](https://github.com/MarcoFalke/yapf-diff). +Usage is exactly the same as [`clang-format-diff.py`](#format-cc-diffs-with-clang-format-diffpy). You can get it [here](https://github.com/MarcoFalke/yapf-diff). Rebasing/Merging code ------------- @@ -172,11 +172,11 @@ When looking at other's pull requests, it may make sense to add the following se ``` [remote "upstream-pull"] - fetch = +refs/pull/*:refs/remotes/upstream-pull/* + fetch = +refs/pull/*/head:refs/remotes/upstream-pull/* url = git@github.com:bitcoin/bitcoin.git ``` -This will add an `upstream-pull` remote to your git repository, which can be fetched using `git fetch --all` or `git fetch upstream-pull`. Afterwards, you can use `upstream-pull/NUMBER/head` in arguments to `git show`, `git checkout` and anywhere a commit id would be acceptable to see the changes from pull request NUMBER. +This will add an `upstream-pull` remote to your git repository, which can be fetched using `git fetch --all` or `git fetch upstream-pull`. It will download and store on disk quite a lot of data (all PRs, including merged and closed ones). Afterwards, you can use `upstream-pull/NUMBER/head` in arguments to `git show`, `git checkout` and anywhere a commit id would be acceptable to see the changes from pull request NUMBER. ### Diff the diffs with `git range-diff` diff --git a/doc/rapidcheck.md b/doc/rapidcheck.md deleted file mode 100644 index 397a907f17..0000000000 --- a/doc/rapidcheck.md +++ /dev/null @@ -1,84 +0,0 @@ -# RapidCheck property-based testing for Bitcoin Core - -## Concept - -Property-based testing is experimentally being added to Bitcoin Core with -[RapidCheck](https://github.com/emil-e/rapidcheck), a C++ framework for -property-based testing inspired by the Haskell library -[QuickCheck](https://hackage.haskell.org/package/QuickCheck). - -RapidCheck performs random testing of program properties. A specification of the -program is given in the form of properties which functions should satisfy, and -RapidCheck tests that the properties hold in a large number of randomly -generated cases. - -If an exception is found, RapidCheck tries to find the smallest case, for some -definition of smallest, for which the property is still false and displays it as -a counter-example. For example, if the input is an integer, RapidCheck tries to -find the smallest integer for which the property is false. - -## Running - -If RapidCheck is installed, Bitcoin Core will automatically run the -property-based tests with the unit tests during `make check`, unless the -`--without-rapidcheck` flag is passed when configuring. - -For more information, run `./configure --help` and see `--with-rapidcheck` under -Optional Packages. - -## Setup - -The following instructions have been tested with Linux Debian and macOS. - -1. Clone the RapidCheck source code and cd into the repository. - - ```shell - git clone https://github.com/emil-e/rapidcheck.git - cd rapidcheck - ``` - -2. Build RapidCheck (requires CMake to be installed). - - ```shell - cmake -DCMAKE_POSITION_INDEPENDENT_CODE:BOOL=true -DRC_ENABLE_BOOST_TEST=ON $(pwd) - make && make install - ``` - -3. Configure Bitcoin Core with RapidCheck. - - `cd` to the directory of your local bitcoin repository and run - `./configure`. In the output you should see: - - ```shell - checking rapidcheck.h usability... yes - checking rapidcheck.h presence... yes - checking for rapidcheck.h... yes - [...] - Options used to compile and link: - [...] - with test = yes - with prop = yes - ``` - -4. Build Bitcoin Core with RapidCheck. - - Now you can run `make` and should see the property-based tests compiled with - the unit tests: - - ```shell - Making all in src - [...] - CXX test/gen/test_bitcoin-crypto_gen.o - CXX test/test_bitcoin-key_properties.o - ``` - -5. Run the unit tests with `make check`. The property-based tests will be run - with the unit tests. - - ```shell - Running tests: crypto_tests from test/crypto_tests.cpp - [...] - Running tests: key_properties from test/key_properties.cpp - ``` - -That's it! You are now running property-based tests in Bitcoin Core. diff --git a/doc/reduce-memory.md b/doc/reduce-memory.md index b0faf0825a..6e7a578ecc 100644 --- a/doc/reduce-memory.md +++ b/doc/reduce-memory.md @@ -24,8 +24,9 @@ The size of some in-memory caches can be reduced. As caches trade off memory usa ## Number of peers -- `-maxconnections=<n>` - the maximum number of connections, this defaults to `125`. Each active connection takes up some memory. Only significant if incoming - connections are enabled, otherwise the number of connections will never be more than `8`. +- `-maxconnections=<n>` - the maximum number of connections, this defaults to 125. Each active connection takes up some + memory. This option applies only if incoming connections are enabled, otherwise the number of connections will never + be more than 10. Of the 10 outbound peers, there can be 8 full-relay connections and 2 block-relay-only ones. ## Thread configuration diff --git a/doc/reduce-traffic.md b/doc/reduce-traffic.md index 5a71f62e0f..ce77a00dd5 100644 --- a/doc/reduce-traffic.md +++ b/doc/reduce-traffic.md @@ -3,8 +3,10 @@ Reduce Traffic Some node operators need to deal with bandwidth caps imposed by their ISPs. -By default, Bitcoin Core allows up to 125 connections to different peers, 8 of -which are outbound. You can therefore, have at most 117 inbound connections. +By default, Bitcoin Core allows up to 125 connections to different peers, 10 of +which are outbound. You can therefore, have at most 115 inbound connections. +Of the 10 outbound peers, there can be 8 full-relay connections and 2 +block-relay-only ones. The default settings can result in relatively significant traffic consumption. @@ -21,12 +23,12 @@ longer serving historic blocks (blocks older than one week). Keep in mind that new nodes require other nodes that are willing to serve historic blocks. -Whitelisted peers will never be disconnected, although their traffic counts for +Peers with the `noban` permission will never be disconnected, although their traffic counts for calculating the target. ## 2. Disable "listening" (`-listen=0`) -Disabling listening will result in fewer nodes connected (remember the maximum of 8 +Disabling listening will result in fewer nodes connected (remember the maximum of 10 outbound peers). Fewer nodes will result in less traffic usage as you are relaying blocks and transactions to fewer nodes. @@ -44,7 +46,11 @@ with other peers, you can disable transaction relay. Be reminded of the effects of this setting. - Fee estimation will no longer work. -- Not relaying other's transactions could hurt your privacy if used while a - wallet is loaded or if you use the node to broadcast transactions. +- It sets the flag "-walletbroadcast" to be "0", only if it is currently unset. + Doing so disables the automatic broadcasting of transactions from wallet. Not + relaying other's transactions could hurt your privacy if used while a wallet + is loaded or if you use the node to broadcast transactions. +- If a peer is whitelisted and "-whitelistforcerelay" is set to "1" (which will + also set "whitelistrelay" to "1"), we will still receive and relay their transactions. - It makes block propagation slower because compact block relay can only be used when transaction relay is enabled. diff --git a/doc/release-notes-16528.md b/doc/release-notes-16528.md new file mode 100644 index 0000000000..e69de29bb2 --- /dev/null +++ b/doc/release-notes-16528.md diff --git a/doc/release-notes-18918.md b/doc/release-notes-18918.md new file mode 100644 index 0000000000..e69de29bb2 --- /dev/null +++ b/doc/release-notes-18918.md diff --git a/doc/release-notes.md b/doc/release-notes.md index 22d5767b7b..d9d0ecd631 100644 --- a/doc/release-notes.md +++ b/doc/release-notes.md @@ -34,173 +34,226 @@ How to Upgrade ============== If you are running an older version, shut it down. Wait until it has completely -shut down (which might take a few minutes for older versions), then run the +shut down (which might take a few minutes in some cases), then run the installer (on Windows) or just copy over `/Applications/Bitcoin-Qt` (on Mac) or `bitcoind`/`bitcoin-qt` (on Linux). Upgrading directly from a version of Bitcoin Core that has reached its EOL is -possible, but it might take some time if the datadir needs to be migrated. Old +possible, but it might take some time if the data directory needs to be migrated. Old wallet versions of Bitcoin Core are generally supported. Compatibility ============== -Bitcoin Core is supported and extensively tested on operating systems using -the Linux kernel, macOS 10.12+, and Windows 7 and newer. It is not recommended -to use Bitcoin Core on unsupported systems. - -Bitcoin Core should also work on most other Unix-like systems but is not -as frequently tested on them. +Bitcoin Core is supported and extensively tested on operating systems +using the Linux kernel, macOS 10.12+, and Windows 7 and newer. Bitcoin +Core should also work on most other Unix-like systems but is not as +frequently tested on them. It is not recommended to use Bitcoin Core on +unsupported systems. From Bitcoin Core 0.20.0 onwards, macOS versions earlier than 10.12 are no longer supported. Additionally, Bitcoin Core does not yet change appearance when macOS "dark mode" is activated. -In addition to previously supported CPU platforms, this release's pre-compiled -distribution provides binaries for the RISC-V platform. - Notable changes =============== P2P and network changes ----------------------- -#### Removal of reject network messages from Bitcoin Core (BIP61) - -The command line option to enable BIP61 (`-enablebip61`) has been removed. - -This feature has been disabled by default since Bitcoin Core version 0.18.0. -Nodes on the network can not generally be trusted to send valid ("reject") -messages, so this should only ever be used when connected to a trusted node. -Please use the recommended alternatives if you rely on this deprecated feature: - -* Testing or debugging of implementations of the Bitcoin P2P network protocol - should be done by inspecting the log messages that are produced by a recent - version of Bitcoin Core. Bitcoin Core logs debug messages - (`-debug=<category>`) to a stream (`-printtoconsole`) or to a file - (`-debuglogfile=<debug.log>`). - -* Testing the validity of a block can be achieved by specific RPCs: - - `submitblock` - - `getblocktemplate` with `'mode'` set to `'proposal'` for blocks with - potentially invalid POW - -* Testing the validity of a transaction can be achieved by specific RPCs: - - `sendrawtransaction` - - `testmempoolaccept` - -* Wallets should not use the absence of "reject" messages to indicate a - transaction has propagated the network, nor should wallets use "reject" - messages to set transaction fees. Wallets should rather use fee estimation - to determine transaction fees and set replace-by-fee if desired. Thus, they - could wait until the transaction has confirmed (taking into account the fee - target they set (compare the RPC `estimatesmartfee`)) or listen for the - transaction announcement by other network peers to check for propagation. - -The removal of BIP61 REJECT message support also has the following minor RPC -and logging implications: - -* `testmempoolaccept` and `sendrawtransaction` no longer return the P2P REJECT - code when a transaction is not accepted to the mempool. They still return the - verbal reject reason. - -* Log messages that previously reported the REJECT code when a transaction was - not accepted to the mempool now no longer report the REJECT code. The reason - for rejection is still reported. +- The mempool now tracks whether transactions submitted via the wallet or RPCs + have been successfully broadcast. Every 10-15 minutes, the node will try to + announce unbroadcast transactions until a peer requests it via a `getdata` + message or the transaction is removed from the mempool for other reasons. + The node will not track the broadcast status of transactions submitted to the + node using P2P relay. This version reduces the initial broadcast guarantees + for wallet transactions submitted via P2P to a node running the wallet. (#18038) Updated RPCs ------------ -- `testmempoolaccept` and `sendrawtransaction` no longer return the P2P REJECT - code when a transaction is not accepted to the mempool. See the Section - _Removal of reject network messages from Bitcoin Core (BIP61)_ for details on - the removal of BIP61 REJECT message support. - -- A new descriptor type `sortedmulti(...)` has been added to support multisig scripts where the public keys are sorted lexicographically in the resulting script. - -- `walletprocesspsbt` and `walletcreatefundedpsbt` now include BIP 32 derivation paths by default for public keys if we know them. This can be disabled by setting `bip32derivs` to `false`. +- `getmempoolinfo` now returns an additional `unbroadcastcount` field. The + mempool tracks locally submitted transactions until their initial broadcast + is acknowledged by a peer. This field returns the count of transactions + waiting for acknowledgement. -Build System ------------- +- Mempool RPCs such as `getmempoolentry` and `getrawmempool` with + `verbose=true` now return an additional `unbroadcast` field. This indicates + whether initial broadcast of the transaction has been acknowledged by a + peer. `getmempoolancestors` and `getmempooldescendants` are also updated. -- OpenSSL is no longer used by Bitcoin Core. The last usage of the library -was removed in #17265. -- glibc 2.17 or greater is now required to run the release binaries. This -retains compatibility with RHEL 7, CentOS 7, Debian 8 and Ubuntu 14.04 LTS. -Further details can be found in #17538. +Changes to Wallet or GUI related RPCs can be found in the GUI or Wallet section below. New RPCs -------- -New settings +Build System ------------ -- RPC Whitelist system. It can give certain RPC users permissions to only some RPC calls. -It can be set with two command line arguments (`rpcwhitelist` and `rpcwhitelistdefault`). (#12763) - -- A new `-asmap` configuration option has been added to enable IP-to-ASN mapping - for bucketing of the network peers to diversify the network connections. The - legacy /16 prefix mapping remains the default. See [issue - #16599](https://github.com/bitcoin/bitcoin/issues/16599), [PR - #16702](https://github.com/bitcoin/bitcoin/pull/16702), and the `bitcoind - help` for more information. This option is experimental and subject to changes - or removal in future releases. - Updated settings ---------------- -Importing blocks upon startup via the `bootstrap.dat` file no longer occurs by default. The file must now be specified with `-loadblock=<file>`. - -- The `-debug=db` logging category has been renamed to `-debug=walletdb`, to distinguish it from `coindb`. - `-debug=db` has been deprecated and will be removed in the next major release. +- The `-debug=db` logging category, which was deprecated in 0.20 and replaced by + `-debug=walletdb` to distinguish it from `coindb`, has been removed. (#19202) -GUI changes ------------ +Changes to Wallet or GUI related settings can be found in the GUI or Wallet section below. -- The "Start Bitcoin Core on system login" option has been removed on macOS. +New settings +------------ Wallet ------ -- The wallet now by default uses bech32 addresses when using RPC, and creates native segwit change outputs. -- The way that output trust was computed has been fixed in #16766, which impacts confirmed/unconfirmed balance status and coin selection. - -- The RPC gettransaction, listtransactions and listsinceblock responses now also -includes the height of the block that contains the wallet transaction, if any. - -- RPC `getaddressinfo` changes: +- To improve wallet privacy, the frequency of wallet rebroadcast attempts is + reduced from approximately once every 15 minutes to once every 12-36 hours. + To maintain a similar level of guarantee for initial broadcast of wallet + transactions, the mempool tracks these transactions as a part of the newly + introduced unbroadcast set. See the "P2P and network changes" section for + more information on the unbroadcast set. (#18038) + +- The wallet can create a transaction without change even when the keypool is + empty. Previously it failed. (#17219) + +- The `-salvagewallet` startup option has been removed. A new `salvage` command + has been added to the `bitcoin-wallet` tool which performs the salvage + operations that `-salvagewallet` did. (#18918) + +### Experimental Descriptor Wallets + +Please note that Descriptor Wallets are still experimental and not all expected functionality +is available. Additionally there may be some bugs and current functions may change in the future. +Bugs and missing functionality can be reported to the [issue tracker](https://github.com/bitcoin/bitcoin/issues). + +0.21 introduces a new type of wallet - Descriptor Wallets. Descriptor Wallets store +scriptPubKey information using descriptors. This is in contrast to the Legacy Wallet +structure where keys are used to generate scriptPubKeys and addresses. Because of this +shift to being script based instead of key based, many of the confusing things that Legacy +Wallets do are not possible with Descriptor Wallets. Descriptor Wallets use a definition +of "mine" for scripts which is simpler and more intuitive than that used by Legacy Wallets. +Descriptor Wallets also uses different semantics for watch-only things and imports. + +As Descriptor Wallets are a new type of wallet, their introduction does not affect existing wallets. +Users who already have a Bitcoin Core wallet can continue to use it as they did before without +any change in behavior. Newly created Legacy Wallets (which is the default type of wallet) will +behave as they did in previous versions of Bitcoin Core. + +The differences between Descriptor Wallets and Legacy Wallets are largely limited to non user facing +things. They are intended to behave similarly except for the import/export and watchonly functionality +as described below. + +#### Creating Descriptor Wallets + +Descriptor Wallets are not created by default. They must be explicitly created using the +`createwallet` RPC or via the GUI. A `descriptors` option has been added to `createwallet`. +Setting `descriptors` to `true` will create a Descriptor Wallet instead of a Legacy Wallet. + +In the GUI, a checkbox has been added to the Create Wallet Dialog to indicate that a +Descriptor Wallet should be created. + +Without those options being set, a Legacy Wallet will be created instead. Additionally the +Default Wallet created upon first startup of Bitcoin Core will be a Legacy Wallet. + +#### `IsMine` Semantics + +`IsMine` refers to the function used to determine whether a script belongs to the wallet. +This is used to determine whether an output belongs to the wallet. `IsMine` in Legacy Wallets +returns true if the wallet would be able to sign an input that spends an output with that script. +Since keys can be involved in a variety of different scripts, this definition for `IsMine` can +lead to many unexpected scripts being considered part of the wallet. + +With Descriptor Wallets, descriptors explicitly specify the set of scripts that are owned by +the wallet. Since descriptors are deterministic and easily enumerable, users will know exactly +what scripts the wallet will consider to belong to it. Additionally the implementation of `IsMine` +in Descriptor Wallets is far simpler than for Legacy Wallets. Notably, in Legacy Wallets, `IsMine` +allowed for users to take one type of address (e.g. P2PKH), mutate it into another address type +(e.g. P2WPKH), and the wallet would still detect outputs sending to the new address type +even without that address being requested from the wallet. Descriptor Wallets does not +allow for this and will only watch for the addresses that were explicitly requested from the wallet. + +These changes to `IsMine` will make it easier to reason about what scripts the wallet will +actually be watching for in outputs. However for the vast majority of users, this change is +largely transparent and will not have noticeable effect. + +#### Imports and Exports + +In Legacy Wallets, raw scripts and keys could be imported to the wallet. Those imported scripts +and keys are treated separately from the keys generated by the wallet. This complicates the `IsMine` +logic as it has to distinguish between spendable and watchonly. + +Descriptor Wallets handle importing scripts and keys differently. Only complete descriptors can be +imported. These descriptors are then added to the wallet as if it were a descriptor generated by +the wallet itself. This simplifies the `IsMine` logic so that it no longer has to distinguish +between spendable and watchonly. As such, the watchonly model for Descriptor Wallets is also +different and described in more detail in the next section. + +To import into a Descriptor Wallet, a new `importdescriptors` RPC has been added that uses a syntax +similar to that of `importmulti`. + +As Legacy Wallets and Descriptor Wallets use different mechanisms for storing and importing scripts and keys +the existing import RPCs have been disabled for descriptor wallets. +New export RPCs for Descriptor Wallets have not yet been added. + +The following RPCs are disabled for Descriptor Wallets: + +* importprivkey +* importpubkey +* importaddress +* importwallet +* dumpprivkey +* dumpwallet +* importmulti +* addmultisigaddress +* sethdseed + +#### Watchonly Wallets + +A Legacy Wallet contains both private keys and scripts that were being watched. +Those watched scripts would not contribute to your normal balance. In order to see the watchonly +balance and to use watchonly things in transactions, an `include_watchonly` option was added +to many RPCs that would allow users to do that. However it is easy to forget to include this option. + +Descriptor Wallets move to a per-wallet watchonly model. Instead an entire wallet is considered to be +watchonly depending on whether it was created with private keys disabled. This eliminates the need +to distinguish between things that are watchonly and things that are not within a wallet itself. + +This change does have a caveat. If a Descriptor Wallet with private keys *enabled* has +a multiple key descriptor without all of the private keys (e.g. `multi(...)` with only one private key), +then the wallet will fail to sign and broadcast transactions. Such wallets would need to use the PSBT +workflow but the typical GUI Send, `sendtoaddress`, etc. workflows would still be available, just +non-functional. + +This issue is worsened if the wallet contains both single key (e.g. `wpkh(...)`) descriptors and such +multiple key descriptors as some transactions could be signed and broadast and others not. This is +due to some transactions containing only single key inputs, while others would contain both single +key and multiple key inputs, depending on which are available and how the coin selection algorithm +selects inputs. However this is not considered to be a supported use case; multisigs +should be in their own wallets which do not already have descriptors. Although users cannot export +descriptors with private keys for now as explained earlier. + +#### BIP 44/49/84 Support + +The change to using descriptors changes the default derivation paths used by Bitcoin Core +to adhere to BIP 44/49/84. Descriptors with different derivation paths can be imported without +issue. + +### Wallet RPC changes + +- The `upgradewallet` RPC replaces the `-upgradewallet` command line option. + (#15761) +- The `settxfee` RPC will fail if the fee was set higher than the `-maxtxfee` + command line setting. The wallet will already fail to create transactions + with fees higher than `-maxtxfee`. (#18467) - - the `label` field has been deprecated in favor of the `labels` field and - will be removed in 0.21. It can be re-enabled in the interim by launching - with `-deprecatedrpc=label`. - - - the `labels` behavior of returning an array of JSON objects containing name - and purpose key/value pairs has been deprecated in favor of an array of - label names and will be removed in 0.21. The previous behavior can be - re-enabled in the interim by launching with `-deprecatedrpc=labelspurpose`. +GUI changes +----------- Low-level changes ================= -Command line ------------- - -Command line options prefixed with main/test/regtest network names like -`-main.port=8333` `-test.server=1` previously were allowed but ignored. Now -they trigger "Invalid parameter" errors on startup. - Tests ----- -- It is now an error to use an unqualified `walletdir=path` setting in the config file if running on testnet or regtest - networks. The setting now needs to be qualified as `chain.walletdir=path` or placed in the appropriate `[chain]` - section. (#17447) - -- `-fallbackfee` was 0 (disabled) by default for the main chain, but 0.0002 by default for the test chains. Now it is 0 - by default for all chains. Testnet and regtest users will have to add `fallbackfee=0.0002` to their configuration if - they weren't setting it and they want it to keep working like before. (#16524) - Credits ======= diff --git a/doc/release-notes/release-notes-0.20.0.md b/doc/release-notes/release-notes-0.20.0.md new file mode 100644 index 0000000000..6fc1606b37 --- /dev/null +++ b/doc/release-notes/release-notes-0.20.0.md @@ -0,0 +1,987 @@ +0.20.0 Release Notes +==================== + +Bitcoin Core version 0.20.0 is now available from: + + <https://bitcoincore.org/bin/bitcoin-core-0.20.0/> + +This release includes new features, various bug fixes and performance +improvements, as well as updated translations. + +Please report bugs using the issue tracker at GitHub: + + <https://github.com/bitcoin/bitcoin/issues> + +To receive security and update notifications, please subscribe to: + + <https://bitcoincore.org/en/list/announcements/join/> + +How to Upgrade +============== + +If you are running an older version, shut it down. Wait until it has completely +shut down (which might take a few minutes in some cases), then run the +installer (on Windows) or just copy over `/Applications/Bitcoin-Qt` (on Mac) +or `bitcoind`/`bitcoin-qt` (on Linux). + +Upgrading directly from a version of Bitcoin Core that has reached its EOL is +possible, but it might take some time if the data directory needs to be migrated. Old +wallet versions of Bitcoin Core are generally supported. + +Compatibility +============== + +Bitcoin Core is supported and extensively tested on operating systems +using the Linux kernel, macOS 10.12+, and Windows 7 and newer. Bitcoin +Core should also work on most other Unix-like systems but is not as +frequently tested on them. It is not recommended to use Bitcoin Core on +unsupported systems. + +From Bitcoin Core 0.20.0 onwards, macOS versions earlier than 10.12 are no +longer supported. Additionally, Bitcoin Core does not yet change appearance +when macOS "dark mode" is activated. + +Known Bugs +========== + +The process for generating the source code release ("tarball") has changed in an +effort to make it more complete, however, there are a few regressions in +this release: + +- The generated `configure` script is currently missing, and you will need to + install autotools and run `./autogen.sh` before you can run + `./configure`. This is the same as when checking out from git. + +- Instead of running `make` simply, you should instead run + `BITCOIN_GENBUILD_NO_GIT=1 make`. + +Notable changes +=============== + +P2P and network changes +----------------------- + +#### Removal of BIP61 reject network messages from Bitcoin Core + +The `-enablebip61` command line option to enable BIP61 has been removed. +(#17004) + +This feature has been disabled by default since Bitcoin Core version 0.18.0. +Nodes on the network can not generally be trusted to send valid messages +(including reject messages), so this should only ever be used when +connected to a trusted node. Please use the alternatives recommended +below if you rely on this removed feature: + +- Testing or debugging of implementations of the Bitcoin P2P network protocol + should be done by inspecting the log messages that are produced by a recent + version of Bitcoin Core. Bitcoin Core logs debug messages + (`-debug=<category>`) to a stream (`-printtoconsole`) or to a file + (`-debuglogfile=<debug.log>`). + +- Testing the validity of a block can be achieved by specific RPCs: + + - `submitblock` + + - `getblocktemplate` with `'mode'` set to `'proposal'` for blocks with + potentially invalid POW + +- Testing the validity of a transaction can be achieved by specific RPCs: + + - `sendrawtransaction` + + - `testmempoolaccept` + +- Wallets should not assume a transaction has propagated to the network + just because there are no reject messages. Instead, listen for the + transaction to be announced by other peers on the network. Wallets + should not assume a lack of reject messages means a transaction pays + an appropriate fee. Instead, set fees using fee estimation and use + replace-by-fee to increase a transaction's fee if it hasn't confirmed + within the desired amount of time. + +The removal of BIP61 reject message support also has the following minor RPC +and logging implications: + +- `testmempoolaccept` and `sendrawtransaction` no longer return the P2P reject + code when a transaction is not accepted to the mempool. They still return the + verbal reject reason. + +- Log messages that previously reported the reject code when a transaction was + not accepted to the mempool now no longer report the reject code. The reason + for rejection is still reported. + +Updated RPCs +------------ + +- The RPCs which accept descriptors now accept the new `sortedmulti(...)` descriptor + type which supports multisig scripts where the public keys are sorted + lexicographically in the resulting script. (#17056) + +- The `walletprocesspsbt` and `walletcreatefundedpsbt` RPCs now include + BIP32 derivation paths by default for public keys if we know them. + This can be disabled by setting the `bip32derivs` parameter to + `false`. (#17264) + +- The `bumpfee` RPC's parameter `totalFee`, which was deprecated in + 0.19, has been removed. (#18312) + +- The `bumpfee` RPC will return a PSBT when used with wallets that have + private keys disabled. (#16373) + +- The `getpeerinfo` RPC now includes a `mapped_as` field to indicate the + mapped Autonomous System used for diversifying peer selection. See the + `-asmap` configuration option described below in _New Settings_. (#16702) + +- The `createmultisig` and `addmultisigaddress` RPCs now return an + output script descriptor for the newly created address. (#18032) + +Build System +------------ + +- OpenSSL is no longer used by Bitcoin Core. (#17265) + +- BIP70 support has been fully removed from Bitcoin Core. The + `--enable-bip70` option remains, but it will throw an error during configure. + (#17165) + +- glibc 2.17 or greater is now required to run the release binaries. This + retains compatibility with RHEL 7, CentOS 7, Debian 8 and Ubuntu 14.04 LTS. (#17538) + +- The source code archives that are provided with gitian builds no longer contain + any autotools artifacts. Therefore, to build from such source, a user + should run the `./autogen.sh` script from the root of the unpacked archive. + This implies that `autotools` and other required packages are installed on the + user's system. (#18331) + +New settings +------------ + +- New `rpcwhitelist` and `rpcwhitelistdefault` configuration parameters + allow giving certain RPC users permissions to only some RPC calls. + (#12763) + +- A new `-asmap` configuration option has been added to diversify the + node's network connections by mapping IP addresses Autonomous System + Numbers (ASNs) and then limiting the number of connections made to any + single ASN. See [issue #16599](https://github.com/bitcoin/bitcoin/issues/16599), + [PR #16702](https://github.com/bitcoin/bitcoin/pull/16702), and the + `bitcoind help` for more information. This option is experimental and + subject to removal or breaking changes in future releases, so the + legacy /16 prefix mapping of IP addresses remains the default. (#16702) + +Updated settings +---------------- + +- All custom settings configured when Bitcoin Core starts are now + written to the `debug.log` file to assist troubleshooting. (#16115) + +- Importing blocks upon startup via the `bootstrap.dat` file no longer + occurs by default. The file must now be specified with + `-loadblock=<file>`. (#17044) + +- The `-debug=db` logging category has been renamed to + `-debug=walletdb` to distinguish it from `coindb`. The `-debug=db` + option has been deprecated and will be removed in the next major + release. (#17410) + +- The `-walletnotify` configuration parameter will now replace any `%w` + in its argument with the name of the wallet generating the + notification. This is not supported on Windows. (#13339) + +Removed settings +---------------- + +- The `-whitelistforcerelay` configuration parameter has been removed after + it was discovered that it was rendered ineffective in version 0.13 and + hasn't actually been supported for almost four years. (#17985) + +GUI changes +----------- + +- The "Start Bitcoin Core on system login" option has been removed on macOS. + (#17567) + +- In the Peers window, the details for a peer now displays a `Mapped AS` + field to indicate the mapped Autonomous System used for diversifying + peer selection. See the `-asmap` configuration option in _New + Settings_, above. (#18402) + +- A "known bug" [announced](https://bitcoincore.org/en/releases/0.18.0/#wallet-gui) + in the release notes of version 0.18 has been fixed. The issue + affected anyone who simultaneously used multiple Bitcoin Core wallets + and the GUI coin control feature. (#18894) + +- For watch-only wallets, creating a new transaction in the Send screen + or fee bumping an existing transaction in the Transactions screen will + automatically copy a Partially-Signed Bitcoin Transaction (PSBT) to + the system clipboard. This can then be pasted into an external + program such as [HWI](https://github.com/bitcoin-core/HWI) for + signing. Future versions of Bitcoin Core should support a GUI option + for finalizing and broadcasting PSBTs, but for now the debug console + may be used with the `finalizepsbt` and `sendrawtransaction` RPCs. + (#16944, #17492) + +Wallet +------ + +- The wallet now by default uses bech32 addresses when using RPC, and + creates native segwit change outputs. (#16884) + +- The way that output trust was computed has been fixed, which affects + confirmed/unconfirmed balance status and coin selection. (#16766) + +- The `gettransaction`, `listtransactions` and `listsinceblock` RPC + responses now also include the height of the block that contains the + wallet transaction, if any. (#17437) + +- The `getaddressinfo` RPC has had its `label` field deprecated + (re-enable for this release using the configuration parameter + `-deprecatedrpc=label`). The `labels` field is altered from returning + JSON objects to returning a JSON array of label names (re-enable + previous behavior for this release using the configuration parameter + `-deprecatedrpc=labelspurpose`). Backwards compatibility using the + deprecated configuration parameters is expected to be dropped in the + 0.21 release. (#17585, #17578) + +Documentation changes +--------------------- + +- Bitcoin Core's automatically-generated source code documentation is + now available at https://doxygen.bitcoincore.org. (#17596) + +Low-level changes +================= + +Utilities +--------- + +- The `bitcoin-cli` utility used with the `-getinfo` parameter now + returns a `headers` field with the number of downloaded block headers + on the best headers chain (similar to the `blocks` field that is also + returned) and a `verificationprogress` field that estimates how much + of the best block chain has been synced by the local node. The + information returned no longer includes the `protocolversion`, + `walletversion`, and `keypoololdest` fields. (#17302, #17650) + +- The `bitcoin-cli` utility now accepts a `-stdinwalletpassphrase` + parameter that can be used when calling the `walletpassphrase` and + `walletpassphrasechange` RPCs to read the passphrase from standard + input without echoing it to the terminal, improving security against + anyone who can look at your screen. The existing `-stdinrpcpass` + parameter is also updated to not echo the passphrase. (#13716) + +Command line +------------ + +- Command line options prefixed with main/test/regtest network names like + `-main.port=8333` `-test.server=1` previously were allowed but ignored. Now + they trigger "Invalid parameter" errors on startup. (#17482) + +New RPCs +-------- + +- The `dumptxoutset` RPC outputs a serialized snapshot of the current + UTXO set. A script is provided in the `contrib/devtools` directory + for generating a snapshot of the UTXO set at a particular block + height. (#16899) + +- The `generatetodescriptor` RPC allows testers using regtest mode to + generate blocks that pay an arbitrary output script descriptor. + (#16943) + +Updated RPCs +------------ + +- The `verifychain` RPC default values are now static instead of + depending on the command line options or configuration file + (`-checklevel`, and `-checkblocks`). Users can pass in the RPC + arguments explicitly when they don't want to rely on the default + values. (#18541) + +- The `getblockchaininfo` RPC's `verificationprogress` field will no + longer report values higher than 1. Previously it would occasionally + report the chain was more than 100% verified. (#17328) + +Tests +----- + +- It is now an error to use an unqualified `walletdir=path` setting in + the config file if running on testnet or regtest networks. The setting + now needs to be qualified as `chain.walletdir=path` or placed in the + appropriate `[chain]` section. (#17447) + +- `-fallbackfee` was 0 (disabled) by default for the main chain, but + 0.0002 by default for the test chains. Now it is 0 by default for all + chains. Testnet and regtest users will have to add + `fallbackfee=0.0002` to their configuration if they weren't setting it + and they want it to keep working like before. (#16524) + +Build system +------------ + +- Support is provided for building with the Android Native Development + Kit (NDK). (#16110) + +0.20.0 change log +================= + +### Mining +- #18742 miner: Avoid stack-use-after-return in validationinterface (MarcoFalke) + +### Block and transaction handling +- #15283 log: Fix UB with bench on genesis block (instagibbs) +- #16507 feefilter: Compute the absolute fee rather than stored rate (instagibbs) +- #16688 log: Add validation interface logging (jkczyz) +- #16805 log: Add timing information to FlushStateToDisk() (jamesob) +- #16902 O(1) `OP_IF/NOTIF/ELSE/ENDIF` script implementation (sipa) +- #16945 introduce CChainState::GetCoinsCacheSizeState (jamesob) +- #16974 Walk pindexBestHeader back to ChainActive().Tip() if it is invalid (TheBlueMatt) +- #17004 Remove REJECT code from CValidationState (jnewbery) +- #17080 Explain why `fCheckDuplicateInputs` can not be skipped and remove it (MarcoFalke) +- #17328 GuessVerificationProgress: cap the ratio to 1 (darosior) +- #17399 Templatize ValidationState instead of subclassing (jkczyz) +- #17407 node: Add reference to mempool in NodeContext (MarcoFalke) +- #17708 prevector: Avoid misaligned member accesses (ajtowns) +- #17850,#17896,#17957,#18021,#18021,#18112 Serialization improvements (sipa) +- #17925 Improve UpdateTransactionsFromBlock with Epochs (JeremyRubin) +- #18002 Abstract out script execution out of `VerifyWitnessProgram()` (sipa) +- #18388 Make VerifyWitnessProgram use a Span stack (sipa) +- #18433 serialization: prevent int overflow for big Coin::nHeight (pierreN) +- #18500 chainparams: Bump assumed valid hash (MarcoFalke) +- #18551 Do not clear validationinterface entries being executed (sipa) + +### P2P protocol and network code +- #15437 Remove BIP61 reject messages (MarcoFalke) +- #16702 Supply and use asmap to improve IP bucketing in addrman (naumenkogs) +- #16851 Continue relaying transactions after they expire from mapRelay (ajtowns) +- #17164 Avoid allocating memory for addrKnown where we don't need it (naumenkogs) +- #17243 tools: add PoissonNextSend method that returns mockable time (amitiuttarwar) +- #17251 SocketHandler logs peer id for close and disconnect (Sjors) +- #17573 Seed RNG with precision timestamps on receipt of net messages (TheBlueMatt) +- #17624 Fix an uninitialized read in ProcessMessage(…, "tx", …) when receiving a transaction we already have (practicalswift) +- #17754 Don't allow resolving of std::string with embedded NUL characters. Add tests (practicalswift) +- #17758 Fix CNetAddr::IsRFC2544 comment + tests (tynes) +- #17812 config, net, test: Asmap feature refinements and functional tests (jonatack) +- #17951 Use rolling bloom filter of recent block txs for AlreadyHave() check (sdaftuar) +- #17985 Remove forcerelay of rejected txs (MarcoFalke) +- #18023 Fix some asmap issues (sipa) +- #18054 Reference instead of copy in BlockConnected range loop (jonatack) +- #18376 Fix use-after-free in tests (vasild) +- #18454 Make addr relay mockable, add test (MarcoFalke) +- #18458 Add missing `cs_vNodes` lock (MarcoFalke) +- #18506 Hardcoded seeds update for 0.20 (laanwj) +- #18808 Drop unknown types in getdata (jnewbery) +- #18962 Only send a getheaders for one block in an INV (jnewbery) + +### Wallet +- #13339 Replace %w by wallet name in -walletnotify script (promag) +- #15931 Remove GetDepthInMainChain dependency on locked chain interface (ariard) +- #16373 bumpfee: Return PSBT when wallet has privkeys disabled (instagibbs) +- #16524 Disable -fallbackfee by default (jtimon) +- #16766 Make IsTrusted scan parents recursively (JeremyRubin) +- #16884 Change default address type to bech32 (instagibbs) +- #16911 Only check the hash of transactions loaded from disk (achow101) +- #16923 Handle duplicate fileid exception (promag) +- #17056 descriptors: Introduce sortedmulti descriptor (achow101) +- #17070 Avoid showing GUI popups on RPC errors (MarcoFalke) +- #17138 Remove wallet access to some node arguments (jnewbery) +- #17237 LearnRelatedScripts only if KeepDestination (promag) +- #17260 Split some CWallet functions into new LegacyScriptPubKeyMan (achow101) +- #17261 Make ScriptPubKeyMan an actual interface and the wallet to have multiple (achow101) +- #17290 Enable BnB coin selection for preset inputs and subtract fee from outputs (achow101) +- #17373 Various fixes and cleanup to keypool handling in LegacyScriptPubKeyMan and CWallet (achow101) +- #17410 Rename `db` log category to `walletdb` (like `coindb`) (laanwj) +- #17444 Avoid showing GUI popups on RPC errors (take 2) (MarcoFalke) +- #17447 Make -walletdir network only (promag) +- #17537 Cleanup and move opportunistic and superfluous TopUp()s (achow101) +- #17553 Remove out of date comments for CalculateMaximumSignedTxSize (instagibbs) +- #17568 Fix when sufficient preset inputs and subtractFeeFromOutputs (achow101) +- #17677 Activate watchonly wallet behavior for LegacySPKM only (instagibbs) +- #17719 Document better -keypool as a look-ahead safety mechanism (ariard) +- #17843 Reset reused transactions cache (fjahr) +- #17889 Improve CWallet:MarkDestinationsDirty (promag) +- #18034 Get the OutputType for a descriptor (achow101) +- #18067 Improve LegacyScriptPubKeyMan::CanProvide script recognition (ryanofsky) +- #18115 Pass in transactions and messages for signing instead of exporting the private keys (achow101) +- #18192,#18546 Bugfix: Wallet: Safely deal with change in the address book (luke-jr) +- #18204 descriptors: Improve descriptor cache and cache xpubs (achow101) +- #18274 rpc/wallet: Initialize nFeeRequired to avoid using garbage value on failure (kallewoof) +- #18312 Remove deprecated fee bumping by totalFee (jonatack) +- #18338 Fix wallet unload race condition (promag) + +### RPC and other APIs +- #12763 Add RPC Whitelist Feature from #12248 (JeremyRubin) +- #13716 cli: `-stdinwalletpassphrase` and non-echo stdin passwords (kallewoof) +- #16689 Add missing fields to wallet rpc help output (ariard) +- #16821 Fix bug where duplicate PSBT keys are accepted (erasmospunk) +- #16899 UTXO snapshot creation (dumptxoutset) +- #17156 psbt: Check that various indexes and amounts are within bounds (achow101) +- #17264 Set default bip32derivs to true for psbt methods (Sjors) +- #17283 improve getaddressinfo test coverage, help, code docs (jonatack) +- #17302 cli: Add "headers" and "verificationprogress" to -getinfo (laanwj) +- #17318 replace asserts in RPC code with `CHECK_NONFATAL` and add linter (adamjonas) +- #17437 Expose block height of wallet transactions (promag) +- #17519 Remove unused `COINBASE_FLAGS` (narula) +- #17578 Simplify getaddressinfo labels, deprecate previous behavior (jonatack) +- #17585 deprecate getaddressinfo label (jonatack) +- #17746 Remove vector copy from listtransactions (promag) +- #17809 Auto-format RPCResult (MarcoFalke) +- #18032 Output a descriptor in createmultisig and addmultisigaddress (achow101) +- #18122 Update validateaddress RPCExamples to bech32 (theStack) +- #18208 Change RPCExamples to bech32 (yusufsahinhamza) +- #18268 Remove redundant types from descriptions (docallag) +- #18346 Document an RPCResult for all calls; Enforce at compile time (MarcoFalke) +- #18396 Add missing HelpExampleRpc for getblockfilter (theStack) +- #18398 Fix broken RPCExamples for waitforblock(height) (theStack) +- #18444 Remove final comma for last entry of fixed-size arrays/objects in RPCResult (luke-jr) +- #18459 Remove unused getbalances() code (jonatack) +- #18484 Correctly compute redeemScript from witnessScript for signrawtransaction (achow101) +- #18487 Fix rpcRunLater race in walletpassphrase (promag) +- #18499 Make rpc documentation not depend on call-time rpc args (MarcoFalke) +- #18532 Avoid initialization-order-fiasco on static CRPCCommand tables (MarcoFalke) +- #18541 Make verifychain default values static, not depend on global args (MarcoFalke) +- #18809 Do not advertise dumptxoutset as a way to flush the chainstate (MarcoFalke) +- #18814 Relock wallet only if most recent callback (promag) + +### GUI +- #15023 Restore RPC Console to non-wallet tray icon menu (luke-jr) +- #15084 Don't disable the sync overlay when wallet is disabled (benthecarman) +- #15098 Show addresses for "SendToSelf" transactions (hebasto) +- #15756 Add shortcuts for tab tools (promag) +- #16944 create PSBT with watch-only wallet (Sjors) +- #16964 Change sendcoins dialogue Yes to Send (instagibbs) +- #17068 Always generate `bitcoinstrings.cpp` on `make translate` (D4nte) +- #17096 Rename debug window (Zero-1729) +- #17105 Make RPCConsole::TabTypes an enum class (promag) +- #17125 Add toolTip and placeholderText to sign message fields (dannmat) +- #17165 Remove BIP70 support (fanquake) +- #17180 Improved tooltip for send amount field (JeremyCrookshank) +- #17186 Add placeholder text to the sign message field (Danny-Scott) +- #17195 Send amount placeholder value (JeremyCrookshank) +- #17226 Fix payAmount tooltip in SendCoinsEntry (promag) +- #17360 Cleaning up hide button tool tip (Danny-Scott) +- #17446 Changed tooltip for 'Label' & 'Message' text fields to be more clear (dannmat) +- #17453 Fix intro dialog labels when the prune button is toggled (hebasto) +- #17474 Bugfix: GUI: Recognise `NETWORK_LIMITED` in formatServicesStr (luke-jr) +- #17492 Bump fee returns PSBT on clipboard for watchonly-only wallets (instagibbs) +- #17567 Remove macOS start on login code (fanquake) +- #17587 Show watch-only balance in send screen (Sjors) +- #17694 Disable 3rd-party tx-urls when wallet disabled (brakmic) +- #17696 Force set nPruneSize in QSettings after the intro dialog (hebasto) +- #17702 Move static placeholder texts to forms (laanwj) +- #17826 Log Qt related info (hebasto) +- #17886 Restore English translation option (achow101) +- #17906 Set CConnman byte counters earlier to avoid uninitialized reads (ryanofsky) +- #17935 Hide HD & encryption icons when no wallet loaded (brakmic) +- #17998 Shortcut to close ModalOverlay (emilengler) +- #18007 Bugfix: GUI: Hide the HD/encrypt icons earlier so they get re-shown if another wallet is open (luke-jr) +- #18060 Drop PeerTableModel dependency to ClientModel (promag) +- #18062 Fix unintialized WalletView::progressDialog (promag) +- #18091 Pass clientmodel changes from walletframe to walletviews (jonasschnelli) +- #18101 Fix deprecated QCharRef usage (hebasto) +- #18121 Throttle GUI update pace when -reindex (hebasto) +- #18123 Fix race in WalletModel::pollBalanceChanged (ryanofsky) +- #18160 Avoid Wallet::GetBalance in WalletModel::pollBalanceChanged (promag) +- #18360 Bump transifex slug and update English translations for 0.20 (laanwj) +- #18402 Display mapped AS in peers info window (jonatack) +- #18492 Translations update pre-branch (laanwj) +- #18549 Fix Window -> Minimize menu item (hebasto) +- #18578 Fix leak in CoinControlDialog::updateView (promag) +- #18894 Fix manual coin control with multiple wallets loaded (promag) + +### Build system +- #16667 Remove mingw linker workaround from win gitian descriptor (fanquake) +- #16669 Use new fork of osslsigncode for windows gitian signing (fanquake) +- #16949 Only pass --disable-dependency-tracking to packages that understand it (fanquake) +- #17008 Bump libevent to 2.1.11 in depends (stefanwouldgo) +- #17029 gitian: Various improvements for windows descriptor (dongcarl) +- #17033 Disable _FORTIFY_SOURCE when enable-debug (achow101) +- #17057 Switch to upstream libdmg-hfsplus (fanquake) +- #17066 Remove workaround for ancient libtool (hebasto) +- #17074 Added double quotes (mztriz) +- #17087 Add variable printing target to Makefiles (dongcarl) +- #17118 depends macOS: point --sysroot to SDK (Sjors) +- #17231 Fix boost mac cross build with clang 9+ (theuni) +- #17265 Remove OpenSSL (fanquake) +- #17284 Update retry to current version (RandyMcMillan) +- #17308 nsis: Write to correct filename in first place (dongcarl) +- #17324,#18099 Update univalue subtree (MarcoFalke) +- #17398 Update leveldb to 1.22+ (laanwj) +- #17409 Avoid hardcoded libfaketime dir in gitian (MarcoFalke) +- #17466 Fix C{,XX} pickup (dongcarl) +- #17483 Set gitian arch back to amd64 (MarcoFalke) +- #17486 Make Travis catch unused variables (Sjors) +- #17538 Bump minimum libc to 2.17 for release binaries (fanquake) +- #17542 Create test utility library from src/test/util/ (brakmic) +- #17545 Remove libanl.so.1 from ALLOWED_LIBRARIES (fanquake) +- #17547 Fix configure report about qr (hebasto) +- #17569 Allow export of environ symbols and work around rv64 toolchain issue (laanwj) +- #17647 lcov: filter depends from coverage reports (nijynot) +- #17658 Add ability to skip building qrencode (fanquake) +- #17678 Support for S390X and POWER targets (MarcoFalke) +- #17682 util: Update tinyformat to upstream (laanwj) +- #17698 Don't configure `xcb_proto` (fanquake) +- #17730 Remove Qt networking features (fanquake) +- #17738 Remove linking librt for backwards compatibility (fanquake) +- #17740 Remove configure checks for win libraries we don't link against (fanquake) +- #17741 Included `test_bitcoin-qt` in msvc build (sipsorcery) +- #17756 Remove `WINDOWS_BITS` from build system (fanquake) +- #17769 Set `AC_PREREQ` to 2.69 (fanquake) +- #17880 Add -Wdate-time to Werror flags (fanquake) +- #17910 Remove double `LIBBITCOIN_SERVER` linking (fanquake) +- #17928 Consistent use of package variable (Bushstar) +- #17933 guix: Pin Guix using `guix time-machine` (dongcarl) +- #17948 pass -fno-ident in Windows gitian descriptor (fanquake) +- #18003 Remove --large-address-aware linker flag (fanquake) +- #18004 Don't embed a build-id when building libdmg-hfsplus (fanquake) +- #18051 Fix behavior when `ALLOW_HOST_PACKAGES` unset (hebasto) +- #18059 Add missing attributes to Win installer (fanquake) +- #18104 Skip i686 build by default in guix and gitian (MarcoFalke) +- #18107 Add `cov_fuzz` target (MarcoFalke) +- #18135 Add --enable-determinism configure flag (fanquake) +- #18145 Add Wreturn-type to Werror flags, check on more Travis machines (Sjors) +- #18264 Remove Boost Chrono (fanquake) +- #18290 Set minimum Automake version to 1.13 (hebasto) +- #18320 guix: Remove now-unnecessary gcc make flag (dongcarl) +- #18331 Use git archive as source tarball (hebasto) +- #18397 Fix libevent linking for `bench_bitcoin` binary (hebasto) +- #18426 scripts: `Previous_release`: improve behaviour on failed download (theStack) +- #18429 Remove double `LIBBITCOIN_SERVER` from bench-Makefile (brakmic) +- #18528 Create `test_fuzz` library from src/test/fuzz/fuzz.cpp (brakmic) +- #18558 Fix boost detection for arch armv7l (hebasto) +- #18598 gitian: Add missing automake package to gitian-win-signer.yml (achow101) +- #18676 Check libevent minimum version in configure script (hebasto) +- #18945 Ensure source tarball has leading directory name (laanwj) + +### Platform support +- #16110 Add Android NDK support (icota) +- #16392 macOS toolchain update (fanquake) +- #16569 Increase init file stop timeout (setpill) +- #17151 Remove OpenSSL PRNG seeding (Windows, Qt only) (fanquake) +- #17365 Update README.md with working Android targets and API levels (icota) +- #17521 Only use D-Bus with Qt on linux (fanquake) +- #17550 Set minimum supported macOS to 10.12 (fanquake) +- #17592 Appveyor install libevent[thread] vcpkg (sipsorcery) +- #17660 Remove deprecated key from macOS Info.plist (fanquake) +- #17663 Pass `-dead_strip_dylibs` to ld on macOS (fanquake) +- #17676 Don't use OpenGL in Qt on macOS (fanquake) +- #17686 Add `-bind_at_load` to macOS hardened LDFLAGS (fanquake) +- #17787 scripts: Add macho pie check to security-check.py (fanquake) +- #17800 random: don't special case clock usage on macOS (fanquake) +- #17863 scripts: Add macho dylib checks to symbol-check.py (fanquake) +- #17899 msvc: Ignore msvc linker warning and update to msvc build instructions (sipsorcery) +- #17916 windows: Enable heap terminate-on-corruption (fanquake) +- #18082 logging: Enable `thread_local` usage on macos (fanquake) +- #18108 Fix `.gitignore` policy in `build_msvc` directory (hebasto) +- #18295 scripts: Add macho lazy bindings check to security-check.py (fanquake) +- #18358 util: Fix compilation with mingw-w64 7.0.0 (fanquake) +- #18359 Fix sysctl() detection on macOS (fanquake) +- #18364 random: remove getentropy() fallback for macOS < 10.12 (fanquake) +- #18395 scripts: Add pe dylib checking to symbol-check.py (fanquake) +- #18415 scripts: Add macho tests to test-security-check.py (fanquake) +- #18425 releases: Update with new Windows code signing certificate (achow101) +- #18702 Fix ASLR for bitcoin-cli on Windows (fanquake) + +### Tests and QA +- #12134 Build previous releases and run functional tests (Sjors) +- #13693 Add coverage to estimaterawfee and estimatesmartfee (Empact) +- #13728 lint: Run the ci lint stage on mac (Empact) +- #15443 Add getdescriptorinfo functional test (promag) +- #15888 Add `wallet_implicitsegwit` to test the ability to transform keys between address types (luke-jr) +- #16540 Add `ASSERT_DEBUG_LOG` to unit test framework (MarcoFalke) +- #16597 travis: Run full test suite on native macos (Sjors) +- #16681 Use self.chain instead of 'regtest' in all current tests (jtimon) +- #16786 add unit test for wallet watch-only methods involving PubKeys (theStack) +- #16943 Add generatetodescriptor RPC (MarcoFalke) +- #16973 Fix `combine_logs.py` for AppVeyor build (mzumsande) +- #16975 Show debug log on unit test failure (MarcoFalke) +- #16978 Seed test RNG context for each test case, print seed (MarcoFalke) +- #17009, #17018, #17050, #17051, #17071, #17076, #17083, #17093, #17109, #17113, #17136, #17229, #17291, #17357, #17771, #17777, #17917, #17926, #17972, #17989, #17996, #18009, #18029, #18047, #18126, #18176, #18206, #18353, #18363, #18407, #18417, #18423, #18445, #18455, #18565 Add fuzzing harnesses (practicalswift) +- #17011 ci: Use busybox utils for one build (MarcoFalke) +- #17030 Fix Python Docstring to include all Args (jbampton) +- #17041 ci: Run tests on arm (MarcoFalke) +- #17069 Pass fuzzing inputs as constant references (practicalswift) +- #17091 Add test for loadblock option and linearize scripts (fjahr) +- #17108 fix "tx-size-small" errors after default address change (theStack) +- #17121 Speed up `wallet_backup` by whitelisting peers (immediate tx relay) (theStack) +- #17124 Speed up `wallet_address_types` by whitelisting peers (immediate tx relay) (theStack) +- #17140 Fix bug in `blockfilter_index_tests` (jimpo) +- #17199 use default address type (bech32) for `wallet_bumpfee` tests (theStack) +- #17205 ci: Enable address sanitizer (asan) stack-use-after-return checking (practicalswift) +- #17206 Add testcase to simulate bitcoin schema in leveldb (adamjonas) +- #17209 Remove no longer needed UBSan suppressions (issues fixed). Add documentation (practicalswift) +- #17220 Add unit testing for the CompressScript function (adamjonas) +- #17225 Test serialisation as part of deserialisation fuzzing. Test round-trip equality where possible (practicalswift) +- #17228 Add RegTestingSetup to `setup_common` (MarcoFalke) +- #17233 travis: Run unit and functional tests on native arm (MarcoFalke) +- #17235 Skip unnecessary fuzzer initialisation. Hold ECCVerifyHandle only when needed (practicalswift) +- #17240 ci: Disable functional tests on mac host (MarcoFalke) +- #17254 Fix `script_p2sh_tests` `OP_PUSHBACK2/4` missing (adamjonas) +- #17267 bench: Fix negative values and zero for -evals flag (nijynot) +- #17275 pubkey: Assert CPubKey's ECCVerifyHandle precondition (practicalswift) +- #17288 Added TestWrapper class for interactive Python environments (jachiang) +- #17292 Add new mempool benchmarks for a complex pool (JeremyRubin) +- #17299 add reason checks for non-standard txs in `test_IsStandard` (theStack) +- #17322 Fix input size assertion in `wallet_bumpfee.py` (instagibbs) +- #17327 Add `rpc_fundrawtransaction` logging (jonatack) +- #17330 Add `shrinkdebugfile=0` to regtest bitcoin.conf (sdaftuar) +- #17340 Speed up fundrawtransaction test (jnewbery) +- #17345 Do not instantiate CAddrDB for static call CAddrDB::Read() (hebasto) +- #17362 Speed up `wallet_avoidreuse`, add logging (jonatack) +- #17363 add "diamond" unit test to MempoolAncestryTests (theStack) +- #17366 Reset global args between test suites (MarcoFalke) +- #17367 ci: Run non-cross-compile builds natively (MarcoFalke) +- #17378 TestShell: Fix typos & implement cleanups (jachiang) +- #17384 Create new test library (MarcoFalke) +- #17387 `wallet_importmulti`: use addresses of the same type as being imported (achow101) +- #17388 Add missing newline in `util_ChainMerge` test (ryanofsky) +- #17390 Add `util_ArgParsing` test (ryanofsky) +- #17420 travis: Rework `cache_err_msg` (MarcoFalke) +- #17423 ci: Make ci system read-only on the git work tree (MarcoFalke) +- #17435 check custom ancestor limit in `mempool_packages.py` (theStack) +- #17455 Update valgrind suppressions (practicalswift) +- #17461 Check custom descendant limit in `mempool_packages.py` (theStack) +- #17469 Remove fragile `assert_memory_usage_stable` (MarcoFalke) +- #17470 ci: Use clang-8 for fuzzing to run on aarch64 ci systems (MarcoFalke) +- #17480 Add unit test for non-standard txs with too large scriptSig (theStack) +- #17497 Skip tests when utils haven't been compiled (fanquake) +- #17502 Add unit test for non-standard bare multisig txs (theStack) +- #17511 Add bounds checks before base58 decoding (sipa) +- #17517 ci: Bump to clang-8 for asan build to avoid segfaults on ppc64le (MarcoFalke) +- #17522 Wait until mempool is loaded in `wallet_abandonconflict` (MarcoFalke) +- #17532 Add functional test for non-standard txs with too large scriptSig (theStack) +- #17541 Add functional test for non-standard bare multisig txs (theStack) +- #17555 Add unit test for non-standard txs with wrong nVersion (dspicher) +- #17571 Add `libtest_util` library to msvc build configuration (sipsorcery) +- #17591 ci: Add big endian platform - s390x (elichai) +- #17593 Move more utility functions into test utility library (mzumsande) +- #17633 Add option --valgrind to run the functional tests under Valgrind (practicalswift) +- #17635 ci: Add centos 7 build (hebasto) +- #17641 Add unit test for leveldb creation with unicode path (sipsorcery) +- #17674 Add initialization order fiasco detection in Travis (practicalswift) +- #17675 Enable tests which are incorrectly skipped when running `test_runner.py --usecli` (practicalswift) +- #17685 Fix bug in the descriptor parsing fuzzing harness (`descriptor_parse`) (practicalswift) +- #17705 re-enable CLI test support by using EncodeDecimal in json.dumps() (fanquake) +- #17720 add unit test for non-standard "scriptsig-not-pushonly" txs (theStack) +- #17767 ci: Fix qemu issues (MarcoFalke) +- #17793 ci: Update github actions ci vcpkg cache on msbuild update (hebasto) +- #17806 Change filemode of `rpc_whitelist.py` (emilengler) +- #17849 ci: Fix brew python link (hebasto) +- #17851 Add `std::to_string` to list of locale dependent functions (practicalswift) +- #17893 Fix double-negative arg test (hebasto) +- #17900 ci: Combine 32-bit build with centos 7 build (theStack) +- #17921 Test `OP_CSV` empty stack fail in `feature_csv_activation.py` (theStack) +- #17931 Fix `p2p_invalid_messages` failing in Python 3.8 because of warning (elichai) +- #17947 add unit test for non-standard txs with too large tx size (theStack) +- #17959 Check specific reject reasons in `feature_csv_activation.py` (theStack) +- #17984 Add p2p test for forcerelay permission (MarcoFalke) +- #18001 Updated appveyor job to checkout a specific vcpkg commit ID (sipsorcery) +- #18008 fix fuzzing using libFuzzer on macOS (fanquake) +- #18013 bench: Fix benchmarks filters (elichai) +- #18018 reset fIsBareMultisigStd after bare-multisig tests (fanquake) +- #18022 Fix appveyor `test_bitcoin` build of `*.raw` (MarcoFalke) +- #18037 util: Allow scheduler to be mocked (amitiuttarwar) +- #18056 ci: Check for submodules (emilengler) +- #18069 Replace 'regtest' leftovers by self.chain (theStack) +- #18081 Set a name for CI Docker containers (fanquake) +- #18109 Avoid hitting some known minor tinyformat issues when fuzzing strprintf(…) (practicalswift) +- #18155 Add harness which fuzzes EvalScript and VerifyScript using a fuzzed signature checker (practicalswift) +- #18159 Add --valgrind option to `test/fuzz/test_runner.py` for running fuzzing test cases under valgrind (practicalswift) +- #18166 ci: Run fuzz testing test cases (bitcoin-core/qa-assets) under valgrind to catch memory errors (practicalswift) +- #18172 Transaction expiry from mempool (0xB10C) +- #18181 Remove incorrect assumptions in `validation_flush_tests` (MarcoFalke) +- #18183 Set `catch_system_errors=no` on boost unit tests (MarcoFalke) +- #18195 Add `cost_of_change` parameter assertions to `bnb_search_test` (yancyribbens) +- #18209 Reduce unneeded whitelist permissions in tests (MarcoFalke) +- #18211 Disable mockforward scheduler unit test for now (MarcoFalke) +- #18213 Fix race in `p2p_segwit` (MarcoFalke) +- #18224 Make AnalyzePSBT next role calculation simple, correct (instagibbs) +- #18228 Add missing syncwithvalidationinterfacequeue (MarcoFalke) +- #18247 Wait for both veracks in `add_p2p_connection` (MarcoFalke) +- #18249 Bump timeouts to accomodate really slow disks (MarcoFalke) +- #18255 Add `bad-txns-*-toolarge` test cases to `invalid_txs` (MarcoFalke) +- #18263 rpc: change setmocktime check to use IsMockableChain (gzhao408) +- #18285 Check that `wait_until` returns if time point is in the past (MarcoFalke) +- #18286 Add locale fuzzer to `FUZZERS_MISSING_CORPORA` (practicalswift) +- #18292 fuzz: Add `assert(script == decompressed_script)` (MarcoFalke) +- #18299 Update `FUZZERS_MISSING_CORPORA` to enable regression fuzzing for all harnesses in master (practicalswift) +- #18300 fuzz: Add option to merge input dir to test runner (MarcoFalke) +- #18305 Explain why test logging should be used (MarcoFalke) +- #18306 Add logging to `wallet_listsinceblock.py` (jonatack) +- #18311 Bumpfee test fix (instagibbs) +- #18314 Add deserialization fuzzing of SnapshotMetadata (`utxo_snapshot`) (practicalswift) +- #18319 fuzz: Add missing `ECC_Start` to `key_io` test (MarcoFalke) +- #18334 Add basic test for BIP 37 (MarcoFalke) +- #18350 Fix mining to an invalid target + ensure that a new block has the correct hash internally (TheQuantumPhysicist) +- #18378 Bugfix & simplify bn2vch using `int.to_bytes` (sipa) +- #18393 Don't assume presence of `__builtin_mul_overflow(…)` in `MultiplicationOverflow(…)` fuzzing harness (practicalswift) +- #18406 add executable flag for `rpc_estimatefee.py` (theStack) +- #18420 listsinceblock block height checks (jonatack) +- #18430 ci: Only clone bitcoin-core/qa-assets when fuzzing (MarcoFalke) +- #18438 ci: Use homebrew addon on native macos (hebasto) +- #18447 Add coverage for script parse error in ParseScript (pierreN) +- #18472 Remove unsafe `BOOST_TEST_MESSAGE` (MarcoFalke) +- #18474 check that peer is connected when calling sync_* (MarcoFalke) +- #18477 ci: Use focal for fuzzers (MarcoFalke) +- #18481 add BIP37 'filterclear' test to p2p_filter.py (theStack) +- #18496 Remove redundant `sync_with_ping` after `add_p2p_connection` (jonatack) +- #18509 fuzz: Avoid running over all inputs after merging them (MarcoFalke) +- #18510 fuzz: Add CScriptNum::getint coverage (MarcoFalke) +- #18514 remove rapidcheck integration and tests (fanquake) +- #18515 Add BIP37 remote crash bug [CVE-2013-5700] test to `p2p_filter.py` (theStack) +- #18516 relax bumpfee `dust_to_fee` txsize an extra vbyte (jonatack) +- #18518 fuzz: Extend descriptor fuzz test (MarcoFalke) +- #18519 fuzz: Extend script fuzz test (MarcoFalke) +- #18521 fuzz: Add `process_messages` harness (MarcoFalke) +- #18529 Add fuzzer version of randomized prevector test (sipa) +- #18534 skip backwards compat tests if not compiled with wallet (fanquake) +- #18540 `wallet_bumpfee` assertion fixup (jonatack) +- #18543 Use one node to avoid a race due to missing sync in `rpc_signrawtransaction` (MarcoFalke) +- #18561 Properly raise FailedToStartError when rpc shutdown before warmup finished (MarcoFalke) +- #18562 ci: Run unit tests sequential once (MarcoFalke) +- #18563 Fix `unregister_all_during_call` cleanup (ryanofsky) +- #18566 Set `-use_value_profile=1` when merging fuzz inputs (MarcoFalke) +- #18757 Remove enumeration of expected deserialization exceptions in ProcessMessage(…) fuzzer (practicalswift) +- #18878 Add test for conflicted wallet tx notifications (ryanofsky) +- #18975 Remove const to work around compiler error on xenial (laanwj) + +### Documentation +- #16947 Doxygen-friendly script/descriptor.h comments (ch4ot1c) +- #16983 Add detailed info about Bitcoin Core files (hebasto) +- #16986 Doxygen-friendly CuckooCache comments (ch4ot1c) +- #17022 move-only: Steps for "before major release branch-off" (MarcoFalke) +- #17026 Update bips.md for default bech32 addresses in 0.20.0 (MarcoFalke) +- #17081 Fix Makefile target in benchmarking.md (theStack) +- #17102 Add missing indexes/blockfilter/basic to doc/files.md (MarcoFalke) +- #17119 Fix broken bitcoin-cli examples (andrewtoth) +- #17134 Add switch on enum example to developer notes (hebasto) +- #17142 Update macdeploy README to include all files produced by `make deploy` (za-kk) +- #17146 github: Add warning for bug reports (laanwj) +- #17157 Added instructions for how to add an upsteam to forked repo (dannmat) +- #17159 Add a note about backporting (carnhofdaki) +- #17169 Correct function name in ReportHardwareRand() (fanquake) +- #17177 Describe log files + consistent paths in test READMEs (fjahr) +- #17239 Changed miniupnp links to https (sandakersmann) +- #17281 Add developer note on `c_str()` (laanwj) +- #17285 Bip70 removal follow-up (fjahr) +- #17286 Fix help-debug -checkpoints (ariard) +- #17309 update MSVC instructions to remove Qt OpenSSL linking (fanquake) +- #17339 Add template for good first issues (michaelfolkson) +- #17351 Fix some misspellings (RandyMcMillan) +- #17353 Add ShellCheck to lint tests dependencies (hebasto) +- #17370 Update doc/bips.md with recent changes in master (MarcoFalke) +- #17393 Added regtest config for linearize script (gr0kchain) +- #17411 Add some better examples for scripted diff (laanwj) +- #17503 Remove bitness from bitcoin-qt help message and manpage (laanwj) +- #17539 Update and improve Developer Notes (hebasto) +- #17561 Changed MiniUPnPc link to https in dependencies.md (sandakersmann) +- #17596 Change doxygen URL to doxygen.bitcoincore.org (laanwj) +- #17598 Update release process with latest changes (MarcoFalke) +- #17617 Unify unix epoch time descriptions (jonatack) +- #17637 script: Add keyserver to verify-commits readme (emilengler) +- #17648 Rename wallet-tool references to bitcoin-wallet (hel-o) +- #17688 Add "ci" prefix to CONTRIBUTING.md (hebasto) +- #17751 Use recommended shebang approach in documentation code block (hackerrdave) +- #17752 Fix directory path for secp256k1 subtree in developer-notes (hackerrdave) +- #17772 Mention PR Club in CONTRIBUTING.md (emilengler) +- #17804 Misc RPC help fixes (MarcoFalke) +- #17819 Developer notes guideline on RPCExamples addresses (jonatack) +- #17825 Update dependencies.md (hebasto) +- #17873 Add to Doxygen documentation guidelines (jonatack) +- #17907 Fix improper Doxygen inline comments (Empact) +- #17942 Improve fuzzing docs for macOS users (fjahr) +- #17945 Fix doxygen errors (Empact) +- #18025 Add missing supported rpcs to doc/descriptors.md (andrewtoth) +- #18070 Add note about `brew doctor` (givanse) +- #18125 Remove PPA note from release-process.md (fanquake) +- #18170 Minor grammatical changes and flow improvements (travinkeith) +- #18212 Add missing step in win deployment instructions (dangershony) +- #18219 Add warning against wallet.dat re-use (corollari) +- #18253 Correct spelling errors in comments (Empact) +- #18278 interfaces: Describe and follow some code conventions (ryanofsky) +- #18283 Explain rebase policy in CONTRIBUTING.md (MarcoFalke) +- #18340 Mention MAKE=gmake workaround when building on a BSD (fanquake) +- #18341 Replace remaining literal BTC with `CURRENCY_UNIT` (domob1812) +- #18342 Add fuzzing quickstart guides for libFuzzer and afl-fuzz (practicalswift) +- #18344 Fix nit in getblockchaininfo (stevenroose) +- #18379 Comment fix merkle.cpp (4d55397500) +- #18382 note the costs of fetching all pull requests (vasild) +- #18391 Update init and reduce-traffic docs for -blocksonly (glowang) +- #18464 Block-relay-only vs blocksonly (MarcoFalke) +- #18486 Explain new test logging (MarcoFalke) +- #18505 Update webchat URLs in README.md (SuriyaaKudoIsc) +- #18513 Fix git add argument (HashUnlimited) +- #18577 Correct scripted-diff example link (yahiheb) +- #18589 Fix naming of macOS SDK and clarify version (achow101) + +### Miscellaneous +- #15600 lockedpool: When possible, use madvise to avoid including sensitive information in core dumps (luke-jr) +- #15934 Merge settings one place instead of five places (ryanofsky) +- #16115 On bitcoind startup, write config args to debug.log (LarryRuane) +- #16117 util: Replace boost sleep with std sleep (MarcoFalke) +- #16161 util: Fix compilation errors in support/lockedpool.cpp (jkczyz) +- #16802 scripts: In linearize, search for next position of magic bytes rather than fail (takinbo) +- #16889 Add some general std::vector utility functions (sipa) +- #17049 contrib: Bump gitian descriptors for 0.20 (MarcoFalke) +- #17052 scripts: Update `copyright_header` script to include additional files (GChuf) +- #17059 util: Simplify path argument for cblocktreedb ctor (hebasto) +- #17191 random: Remove call to `RAND_screen()` (Windows only) (fanquake) +- #17192 util: Add `check_nonfatal` and use it in src/rpc (MarcoFalke) +- #17218 Replace the LogPrint function with a macro (jkczyz) +- #17266 util: Rename decodedumptime to parseiso8601datetime (elichai) +- #17270 Feed environment data into RNG initializers (sipa) +- #17282 contrib: Remove accounts from bash completion (fanquake) +- #17293 Add assertion to randrange that input is not 0 (JeremyRubin) +- #17325 log: Fix log message for -par=1 (hebasto) +- #17329 linter: Strip trailing / in path for git-subtree-check (jnewbery) +- #17336 scripts: Search for first block file for linearize-data with some block files pruned (Rjected) +- #17361 scripts: Lint gitian descriptors with shellcheck (hebasto) +- #17482 util: Disallow network-qualified command line options (ryanofsky) +- #17507 random: mark RandAddPeriodic and SeedPeriodic as noexcept (fanquake) +- #17527 Fix CPUID subleaf iteration (sipa) +- #17604 util: Make schedulebatchpriority advisory only (fanquake) +- #17650 util: Remove unwanted fields from bitcoin-cli -getinfo (malevolent) +- #17671 script: Fixed wget call in gitian-build.py (willyko) +- #17699 Make env data logging optional (sipa) +- #17721 util: Don't allow base58 decoding of non-base58 strings. add base58 tests (practicalswift) +- #17750 util: Change getwarnings parameter to bool (jnewbery) +- #17753 util: Don't allow base32/64-decoding or parsemoney(…) on strings with embedded nul characters. add tests (practicalswift) +- #17823 scripts: Read suspicious hosts from a file instead of hardcoding (sanjaykdragon) +- #18162 util: Avoid potential uninitialized read in `formatiso8601datetime(int64_t)` by checking `gmtime_s`/`gmtime_r` return value (practicalswift) +- #18167 Fix a violation of C++ standard rules where unions are used for type-punning (TheQuantumPhysicist) +- #18225 util: Fail to parse empty string in parsemoney (MarcoFalke) +- #18270 util: Fail to parse whitespace-only strings in parsemoney(…) (instead of parsing as zero) (practicalswift) +- #18316 util: Helpexamplerpc formatting (jonatack) +- #18357 Fix missing header in sync.h (promag) +- #18412 script: Fix `script_err_sig_pushonly` error string (theStack) +- #18416 util: Limit decimal range of numbers parsescript accepts (pierreN) +- #18503 init: Replace `URL_WEBSITE` with `PACKAGE_URL` (MarcoFalke) +- #18526 Remove PID file at the very end (hebasto) +- #18553 Avoid non-trivial global constants in SHA-NI code (sipa) +- #18665 Do not expose and consider `-logthreadnames` when it does not work (hebasto) + +Credits +======= + +Thanks to everyone who directly contributed to this release: + +- 0xb10c +- 251 +- 4d55397500 +- Aaron Clauson +- Adam Jonas +- Albert +- Amiti Uttarwar +- Andrew Chow +- Andrew Toth +- Anthony Towns +- Antoine Riard +- Ava Barron +- Ben Carman +- Ben Woosley +- Block Mechanic +- Brian Solon +- Bushstar +- Carl Dong +- Carnhof Daki +- Cory Fields +- Daki Carnhof +- Dan Gershony +- Daniel Kraft +- dannmat +- Danny-Scott +- darosior +- David O'Callaghan +- Dominik Spicher +- Elichai Turkel +- Emil Engler +- emu +- Fabian Jahr +- fanquake +- Filip Gospodinov +- Franck Royer +- Gastón I. Silva +- gchuf +- Gleb Naumenko +- Gloria Zhao +- glowang +- Gr0kchain +- Gregory Sanders +- hackerrdave +- Harris +- hel0 +- Hennadii Stepanov +- ianliu +- Igor Cota +- James Chiang +- James O'Beirne +- Jan Beich +- Jan Sarenik +- Jeffrey Czyz +- Jeremy Rubin +- JeremyCrookshank +- Jim Posen +- John Bampton +- John L. Jegutanis +- John Newbery +- Jon Atack +- Jon Layton +- Jonas Schnelli +- João Barbosa +- Jorge Timón +- Karl-Johan Alm +- kodslav +- Larry Ruane +- Luke Dashjr +- malevolent +- MapleLaker +- marcaiaf +- MarcoFalke +- Marius Kjærstad +- Mark Erhardt +- Mark Tyneway +- Martin Erlandsson +- Martin Zumsande +- Matt Corallo +- Matt Ward +- Michael Folkson +- Michael Polzer +- Micky Yun Chan +- Neha Narula +- nijynot +- naumenkogs +- NullFunctor +- Peter Bushnell +- pierrenn +- Pieter Wuille +- practicalswift +- randymcmillan +- Rjected +- Russell Yanofsky +- Samer Afach +- Samuel Dobson +- Sanjay K +- Sebastian Falbesoner +- setpill +- Sjors Provoost +- Stefan Richter +- stefanwouldgo +- Steven Roose +- Suhas Daftuar +- Suriyaa Sundararuban +- TheCharlatan +- Tim Akinbo +- Travin Keith +- tryphe +- Vasil Dimov +- Willy Ko +- Wilson Ccasihue S +- Wladimir J. van der Laan +- Yahia Chiheb +- Yancy Ribbens +- Yusuf Sahin HAMZA +- Zakk +- Zero + +As well as to everyone that helped with translations on +[Transifex](https://www.transifex.com/bitcoin/bitcoin/). diff --git a/doc/release-process.md b/doc/release-process.md index e0f29f6ad7..a61b67c35f 100644 --- a/doc/release-process.md +++ b/doc/release-process.md @@ -219,7 +219,7 @@ Codesigner only: Commit the detached codesign payloads: rm -rf * tar xf signature-osx.tar.gz tar xf signature-win.tar.gz - git add -a + git add -A git commit -m "point to ${VERSION}" git tag -s v${VERSION} HEAD git push the current branch and new tag diff --git a/doc/translation_strings_policy.md b/doc/translation_strings_policy.md index 634aca3559..1931302dda 100644 --- a/doc/translation_strings_policy.md +++ b/doc/translation_strings_policy.md @@ -23,7 +23,8 @@ On a high level, these strings are to be translated: ### GUI strings -Anything that appears to the user in the GUI is to be translated. This includes labels, menu items, button texts, tooltips and window titles. +Do not translate technical or extremely rare errors. +Anything else that appears to the user in the GUI is to be translated. This includes labels, menu items, button texts, tooltips and window titles. This includes messages passed to the GUI through the UI interface through `InitMessage`, `ThreadSafeMessageBox` or `ShowProgress`. General recommendations diff --git a/doc/zmq.md b/doc/zmq.md index a309abd0cc..3a1194de1c 100644 --- a/doc/zmq.md +++ b/doc/zmq.md @@ -96,7 +96,7 @@ ZeroMQ endpoint specifiers for TCP (and others) are documented in the Client side, then, the ZeroMQ subscriber socket must have the ZMQ_SUBSCRIBE option set to one or either of these prefixes (for instance, just `hash`); without doing so will result in no messages -arriving. Please see `contrib/zmq/zmq_sub.py` for a working example. +arriving. Please see [`contrib/zmq/zmq_sub.py`](/contrib/zmq/zmq_sub.py) for a working example. ## Remarks |