diff options
Diffstat (limited to 'doc')
43 files changed, 3604 insertions, 601 deletions
diff --git a/doc/README.md b/doc/README.md index aabfe220bc..31c95afab0 100644 --- a/doc/README.md +++ b/doc/README.md @@ -64,6 +64,7 @@ The Bitcoin repo's [root README](/README.md) contains relevant information on th - [BIPS](bips.md) - [Dnsseed Policy](dnsseed-policy.md) - [Benchmarking](benchmarking.md) +- [Internal Design Docs](design/) ### Resources * Discuss on the [BitcoinTalk](https://bitcointalk.org/) forums, in the [Development & Technical Discussion board](https://bitcointalk.org/index.php?board=6.0). @@ -72,15 +73,19 @@ The Bitcoin repo's [root README](/README.md) contains relevant information on th ### Miscellaneous - [Assets Attribution](assets-attribution.md) - [bitcoin.conf Configuration File](bitcoin-conf.md) +- [CJDNS Support](cjdns.md) - [Files](files.md) - [Fuzz-testing](fuzzing.md) - [I2P Support](i2p.md) - [Init Scripts (systemd/upstart/openrc)](init.md) - [Managing Wallets](managing-wallets.md) +- [Multisig Tutorial](multisig-tutorial.md) +- [P2P bad ports definition and list](p2p-bad-ports.md) - [PSBT support](psbt.md) - [Reduce Memory](reduce-memory.md) - [Reduce Traffic](reduce-traffic.md) - [Tor Support](tor.md) +- [Transaction Relay Policy](policy/README.md) - [ZMQ](zmq.md) License diff --git a/doc/REST-interface.md b/doc/REST-interface.md index 8b281acca7..4b46f29153 100644 --- a/doc/REST-interface.md +++ b/doc/REST-interface.md @@ -47,11 +47,31 @@ The HTTP request and response are both handled entirely in-memory. With the /notxdetails/ option JSON response will only contain the transaction hash instead of the complete transaction details. The option only affects the JSON response. #### Blockheaders -`GET /rest/headers/<COUNT>/<BLOCK-HASH>.<bin|hex|json>` +`GET /rest/headers/<BLOCK-HASH>.<bin|hex|json>?count=<COUNT=5>` Given a block hash: returns <COUNT> amount of blockheaders in upward direction. Returns empty if the block doesn't exist or it isn't in the active chain. +*Deprecated (but not removed) since 24.0:* +`GET /rest/headers/<COUNT>/<BLOCK-HASH>.<bin|hex|json>` + +#### Blockfilter Headers +`GET /rest/blockfilterheaders/<FILTERTYPE>/<BLOCK-HASH>.<bin|hex|json>?count=<COUNT=5>` + +Given a block hash: returns <COUNT> amount of blockfilter headers in upward +direction for the filter type <FILTERTYPE>. +Returns empty if the block doesn't exist or it isn't in the active chain. + +*Deprecated (but not removed) since 24.0:* +`GET /rest/blockfilterheaders/<FILTERTYPE>/<COUNT>/<BLOCK-HASH>.<bin|hex|json>` + +#### Blockfilters +`GET /rest/blockfilter/<FILTERTYPE>/<BLOCK-HASH>.<bin|hex|json>` + +Given a block hash: returns the block filter of the given block of type +<FILTERTYPE>. +Responds with 404 if the block doesn't exist. + #### Blockhash by height `GET /rest/blockhashbyheight/<HEIGHT>.<bin|hex|json>` @@ -62,17 +82,7 @@ Given a height: returns hash of block in best-block-chain at height provided. Returns various state info regarding block chain processing. Only supports JSON as output format. -* chain : (string) current network name (main, test, signet, regtest) -* blocks : (numeric) the current number of blocks processed in the server -* headers : (numeric) the current number of headers we have validated -* bestblockhash : (string) the hash of the currently best block -* difficulty : (numeric) the current difficulty -* mediantime : (numeric) the median time of the 11 blocks before the most recent block on the blockchain -* verificationprogress : (numeric) estimate of verification progress [0..1] -* chainwork : (string) total amount of work in active chain, in hexadecimal -* pruned : (boolean) if the blocks are subject to pruning -* pruneheight : (numeric) highest block available -* softforks : (array) status of softforks in progress +Refer to the `getblockchaininfo` RPC help for details. #### Query UTXO set `GET /rest/getutxos/<checkmempool>/<txid>-<n>/<txid>-<n>/.../<txid>-<n>.<bin|hex|json>` @@ -94,6 +104,7 @@ $ curl localhost:18332/rest/getutxos/checkmempool/b2cdfd7b89def827ff8af7cd9bff76 "value" : 8.8687, "scriptPubKey" : { "asm" : "OP_DUP OP_HASH160 1c7cebb529b86a04c683dfa87be49de35bcf589e OP_EQUALVERIFY OP_CHECKSIG", + "desc" : "addr(mi7as51dvLJsizWnTMurtRmrP8hG2m1XvD)#gj9tznmy" "hex" : "76a9141c7cebb529b86a04c683dfa87be49de35bcf589e88ac", "type" : "pubkeyhash", "address" : "mi7as51dvLJsizWnTMurtRmrP8hG2m1XvD" @@ -106,14 +117,15 @@ $ curl localhost:18332/rest/getutxos/checkmempool/b2cdfd7b89def827ff8af7cd9bff76 #### Memory pool `GET /rest/mempool/info.json` -Returns various information about the TX mempool. +Returns various information about the transaction mempool. Only supports JSON as output format. -Refer to the `getmempoolinfo` RPC for documentation of the fields. +Refer to the `getmempoolinfo` RPC help for details. `GET /rest/mempool/contents.json` -Returns transactions in the TX mempool. +Returns the transactions in the mempool. Only supports JSON as output format. +Refer to the `getrawmempool` RPC help for details. Risks ------------- diff --git a/doc/bips.md b/doc/bips.md index 45954bcfd8..0f3f61daf1 100644 --- a/doc/bips.md +++ b/doc/bips.md @@ -1,4 +1,4 @@ -BIPs that are implemented by Bitcoin Core (up-to-date up to **v22.0**): +BIPs that are implemented by Bitcoin Core (up-to-date up to **v23.0**): * [`BIP 9`](https://github.com/bitcoin/bips/blob/master/bip-0009.mediawiki): The changes allowing multiple soft-forks to be deployed in parallel have been implemented since **v0.12.1** ([PR #7575](https://github.com/bitcoin/bitcoin/pull/7575)) * [`BIP 11`](https://github.com/bitcoin/bips/blob/master/bip-0011.mediawiki): Multisig outputs are standard since **v0.6.0** ([PR #669](https://github.com/bitcoin/bitcoin/pull/669)). @@ -32,7 +32,7 @@ BIPs that are implemented by Bitcoin Core (up-to-date up to **v22.0**): * [`BIP 111`](https://github.com/bitcoin/bips/blob/master/bip-0111.mediawiki): `NODE_BLOOM` service bit added, and enforced for all peer versions as of **v0.13.0** ([PR #6579](https://github.com/bitcoin/bitcoin/pull/6579) and [PR #6641](https://github.com/bitcoin/bitcoin/pull/6641)). * [`BIP 112`](https://github.com/bitcoin/bips/blob/master/bip-0112.mediawiki): The CHECKSEQUENCEVERIFY opcode has been implemented since **v0.12.1** ([PR #7524](https://github.com/bitcoin/bitcoin/pull/7524)), and has been *buried* since **v0.19.0** ([PR #16060](https://github.com/bitcoin/bitcoin/pull/16060)). * [`BIP 113`](https://github.com/bitcoin/bips/blob/master/bip-0113.mediawiki): Median time past lock-time calculations have been implemented since **v0.12.1** ([PR #6566](https://github.com/bitcoin/bitcoin/pull/6566)), and has been *buried* since **v0.19.0** ([PR #16060](https://github.com/bitcoin/bitcoin/pull/16060)). -* [`BIP 125`](https://github.com/bitcoin/bips/blob/master/bip-0125.mediawiki): Opt-in full replace-by-fee signaling honoured in mempool and mining as of **v0.12.0** ([PR 6871](https://github.com/bitcoin/bitcoin/pull/6871)). Enabled by default in the wallet GUI as of **v0.18.1** ([PR #11605](https://github.com/bitcoin/bitcoin/pull/11605)) +* [`BIP 125`](https://github.com/bitcoin/bips/blob/master/bip-0125.mediawiki): Opt-in full replace-by-fee signaling partially implemented. See doc/policy/mempool-replacements.md. * [`BIP 130`](https://github.com/bitcoin/bips/blob/master/bip-0130.mediawiki): direct headers announcement is negotiated with peer versions `>=70012` as of **v0.12.0** ([PR 6494](https://github.com/bitcoin/bitcoin/pull/6494)). * [`BIP 133`](https://github.com/bitcoin/bips/blob/master/bip-0133.mediawiki): feefilter messages are respected and sent for peer versions `>=70013` as of **v0.13.0** ([PR 7542](https://github.com/bitcoin/bitcoin/pull/7542)). * [`BIP 141`](https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki): Segregated Witness (Consensus Layer) as of **v0.13.0** ([PR 8149](https://github.com/bitcoin/bitcoin/pull/8149)), defined for mainnet as of **v0.13.1** ([PR 8937](https://github.com/bitcoin/bitcoin/pull/8937)), and *buried* since **v0.19.0** ([PR #16060](https://github.com/bitcoin/bitcoin/pull/16060)). @@ -42,7 +42,8 @@ BIPs that are implemented by Bitcoin Core (up-to-date up to **v22.0**): * [`BIP 147`](https://github.com/bitcoin/bips/blob/master/bip-0147.mediawiki): NULLDUMMY softfork as of **v0.13.1** ([PR 8636](https://github.com/bitcoin/bitcoin/pull/8636) and [PR 8937](https://github.com/bitcoin/bitcoin/pull/8937)), *buried* since **v0.19.0** ([PR #16060](https://github.com/bitcoin/bitcoin/pull/16060)). * [`BIP 152`](https://github.com/bitcoin/bips/blob/master/bip-0152.mediawiki): Compact block transfer and related optimizations are used as of **v0.13.0** ([PR 8068](https://github.com/bitcoin/bitcoin/pull/8068)). * [`BIP 155`](https://github.com/bitcoin/bips/blob/master/bip-0155.mediawiki): The 'addrv2' and 'sendaddrv2' messages which enable relay of Tor V3 addresses (and other networks) are supported as of **v0.21.0** ([PR 19954](https://github.com/bitcoin/bitcoin/pull/19954)). -* [`BIP 158`](https://github.com/bitcoin/bips/blob/master/bip-0158.mediawiki): Compact Block Filters for Light Clients can be indexed as of **v0.19.0** ([PR #14121](https://github.com/bitcoin/bitcoin/pull/14121)). +* [`BIP 157`](https://github.com/bitcoin/bips/blob/master/bip-0157.mediawiki) + [`158`](https://github.com/bitcoin/bips/blob/master/bip-0158.mediawiki): Compact Block Filters for Light Clients can be indexed as of **v0.19.0** ([PR #14121](https://github.com/bitcoin/bitcoin/pull/14121)) and served to peers on the P2P network as of **v0.21.0** ([PR #16442](https://github.com/bitcoin/bitcoin/pull/16442)). * [`BIP 159`](https://github.com/bitcoin/bips/blob/master/bip-0159.mediawiki): The `NODE_NETWORK_LIMITED` service bit is signalled as of **v0.16.0** ([PR 11740](https://github.com/bitcoin/bitcoin/pull/11740)), and such nodes are connected to as of **v0.17.0** ([PR 10387](https://github.com/bitcoin/bitcoin/pull/10387)). * [`BIP 173`](https://github.com/bitcoin/bips/blob/master/bip-0173.mediawiki): Bech32 addresses for native Segregated Witness outputs are supported as of **v0.16.0** ([PR 11167](https://github.com/bitcoin/bitcoin/pull/11167)). Bech32 addresses are generated by default as of **v0.20.0** ([PR 16884](https://github.com/bitcoin/bitcoin/pull/16884)). * [`BIP 174`](https://github.com/bitcoin/bips/blob/master/bip-0174.mediawiki): RPCs to operate on Partially Signed Bitcoin Transactions (PSBT) are present as of **v0.17.0** ([PR 13557](https://github.com/bitcoin/bitcoin/pull/13557)). @@ -57,3 +58,11 @@ BIPs that are implemented by Bitcoin Core (up-to-date up to **v22.0**): with mainnet activation as of **v0.21.1** ([PR 21377](https://github.com/bitcoin/bitcoin/pull/21377), [PR 21686](https://github.com/bitcoin/bitcoin/pull/21686)). * [`BIP 350`](https://github.com/bitcoin/bips/blob/master/bip-0350.mediawiki): Addresses for native v1+ segregated Witness outputs use Bech32m instead of Bech32 as of **v22.0** ([PR 20861](https://github.com/bitcoin/bitcoin/pull/20861)). +* [`BIP 380`](https://github.com/bitcoin/bips/blob/master/bip-0380.mediawiki) + [`381`](https://github.com/bitcoin/bips/blob/master/bip-0381.mediawiki) + [`382`](https://github.com/bitcoin/bips/blob/master/bip-0382.mediawiki) + [`383`](https://github.com/bitcoin/bips/blob/master/bip-0383.mediawiki) + [`384`](https://github.com/bitcoin/bips/blob/master/bip-0384.mediawiki) + [`385`](https://github.com/bitcoin/bips/blob/master/bip-0385.mediawiki): + Output Script Descriptors, and most of Script Expressions are implemented as of **v0.17.0** ([PR 13697](https://github.com/bitcoin/bitcoin/pull/13697)). +* [`BIP 386`](https://github.com/bitcoin/bips/blob/master/bip-0386.mediawiki): tr() Output Script Descriptors are implemented as of **v22.0** ([PR 22051](https://github.com/bitcoin/bitcoin/pull/22051)). diff --git a/doc/bitcoin-conf.md b/doc/bitcoin-conf.md index 8c9035c45b..acd767f234 100644 --- a/doc/bitcoin-conf.md +++ b/doc/bitcoin-conf.md @@ -61,4 +61,12 @@ Windows | `%APPDATA%\Bitcoin\` | `C:\Users\username\AppData\Roaming\Bitcoin\bitc Linux | `$HOME/.bitcoin/` | `/home/username/.bitcoin/bitcoin.conf` macOS | `$HOME/Library/Application Support/Bitcoin/` | `/Users/username/Library/Application Support/Bitcoin/bitcoin.conf` -You can find an example bitcoin.conf file in [share/examples/bitcoin.conf](../share/examples/bitcoin.conf). +An example configuration file can be generated by [contrib/devtools/gen-bitcoin-conf.sh](../contrib/devtools/gen-bitcoin-conf.sh). +Run this script after compiling to generate an up-to-date configuration file. +The output is placed under `share/examples/bitcoin.conf`. +To use the generated configuration file, copy the example file into your data directory and edit it there, like so: + +``` +# example copy command for linux user +cp share/examples/bitcoin.conf ~/.bitcoin +``` diff --git a/doc/build-android.md b/doc/build-android.md index 7a8a9e6a65..2f2e01c441 100644 --- a/doc/build-android.md +++ b/doc/build-android.md @@ -3,9 +3,22 @@ ANDROID BUILD NOTES This guide describes how to build and package the `bitcoin-qt` GUI for Android on Linux and macOS. -## Preparation -You will need to get the Android NDK and build dependencies for Android as described in [depends/README.md](../depends/README.md). +## Dependencies + +Before proceeding with an Android build one needs to get the [Android SDK](https://developer.android.com/studio) and use the "SDK Manager" tool to download the NDK and one or more "Platform packages" (these are Android versions and have a corresponding API level). + +The minimum supported Android NDK version is [r23](https://github.com/android/ndk/wiki/Changelog-r23). + +In order to build `ANDROID_API_LEVEL` (API level corresponding to the Android version targeted, e.g. Android 9.0 Pie is 28 and its "Platform package" needs to be available) and `ANDROID_TOOLCHAIN_BIN` (path to toolchain binaries depending on the platform the build is being performed on) need to be set. + +API levels from 24 to 29 have been tested to work. + +If the build includes Qt, environment variables `ANDROID_SDK` and `ANDROID_NDK` need to be set as well but can otherwise be omitted. +This is an example command for a default build with no disabled dependencies: + + ANDROID_SDK=/home/user/Android/Sdk ANDROID_NDK=/home/user/Android/Sdk/ndk-bundle make HOST=aarch64-linux-android ANDROID_API_LEVEL=28 ANDROID_TOOLCHAIN_BIN=/home/user/Android/Sdk/ndk-bundle/toolchains/llvm/prebuilt/linux-x86_64/bin + ## Building and packaging diff --git a/doc/build-freebsd.md b/doc/build-freebsd.md index da2ab61c2a..a8e643a2ab 100644 --- a/doc/build-freebsd.md +++ b/doc/build-freebsd.md @@ -1,48 +1,21 @@ # FreeBSD Build Guide -**Updated for FreeBSD [12.2](https://www.freebsd.org/releases/12.2R/announce.html)** +**Updated for FreeBSD [12.3](https://www.freebsd.org/releases/12.3R/announce/)** This guide describes how to build bitcoind, command-line utilities, and GUI on FreeBSD. -## Dependencies - -The following dependencies are **required**: - - Library | Purpose | Description - ----------------------------------------------------------------------|------------|---------------------- - [autoconf](https://svnweb.freebsd.org/ports/head/devel/autoconf/) | Build | Automatically configure software source code - [automake](https://svnweb.freebsd.org/ports/head/devel/automake/) | Build | Generate makefile (requires autoconf) - [libtool](https://svnweb.freebsd.org/ports/head/devel/libtool/) | Build | Shared library support - [pkgconf](https://svnweb.freebsd.org/ports/head/devel/pkgconf/) | Build | Configure compiler and linker flags - [git](https://svnweb.freebsd.org/ports/head/devel/git/) | Clone | Version control system - [gmake](https://svnweb.freebsd.org/ports/head/devel/gmake/) | Compile | Generate executables - [boost-libs](https://svnweb.freebsd.org/ports/head/devel/boost-libs/) | Utility | Library for threading, data structures, etc - [libevent](https://svnweb.freebsd.org/ports/head/devel/libevent/) | Networking | OS independent asynchronous networking - - -The following dependencies are **optional**: - - Library | Purpose | Description - ---------------------------------------------------------------------------|------------------|---------------------- - [db5](https://svnweb.freebsd.org/ports/head/databases/db5/) | Berkeley DB | Wallet storage (only needed when wallet enabled) - [qt5](https://svnweb.freebsd.org/ports/head/devel/qt5/) | GUI | GUI toolkit (only needed when GUI enabled) - [libqrencode](https://svnweb.freebsd.org/ports/head/graphics/libqrencode/) | QR codes in GUI | Generating QR codes (only needed when GUI enabled) - [libzmq4](https://svnweb.freebsd.org/ports/head/net/libzmq4/) | ZMQ notification | Allows generating ZMQ notifications (requires ZMQ version >= 4.0.0) - [sqlite3](https://svnweb.freebsd.org/ports/head/databases/sqlite3/) | SQLite DB | Wallet storage (only needed when wallet enabled) - [python3](https://svnweb.freebsd.org/ports/head/lang/python3/) | Testing | Python Interpreter (only needed when running the test suite) - - See [dependencies.md](dependencies.md) for a complete overview. - ## Preparation ### 1. Install Required Dependencies -Install the required dependencies the usual way you [install software on FreeBSD](https://www.freebsd.org/doc/en/books/handbook/ports.html) - either with `pkg` or via the Ports collection. The example commands below use `pkg` which is usually run as `root` or via `sudo`. If you want to use `sudo`, and you haven't set it up: [use this guide](http://www.freebsdwiki.net/index.php/Sudo%2C_configuring) to setup `sudo` access on FreeBSD. +Run the following as root to install the base dependencies for building. ```bash pkg install autoconf automake boost-libs git gmake libevent libtool pkgconf ``` +See [dependencies.md](dependencies.md) for a complete overview. + ### 2. Clone Bitcoin Repo Now that `git` and all the required dependencies are installed, let's clone the Bitcoin Core repository to a directory. All build scripts and commands will run from this directory. ``` bash @@ -52,21 +25,23 @@ git clone https://github.com/bitcoin/bitcoin.git ### 3. Install Optional Dependencies #### Wallet Dependencies -It is not necessary to build wallet functionality to run bitcoind or the GUI. To enable legacy wallets, you must install `db5`. To enable [descriptor wallets](https://github.com/bitcoin/bitcoin/blob/master/doc/descriptors.md), `sqlite3` is required. Skip `db5` if you intend to *exclusively* use descriptor wallets - -###### Legacy Wallet Support -`db5` is required to enable support for legacy wallets. Skip if you don't intend to use legacy wallets - -```bash -pkg install db5 -``` +It is not necessary to build wallet functionality to run either `bitcoind` or `bitcoin-qt`. ###### Descriptor Wallet Support -`sqlite3` is required to enable support for descriptor wallets. Skip if you don't intend to use descriptor wallets. +`sqlite3` is required to support [descriptor wallets](descriptors.md). +Skip if you don't intend to use descriptor wallets. ``` bash pkg install sqlite3 ``` + +###### Legacy Wallet Support +`db5` is only required to support legacy wallets. +Skip if you don't intend to use legacy wallets. + +```bash +pkg install db5 +``` --- #### GUI Dependencies @@ -84,6 +59,14 @@ pkg install libqrencode ``` --- +#### Notifications +###### ZeroMQ + +Bitcoin Core can provide notifications via ZeroMQ. If the package is installed, support will be compiled in. +```bash +pkg install libzmq4 +``` + #### Test Suite Dependencies There is an included test suite that is useful for testing code changes when developing. To run the test suite (recommended), you will need to have Python 3 installed: @@ -98,8 +81,17 @@ pkg install python3 ### 1. Configuration There are many ways to configure Bitcoin Core, here are a few common examples: -##### Wallet (BDB + SQlite) Support, No GUI: -This explicitly enables legacy wallet support and disables the GUI. If `sqlite3` is installed, then descriptor wallet support will be built. + +##### Descriptor Wallet and GUI: +This explicitly enables the GUI and disables legacy wallet support, assuming `sqlite` and `qt` are installed. +```bash +./autogen.sh +./configure --without-bdb --with-gui=yes MAKE=gmake +``` + +##### Descriptor & Legacy Wallet. No GUI: +This enables support for both wallet types and disables the GUI, assuming +`sqlite3` and `db5` are both installed. ```bash ./autogen.sh ./configure --with-gui=no --with-incompatible-bdb \ @@ -108,12 +100,6 @@ This explicitly enables legacy wallet support and disables the GUI. If `sqlite3` MAKE=gmake ``` -##### Wallet (only SQlite) and GUI Support: -This explicitly enables the GUI and disables legacy wallet support. If `qt5` is not installed, this will throw an error. If `sqlite3` is installed then descriptor wallet functionality will be built. If `sqlite3` is not installed, then wallet functionality will be disabled. -```bash -./autogen.sh -./configure --without-bdb --with-gui=yes MAKE=gmake -``` ##### No Wallet or GUI ``` bash ./autogen.sh diff --git a/doc/build-netbsd.md b/doc/build-netbsd.md index edabd71611..9cec201faf 100644 --- a/doc/build-netbsd.md +++ b/doc/build-netbsd.md @@ -1,6 +1,6 @@ -NetBSD build guide +NetBSD Build Guide ====================== -(updated for NetBSD 8.0) +**Updated for NetBSD [8.0](https://www.netbsd.org/releases/formal-8/NetBSD-8.0.html)** This guide describes how to build bitcoind and command-line utilities on NetBSD. @@ -27,15 +27,33 @@ git clone https://github.com/bitcoin/bitcoin.git See [dependencies.md](dependencies.md) for a complete overview. -### Building BerkeleyDB +### Building Bitcoin Core + +**Important**: Use `gmake` (the non-GNU `make` will exit with an error). + +#### With descriptor wallet: + +The descriptor wallet uses `sqlite3`. You can install it using: +```bash +pkgin install sqlite3 +``` + +```bash +./autogen.sh +./configure --with-gui=no --without-bdb \ + CPPFLAGS="-I/usr/pkg/include" \ + LDFLAGS="-L/usr/pkg/lib" \ + BOOST_CPPFLAGS="-I/usr/pkg/include" \ + MAKE=gmake +``` -BerkeleyDB is only necessary for the wallet functionality. To skip this, pass -`--disable-wallet` to `./configure` and skip to the next section. +#### With legacy wallet: + +BerkeleyDB is use for legacy wallet functionality. It is recommended to use Berkeley DB 4.8. You cannot use the BerkeleyDB library -from ports, for the same reason as boost above (g++/libstd++ incompatibility). -If you have to build it yourself, you can use [the installation script included -in contrib/](/contrib/install_db4.sh) like so: +from ports. +You can use [the installation script included in contrib/](/contrib/install_db4.sh) like so: ```bash ./contrib/install_db4.sh `pwd` @@ -47,30 +65,23 @@ from the root of the repository. Then set `BDB_PREFIX` for the next section: export BDB_PREFIX="$PWD/db4" ``` -### Building Bitcoin Core - -**Important**: Use `gmake` (the non-GNU `make` will exit with an error). - -With wallet: ```bash ./autogen.sh ./configure --with-gui=no CPPFLAGS="-I/usr/pkg/include" \ LDFLAGS="-L/usr/pkg/lib" \ BOOST_CPPFLAGS="-I/usr/pkg/include" \ - BOOST_LDFLAGS="-L/usr/pkg/lib" \ BDB_LIBS="-L${BDB_PREFIX}/lib -ldb_cxx-4.8" \ BDB_CFLAGS="-I${BDB_PREFIX}/include" \ MAKE=gmake ``` -Without wallet: +#### Without wallet: ```bash ./autogen.sh ./configure --with-gui=no --disable-wallet \ CPPFLAGS="-I/usr/pkg/include" \ LDFLAGS="-L/usr/pkg/lib" \ BOOST_CPPFLAGS="-I/usr/pkg/include" \ - BOOST_LDFLAGS="-L/usr/pkg/lib" \ MAKE=gmake ``` diff --git a/doc/build-openbsd.md b/doc/build-openbsd.md index 6e54f67edc..afbb5c8e75 100644 --- a/doc/build-openbsd.md +++ b/doc/build-openbsd.md @@ -1,123 +1,130 @@ -OpenBSD build guide -====================== -(updated for OpenBSD 6.9) +# OpenBSD Build Guide -This guide describes how to build bitcoind, bitcoin-qt, and command-line utilities on OpenBSD. +**Updated for OpenBSD [7.1](https://www.openbsd.org/71.html)** -Preparation -------------- +This guide describes how to build bitcoind, command-line utilities, and GUI on OpenBSD. -Run the following as root to install the base dependencies for building: +## Preparation + +### 1. Install Required Dependencies +Run the following as root to install the base dependencies for building. ```bash -pkg_add git gmake libevent libtool boost -pkg_add qt5 # (optional for enabling the GUI) -pkg_add autoconf # (select highest version, e.g. 2.69) -pkg_add automake # (select highest version, e.g. 1.16) -pkg_add python # (select highest version, e.g. 3.8) -pkg_add bash +pkg_add bash git gmake libevent libtool boost +# Select the newest version of the following packages: +pkg_add autoconf automake python +``` +See [dependencies.md](dependencies.md) for a complete overview. + +### 2. Clone Bitcoin Repo +Clone the Bitcoin Core repository to a directory. All build scripts and commands will run from this directory. +``` bash git clone https://github.com/bitcoin/bitcoin.git ``` -See [dependencies.md](dependencies.md) for a complete overview. +### 3. Install Optional Dependencies -**Important**: From OpenBSD 6.2 onwards a C++11-supporting clang compiler is -part of the base image, and while building it is necessary to make sure that -this compiler is used and not ancient g++ 4.2.1. This is done by appending -`CC=cc CC_FOR_BUILD=cc CXX=c++` to configuration commands. Mixing different -compilers within the same executable will result in errors. +#### Wallet Dependencies -### Building BerkeleyDB +It is not necessary to build wallet functionality to run either `bitcoind` or `bitcoin-qt`. + +###### Descriptor Wallet Support + +`sqlite3` is required to support [descriptor wallets](descriptors.md). + +``` bash +pkg_add sqlite3 +``` -BerkeleyDB is only necessary for the wallet functionality. To skip this, pass -`--disable-wallet` to `./configure` and skip to the next section. +###### Legacy Wallet Support +BerkeleyDB is only required to support legacy wallets. It is recommended to use Berkeley DB 4.8. You cannot use the BerkeleyDB library -from ports, for the same reason as boost above (g++/libstd++ incompatibility). -If you have to build it yourself, you can use [the installation script included -in contrib/](/contrib/install_db4.sh) like so: +from ports. However you can build it yourself, [using the installation script included in contrib/](/contrib/install_db4.sh), like so, from the root of the repository. ```bash -./contrib/install_db4.sh `pwd` CC=cc CXX=c++ +./contrib/install_db4.sh `pwd` ``` -from the root of the repository. Then set `BDB_PREFIX` for the next section: +Then set `BDB_PREFIX`: ```bash export BDB_PREFIX="$PWD/db4" ``` -### Building Bitcoin Core +#### GUI Dependencies +###### Qt5 + +Bitcoin Core includes a GUI built with the cross-platform Qt Framework. To compile the GUI, Qt 5 is required. + +```bash +pkg_add qt5 +``` + +## Building Bitcoin Core **Important**: Use `gmake` (the non-GNU `make` will exit with an error). Preparation: ```bash -# Replace this with the autoconf version that you installed. Include only -# the major and minor parts of the version: use "2.69" for "autoconf-2.69p2". -export AUTOCONF_VERSION=2.69 - -# Replace this with the automake version that you installed. Include only -# the major and minor parts of the version: use "1.16" for "automake-1.16.1". +# Adapt the following for the version you installed (major.minor only): +export AUTOCONF_VERSION=2.71 export AUTOMAKE_VERSION=1.16 ./autogen.sh ``` -Make sure `BDB_PREFIX` is set to the appropriate path from the above steps. -Note that building with external signer support currently fails on OpenBSD, -hence you have to explicitly disable it by passing the parameter -`--disable-external-signer` to the configure script. -(Background: the feature requires the header-only library boost::process, which -is available on OpenBSD 6.9 via Boost 1.72.0, but contains certain system calls -and preprocessor defines like `waitid()` and `WEXITED` that are not available.) +### 1. Configuration -To configure with wallet: -```bash -./configure --with-gui=no --disable-external-signer CC=cc CXX=c++ \ - BDB_LIBS="-L${BDB_PREFIX}/lib -ldb_cxx-4.8" \ - BDB_CFLAGS="-I${BDB_PREFIX}/include" \ - MAKE=gmake -``` +Note that external signer support is currently not available on OpenBSD, since +the used header-only library Boost.Process fails to compile (certain system +calls and preprocessor defines like `waitid()` and `WEXITED` are missing). + +There are many ways to configure Bitcoin Core, here are a few common examples: + +##### Descriptor Wallet and GUI: +This enables the GUI and descriptor wallet support, assuming `sqlite` and `qt5` are installed. -To configure without wallet: ```bash -./configure --disable-wallet --with-gui=no --disable-external-signer CC=cc CC_FOR_BUILD=cc CXX=c++ MAKE=gmake +./configure MAKE=gmake ``` -To configure with GUI: +##### Descriptor & Legacy Wallet. No GUI: +This enables support for both wallet types and disables the GUI: + ```bash -./configure --with-gui=yes --disable-external-signer CC=cc CXX=c++ \ +./configure --with-gui=no \ BDB_LIBS="-L${BDB_PREFIX}/lib -ldb_cxx-4.8" \ BDB_CFLAGS="-I${BDB_PREFIX}/include" \ MAKE=gmake ``` -Build and run the tests: +### 2. Compile +**Important**: Use `gmake` (the non-GNU `make` will exit with an error). + ```bash -gmake # use "-j N" here for N parallel jobs -gmake check +gmake # use "-j N" for N parallel jobs +gmake check # Run tests if Python 3 is available ``` -Resource limits -------------------- +## Resource limits If the build runs into out-of-memory errors, the instructions in this section might help. The standard ulimit restrictions in OpenBSD are very strict: - - data(kbytes) 1572864 +```bash +data(kbytes) 1572864 +``` This is, unfortunately, in some cases not enough to compile some `.cpp` files in the project, (see issue [#6658](https://github.com/bitcoin/bitcoin/issues/6658)). If your user is in the `staff` group the limit can be raised with: - - ulimit -d 3000000 - +```bash +ulimit -d 3000000 +``` The change will only affect the current shell and processes spawned by it. To make the change system-wide, change `datasize-cur` and `datasize-max` in `/etc/login.conf`, and reboot. - diff --git a/doc/build-osx.md b/doc/build-osx.md index 467feff410..fdf0a9d414 100644 --- a/doc/build-osx.md +++ b/doc/build-osx.md @@ -4,43 +4,6 @@ This guide describes how to build bitcoind, command-line utilities, and GUI on macOS -**Note:** The following is for Intel Macs only! - -## Dependencies - -The following dependencies are **required**: - -Library | Purpose | Description ------------------------------------------------------------|------------|---------------------- -[automake](https://formulae.brew.sh/formula/automake) | Build | Generate makefile -[libtool](https://formulae.brew.sh/formula/libtool) | Build | Shared library support -[pkg-config](https://formulae.brew.sh/formula/pkg-config) | Build | Configure compiler and linker flags -[boost](https://formulae.brew.sh/formula/boost) | Utility | Library for threading, data structures, etc -[libevent](https://formulae.brew.sh/formula/libevent) | Networking | OS independent asynchronous networking - -The following dependencies are **optional**: - -Library | Purpose | Description ---------------------------------------------------------------- |------------------|---------------------- -[berkeley-db@4](https://formulae.brew.sh/formula/berkeley-db@4) | Berkeley DB | Wallet storage (only needed when wallet enabled) -[qt@5](https://formulae.brew.sh/formula/qt@5) | GUI | GUI toolkit (only needed when GUI enabled) -[qrencode](https://formulae.brew.sh/formula/qrencode) | QR codes in GUI | Generating QR codes (only needed when GUI enabled) -[zeromq](https://formulae.brew.sh/formula/zeromq) | ZMQ notification | Allows generating ZMQ notifications (requires ZMQ version >= 4.0.0) -[sqlite](https://formulae.brew.sh/formula/sqlite) | SQLite DB | Wallet storage (only needed when wallet enabled) -[miniupnpc](https://formulae.brew.sh/formula/miniupnpc) | UPnP Support | Firewall-jumping support (needed for port mapping support) -[libnatpmp](https://formulae.brew.sh/formula/libnatpmp) | NAT-PMP Support | Firewall-jumping support (needed for port mapping support) -[python3](https://formulae.brew.sh/formula/python@3.9) | Testing | Python Interpreter (only needed when running the test suite) - -The following dependencies are **optional** packages required for deploying: - -Library | Purpose | Description -----------------------------------------------------|------------------|---------------------- -[librsvg](https://formulae.brew.sh/formula/librsvg) | Deploy Dependency| Library to render SVG files -[ds_store](https://pypi.org/project/ds-store/) | Deploy Dependency| Examine and modify .DS_Store files -[mac_alias](https://pypi.org/project/mac-alias/) | Deploy Dependency| Generate/Read binary alias and bookmark records - -See [dependencies.md](dependencies.md) for a complete overview. - ## Preparation The commands in this guide should be executed in a Terminal application. @@ -79,6 +42,9 @@ Note: If you run into issues while installing Homebrew or pulling packages, refe The first step is to download the required dependencies. These dependencies represent the packages required to get a barebones installation up and running. + +See [dependencies.md](dependencies.md) for a complete overview. + To install, run the following from your terminal: ``` bash @@ -100,29 +66,21 @@ git clone https://github.com/bitcoin/bitcoin.git #### Wallet Dependencies It is not necessary to build wallet functionality to run `bitcoind` or `bitcoin-qt`. -To enable legacy wallets, you must install `berkeley-db@4`. -To enable [descriptor wallets](https://github.com/bitcoin/bitcoin/blob/master/doc/descriptors.md), `sqlite` is required. -Skip `berkeley-db@4` if you intend to *exclusively* use descriptor wallets. -###### Legacy Wallet Support +###### Descriptor Wallet Support -`berkeley-db@4` is required to enable support for legacy wallets. -Skip if you don't intend to use legacy wallets. +`sqlite` is required to support for descriptor wallets. -``` bash -brew install berkeley-db@4 -``` - -###### Descriptor Wallet Support +macOS ships with a useable `sqlite` package, meaning you don't need to +install anything. -Note: Apple has included a useable `sqlite` package since macOS 10.14. -You may not need to install this package. +###### Legacy Wallet Support -`sqlite` is required to enable support for descriptor wallets. -Skip if you don't intend to use descriptor wallets. +`berkeley-db@4` is only required to support for legacy wallets. +Skip if you don't intend to use legacy wallets. ``` bash -brew install sqlite +brew install berkeley-db@4 ``` --- @@ -219,10 +177,6 @@ This command depends on a couple of python packages, so it is required that you Ensuring that `python` is installed, you can install the deploy dependencies by running the following commands in your terminal: ``` bash -brew install librsvg -``` - -``` bash pip3 install ds_store mac_alias ``` diff --git a/doc/build-unix.md b/doc/build-unix.md index 44b6ad5968..bcfa25dc76 100644 --- a/doc/build-unix.md +++ b/doc/build-unix.md @@ -26,31 +26,7 @@ make install # optional This will build bitcoin-qt as well, if the dependencies are met. -Dependencies ---------------------- - -These dependencies are required: - - Library | Purpose | Description - ------------|------------------|---------------------- - libboost | Utility | Library for threading, data structures, etc - libevent | Networking | OS independent asynchronous networking - -Optional dependencies: - - Library | Purpose | Description - ------------|------------------|---------------------- - miniupnpc | UPnP Support | Firewall-jumping support - libnatpmp | NAT-PMP Support | Firewall-jumping support - libdb4.8 | Berkeley DB | Optional, wallet storage (only needed when wallet enabled) - qt | GUI | GUI toolkit (only needed when GUI enabled) - libqrencode | QR codes in GUI | Optional for generating QR codes (only needed when GUI enabled) - univalue | Utility | JSON parsing and encoding (bundled version will be used unless --with-system-univalue passed to configure) - libzmq3 | ZMQ notification | Optional, allows generating ZMQ notifications (requires ZMQ version >= 4.0.0) - sqlite3 | SQLite DB | Optional, wallet storage (only needed when wallet enabled) - systemtap | Tracing (USDT) | Optional, statically defined tracepoints - -For the versions used, see [dependencies.md](dependencies.md) +See [dependencies.md](dependencies.md) for a complete overview. Memory Requirements -------------------- @@ -83,21 +59,17 @@ Build requirements: Now, you can either build from self-compiled [depends](/depends/README.md) or install the required dependencies: - sudo apt-get install libevent-dev libboost-dev libboost-system-dev libboost-filesystem-dev libboost-test-dev - -Berkeley DB is required for the wallet. - -Ubuntu and Debian have their own `libdb-dev` and `libdb++-dev` packages, but these will install -Berkeley DB 5.1 or later. This will break binary wallet compatibility with the distributed executables, which -are based on BerkeleyDB 4.8. If you do not care about wallet compatibility, -pass `--with-incompatible-bdb` to configure. - -Otherwise, you can build Berkeley DB [yourself](#berkeley-db). + sudo apt-get install libevent-dev libboost-dev SQLite is required for the descriptor wallet: sudo apt install libsqlite3-dev +Berkeley DB is required for the legacy wallet. Ubuntu and Debian have their own `libdb-dev` and `libdb++-dev` packages, +but these will install Berkeley DB 5.1 or later. This will break binary wallet compatibility with the distributed +executables, which are based on BerkeleyDB 4.8. If you do not care about wallet compatibility, pass +`--with-incompatible-bdb` to configure. Otherwise, you can build Berkeley DB [yourself](#berkeley-db). + To build Bitcoin Core without wallet, see [*Disable-wallet mode*](#disable-wallet-mode) Optional port mapping libraries (see: `--with-miniupnpc`, `--enable-upnp-default`, and `--with-natpmp`, `--enable-natpmp-default`): @@ -146,20 +118,18 @@ Now, you can either build from self-compiled [depends](/depends/README.md) or in sudo dnf install libevent-devel boost-devel -Berkeley DB is required for the wallet: +SQLite is required for the descriptor wallet: + + sudo dnf install sqlite-devel + +Berkeley DB is required for the legacy wallet: sudo dnf install libdb4-devel libdb4-cxx-devel Newer Fedora releases, since Fedora 33, have only `libdb-devel` and `libdb-cxx-devel` packages, but these will install Berkeley DB 5.3 or later. This will break binary wallet compatibility with the distributed executables, which are based on Berkeley DB 4.8. If you do not care about wallet compatibility, -pass `--with-incompatible-bdb` to configure. - -Otherwise, you can build Berkeley DB [yourself](#berkeley-db). - -SQLite is required for the descriptor wallet: - - sudo dnf install sqlite-devel +pass `--with-incompatible-bdb` to configure. Otherwise, you can build Berkeley DB [yourself](#berkeley-db). To build Bitcoin Core without wallet, see [*Disable-wallet mode*](#disable-wallet-mode) @@ -225,8 +195,10 @@ turned off by default. See the configure options for NAT-PMP behavior desired: Berkeley DB ----------- -It is recommended to use Berkeley DB 4.8. If you have to build it yourself, -you can use [the installation script included in contrib/](/contrib/install_db4.sh) + +The legacy wallet uses Berkeley DB. To ensure backwards compatibility it is +recommended to use Berkeley DB 4.8. If you have to build it yourself, you can +use [the installation script included in contrib/](/contrib/install_db4.sh) like so: ```shell @@ -237,16 +209,7 @@ from the root of the repository. Otherwise, you can build Bitcoin Core from self-compiled [depends](/depends/README.md). -**Note**: You only need Berkeley DB if the wallet is enabled (see [*Disable-wallet mode*](#disable-wallet-mode)). - -Boost ------ -If you need to build Boost yourself: - - sudo su - ./bootstrap.sh - ./bjam install - +**Note**: You only need Berkeley DB if the legacy wallet is enabled (see [*Disable-wallet mode*](#disable-wallet-mode)). Security -------- @@ -296,12 +259,12 @@ Hardening enables the following features: Disable-wallet mode -------------------- -When the intention is to run only a P2P node without a wallet, Bitcoin Core may be compiled in -disable-wallet mode with: +When the intention is to only run a P2P node, without a wallet, Bitcoin Core can +be compiled in disable-wallet mode with: ./configure --disable-wallet -In this case there is no dependency on Berkeley DB 4.8 and SQLite. +In this case there is no dependency on SQLite or Berkeley DB. Mining is also possible in disable-wallet mode using the `getblocktemplate` RPC call. @@ -314,22 +277,17 @@ A list of additional configure flags can be displayed with: Setup and Build Example: Arch Linux ----------------------------------- -This example lists the steps necessary to setup and build a command line only, non-wallet distribution of the latest changes on Arch Linux: +This example lists the steps necessary to setup and build a command line only distribution of the latest changes on Arch Linux: - pacman -S git base-devel boost libevent python + pacman --sync --needed autoconf automake boost gcc git libevent libtool make pkgconf python sqlite git clone https://github.com/bitcoin/bitcoin.git cd bitcoin/ ./autogen.sh - ./configure --disable-wallet --without-gui --without-miniupnpc + ./configure make check + ./src/bitcoind -Note: -Enabling wallet support requires either compiling against a Berkeley DB newer than 4.8 (package `db`) using `--with-incompatible-bdb`, -or building and depending on a local version of Berkeley DB 4.8. The readily available Arch Linux packages are currently built using -`--with-incompatible-bdb` according to the [PKGBUILD](https://projects.archlinux.org/svntogit/community.git/tree/bitcoin/trunk/PKGBUILD). -As mentioned above, when maintaining portability of the wallet between the standard Bitcoin Core distributions and independently built -node software is desired, Berkeley DB 4.8 must be used. - +If you intend to work with legacy Berkeley DB wallets, see [Berkeley DB](#berkeley-db) section. ARM Cross-compilation ------------------- @@ -348,7 +306,7 @@ To build executables for ARM: make HOST=arm-linux-gnueabihf NO_QT=1 cd .. ./autogen.sh - CONFIG_SITE=$PWD/depends/arm-linux-gnueabihf/share/config.site ./configure --enable-glibc-back-compat --enable-reduce-exports LDFLAGS=-static-libstdc++ + CONFIG_SITE=$PWD/depends/arm-linux-gnueabihf/share/config.site ./configure --enable-reduce-exports LDFLAGS=-static-libstdc++ make diff --git a/doc/build-windows.md b/doc/build-windows.md index f88b9739de..e35d3bcbd0 100644 --- a/doc/build-windows.md +++ b/doc/build-windows.md @@ -5,11 +5,9 @@ Below are some notes on how to build Bitcoin Core for Windows. The options known to work for building Bitcoin Core on Windows are: -* On Linux, using the [Mingw-w64](https://mingw-w64.org/doku.php) cross compiler tool chain. Ubuntu Bionic 18.04 is required -and is the platform used to build the Bitcoin Core Windows release binaries. -* On Windows, using [Windows -Subsystem for Linux (WSL)](https://docs.microsoft.com/windows/wsl/about) and the Mingw-w64 cross compiler tool chain. -* On Windows, using a native compiler tool chain such as [Visual Studio](https://www.visualstudio.com). See [README.md](/build_msvc/README.md). +* On Linux, using the [Mingw-w64](https://www.mingw-w64.org/) cross compiler tool chain. +* On Windows, using [Windows Subsystem for Linux (WSL)](https://docs.microsoft.com/windows/wsl/about) and Mingw-w64. +* On Windows, using [Microsoft Visual Studio](https://www.visualstudio.com). See [README.md](/build_msvc/README.md). Other options which may work, but which have not been extensively tested are (please contribute instructions): @@ -18,40 +16,12 @@ Other options which may work, but which have not been extensively tested are (pl Installing Windows Subsystem for Linux --------------------------------------- -With Windows 10, Microsoft has released a new feature named the [Windows -Subsystem for Linux (WSL)](https://docs.microsoft.com/windows/wsl/about). This -feature allows you to run a bash shell directly on Windows in an Ubuntu-based -environment. Within this environment you can cross compile for Windows without -the need for a separate Linux VM or server. Note that while WSL can be installed with -other Linux variants, such as OpenSUSE, the following instructions have only been -tested with Ubuntu. - -This feature is not supported in versions of Windows prior to Windows 10 or on -Windows Server SKUs. In addition, it is available [only for 64-bit versions of -Windows](https://docs.microsoft.com/windows/wsl/install-win10). - -Full instructions to install WSL are available on the above link. -To install WSL on Windows 10 with Fall Creators Update installed (version >= 16215.0) do the following: - -1. Enable the Windows Subsystem for Linux feature - * Open the Windows Features dialog (`OptionalFeatures.exe`) - * Enable 'Windows Subsystem for Linux' - * Click 'OK' and restart if necessary -2. Install Ubuntu - * Open Microsoft Store and search for "Ubuntu 18.04" or use [this link](https://www.microsoft.com/store/productId/9N9TNGVNDL3Q) - * Click Install -3. Complete Installation - * Open a cmd prompt and type "Ubuntu1804" - * Create a new UNIX user account (this is a separate account from your Windows account) - -After the bash shell is active, you can follow the instructions below, starting -with the "Cross-compilation" section. Compiling the 64-bit version is -recommended, but it is possible to compile the 32-bit version. +Follow the upstream installation instructions, available [here](https://docs.microsoft.com/windows/wsl/install-win10). Cross-compilation for Ubuntu and Windows Subsystem for Linux ------------------------------------------------------------ -The steps below can be performed on Ubuntu (including in a VM) or WSL. The depends system +The steps below can be performed on Ubuntu or WSL. The depends system will also work on other Linux distributions, however the commands for installing the toolchain will be different. @@ -78,28 +48,16 @@ Acquire the source in the usual way: ## Building for 64-bit Windows The first step is to install the mingw-w64 cross-compilation tool chain: + - on modern systems (Ubuntu 21.04 Hirsute Hippo or newer, Debian 11 Bullseye or newer): - sudo apt install g++-mingw-w64-x86-64 - -Next, set the default `mingw32 g++` compiler option to POSIX<sup>[1](#footnote1)</sup>: - -``` -sudo update-alternatives --config x86_64-w64-mingw32-g++ +```sh +sudo apt install g++-mingw-w64-x86-64-posix ``` -After running the above command, you should see output similar to that below. -Choose the option that ends with `posix`. + - on older systems: -``` -There are 2 choices for the alternative x86_64-w64-mingw32-g++ (providing /usr/bin/x86_64-w64-mingw32-g++). - - Selection Path Priority Status ------------------------------------------------------------- - 0 /usr/bin/x86_64-w64-mingw32-g++-win32 60 auto mode -* 1 /usr/bin/x86_64-w64-mingw32-g++-posix 30 manual mode - 2 /usr/bin/x86_64-w64-mingw32-g++-win32 60 manual mode - -Press <enter> to keep the current choice[*], or type selection number: +```sh +sudo apt install g++-mingw-w64-x86-64 ``` Once the toolchain is installed the build steps are common: @@ -142,13 +100,3 @@ way. This will install to `c:\workspace\bitcoin`, for example: You can also create an installer using: make deploy - -Footnotes ---------- - -<a name="footnote1">1</a>: Starting from Ubuntu Xenial 16.04, both the 32 and 64 bit Mingw-w64 packages install two different -compiler options to allow a choice between either posix or win32 threads. The default option is win32 threads which is the more -efficient since it will result in binary code that links directly with the Windows kernel32.lib. Unfortunately, the headers -required to support win32 threads conflict with some of the classes in the C++11 standard library, in particular std::mutex. -It's not possible to build the Bitcoin Core code using the win32 version of the Mingw-w64 cross compilers (at least not without -modifying headers in the Bitcoin Core source code). diff --git a/doc/cjdns.md b/doc/cjdns.md new file mode 100644 index 0000000000..b69564729f --- /dev/null +++ b/doc/cjdns.md @@ -0,0 +1,116 @@ +# CJDNS support in Bitcoin Core + +It is possible to run Bitcoin Core over CJDNS, an encrypted IPv6 network that +uses public-key cryptography for address allocation and a distributed hash table +for routing. + +## What is CJDNS? + +CJDNS is like a distributed, shared VPN with multiple entry points where every +participant can reach any other participant. All participants use addresses from +the `fc00::/8` network (reserved IPv6 range). Installation and configuration is +done outside of Bitcoin Core, similarly to a VPN (either in the host/OS or on +the network router). See https://github.com/cjdelisle/cjdns#readme and +https://github.com/hyperboria/docs#hyperboriadocs for more information. + +Compared to IPv4/IPv6, CJDNS provides end-to-end encryption and protects nodes +from traffic analysis and filtering. + +Used with Tor and I2P, CJDNS is a complementary option that can enhance network +redundancy and robustness for both the Bitcoin network and individual nodes. + +Each network has different characteristics. For instance, Tor is widely used but +somewhat centralized. I2P connections have a source address and I2P is slow. +CJDNS is fast but does not hide the sender and the recipient from intermediate +routers. + +## Installing CJDNS and finding a peer to connect to the network + +To install and set up CJDNS, follow the instructions at +https://github.com/cjdelisle/cjdns#how-to-install-cjdns. + +You need to initiate an outbound connection to a peer on the CJDNS network +before it will work with your Bitcoin Core node. This is described in steps +["2. Find a friend"](https://github.com/cjdelisle/cjdns#2-find-a-friend) and +["3. Connect your node to your friend's +node"](https://github.com/cjdelisle/cjdns#3-connect-your-node-to-your-friends-node) +in the CJDNS documentation. + +One quick way to accomplish these two steps is to query for available public +peers on [Hyperboria](https://github.com/hyperboria) by running the following: + +``` +git clone https://github.com/hyperboria/peers hyperboria-peers +cd hyperboria-peers +./testAvailable.py +``` + +For each peer, the `./testAvailable.py` script prints the filename of the peer's +credentials followed by the ping result. + +Choose one or several peers, copy their credentials from their respective files, +paste them into the relevant IPv4 or IPv6 "connectTo" JSON object in the +`cjdroute.conf` file you created in step ["1. Generate a new configuration +file"](https://github.com/cjdelisle/cjdns#1-generate-a-new-configuration-file), +and save the file. + +## Launching CJDNS + +Typically, CJDNS might be launched from its directory with +`sudo ./cjdroute < cjdroute.conf` and it sheds permissions after setting up the +[TUN](https://en.wikipedia.org/wiki/TUN/TAP) interface. You may also [launch it as an +unprivileged user](https://github.com/cjdelisle/cjdns/blob/master/doc/non-root-user.md) +with some additional setup. + +The network connection can be checked by running `./tools/peerStats` from the +CJDNS directory. + +## Run Bitcoin Core with CJDNS + +Once you are connected to the CJDNS network, the following Bitcoin Core +configuration option makes CJDNS peers automatically reachable: + +``` +-cjdnsreachable +``` + +When enabled, this option tells Bitcoin Core that it is running in an +environment where a connection to an `fc00::/8` address will be to the CJDNS +network instead of to an [RFC4193](https://datatracker.ietf.org/doc/html/rfc4193) +IPv6 local network. This helps Bitcoin Core perform better address management: + - Your node can consider incoming `fc00::/8` connections to be from the CJDNS + network rather than from an IPv6 private one. + - If one of your node's local addresses is `fc00::/8`, then it can choose to + gossip that address to peers. + +## Additional configuration options related to CJDNS + +``` +-onlynet=cjdns +``` + +Make automatic outbound connections only to CJDNS addresses. Inbound and manual +connections are not affected by this option. It can be specified multiple times +to allow multiple networks, e.g. onlynet=cjdns, onlynet=i2p, onlynet=onion. + +CJDNS support was added to Bitcoin Core in version 23.0 and there may be fewer +CJDNS peers than Tor or IP ones. You can use `bitcoin-cli -addrinfo` to see the +number of CJDNS addresses known to your node. + +In general, a node can be run with both an onion service and CJDNS (or any/all +of IPv4/IPv6/onion/I2P/CJDNS), which can provide a potential fallback if one of +the networks has issues. There are a number of ways to configure this; see +[doc/tor.md](https://github.com/bitcoin/bitcoin/blob/master/doc/tor.md) for +details. + +## CJDNS-related information in Bitcoin Core + +There are several ways to see your CJDNS address in Bitcoin Core: +- in the "Local addresses" output of CLI `-netinfo` +- in the "localaddresses" output of RPC `getnetworkinfo` + +To see which CJDNS peers your node is connected to, use `bitcoin-cli -netinfo 4` +or the `getpeerinfo` RPC (i.e. `bitcoin-cli getpeerinfo`). + +To see which CJDNS addresses your node knows, use the `getnodeaddresses 0 cjdns` +RPC. diff --git a/doc/dependencies.md b/doc/dependencies.md index b7634718e8..697d432520 100644 --- a/doc/dependencies.md +++ b/doc/dependencies.md @@ -1,50 +1,50 @@ -Dependencies -============ - -These are the dependencies currently used by Bitcoin Core. You can find instructions for installing them in the `build-*.md` file for your platform. - -| Dependency | Version used | Minimum required | CVEs | Shared | [Bundled Qt library](https://doc.qt.io/qt-5/configure-options.html#third-party-libraries) | -| --- | --- | --- | --- | --- | --- | -| Berkeley DB | [4.8.30](https://www.oracle.com/technetwork/database/database-technologies/berkeleydb/downloads/index.html) | 4.8.x | No | | | -| Boost | [1.71.0](https://www.boost.org/users/download/) | [1.64.0](https://github.com/bitcoin/bitcoin/pull/22320) | No | | | -| Clang<sup>[ \* ](#note1)</sup> | | [5.0+](https://releases.llvm.org/download.html) (C++17 support) | | | | -| Expat | [2.2.7](https://libexpat.github.io/) | | No | Yes | | -| fontconfig | [2.12.1](https://www.freedesktop.org/software/fontconfig/release/) | | No | Yes | | -| FreeType | [2.7.1](https://download.savannah.gnu.org/releases/freetype) | | No | | [Yes](https://github.com/bitcoin/bitcoin/blob/master/depends/packages/qt.mk) (Android only) | -| GCC | | [7+](https://gcc.gnu.org/) (C++17 support) | | | | -| HarfBuzz-NG | | | | | [Yes](https://github.com/bitcoin/bitcoin/blob/master/depends/packages/qt.mk) | -| libevent | [2.1.12-stable](https://github.com/libevent/libevent/releases) | [2.0.21](https://github.com/bitcoin/bitcoin/pull/18676) | No | | | -| libnatpmp | git commit [4536032...](https://github.com/miniupnp/libnatpmp/tree/4536032ae32268a45c073a4d5e91bbab4534773a) | | No | | | -| libpng | | | | | [Yes](https://github.com/bitcoin/bitcoin/blob/master/depends/packages/qt.mk) | -| librsvg | | | | | | -| MiniUPnPc | [2.2.2](https://miniupnp.tuxfamily.org/files) | | No | | | -| PCRE | | | | | [Yes](https://github.com/bitcoin/bitcoin/blob/master/depends/packages/qt.mk) | -| Python (tests) | | [3.6](https://www.python.org/downloads) | | | | -| qrencode | [3.4.4](https://fukuchi.org/works/qrencode) | | No | | | -| Qt | [5.12.11](https://download.qt.io/official_releases/qt/) | [5.9.5](https://github.com/bitcoin/bitcoin/issues/20104) | No | | | -| SQLite | [3.32.1](https://sqlite.org/download.html) | [3.7.17](https://github.com/bitcoin/bitcoin/pull/19077) | | | | -| XCB | | | | | [Yes](https://github.com/bitcoin/bitcoin/blob/master/depends/packages/qt.mk) (Linux only) | -| systemtap ([tracing](tracing.md))| | | | | | -| xkbcommon | | | | | [Yes](https://github.com/bitcoin/bitcoin/blob/master/depends/packages/qt.mk) (Linux only) | -| ZeroMQ | [4.3.1](https://github.com/zeromq/libzmq/releases) | 4.0.0 | No | | | -| zlib | | | | | [Yes](https://github.com/bitcoin/bitcoin/blob/master/depends/packages/qt.mk) | - -<a name="note1">Note \*</a> : When compiling with `-stdlib=libc++`, the minimum supported libc++ version is 7.0. - -Controlling dependencies ------------------------- -Some dependencies are not needed in all configurations. The following are some factors that affect the dependency list. - -#### Options passed to `./configure` -* MiniUPnPc is not needed with `--without-miniupnpc`. -* libnatpmp is not needed with `--without-natpmp`. -* Berkeley DB is not needed with `--disable-wallet` or `--without-bdb`. -* SQLite is not needed with `--disable-wallet` or `--without-sqlite`. -* Qt is not needed with `--without-gui`. -* If the qrencode dependency is absent, QR support won't be added. To force an error when that happens, pass `--with-qrencode`. -* If the systemtap dependency is absent, USDT support won't compiled in. -* ZeroMQ is needed only with the `--with-zmq` option. - -#### Other -* librsvg is only needed if you need to run `make deploy` on (cross-compilation to) macOS. -* Not-Qt-bundled zlib is required to build the [DMG tool](../contrib/macdeploy/README.md#deterministic-macos-dmg-notes) from the libdmg-hfsplus project. +# Dependencies + +These are the dependencies used by Bitcoin Core. +You can find installation instructions in the `build-*.md` file for your platform. +"Runtime" and "Version Used" are both in reference to the release binaries. + +| Dependency | Minimum required | +| --- | --- | +| [Autoconf](https://www.gnu.org/software/autoconf/) | [2.69](https://github.com/bitcoin/bitcoin/pull/17769) | +| [Automake](https://www.gnu.org/software/automake/) | [1.13](https://github.com/bitcoin/bitcoin/pull/18290) | +| [Clang](https://clang.llvm.org) | [8.0](https://github.com/bitcoin/bitcoin/pull/24164) | +| [GCC](https://gcc.gnu.org) | [8.1](https://github.com/bitcoin/bitcoin/pull/23060) | +| [Python](https://www.python.org) (tests) | [3.6](https://github.com/bitcoin/bitcoin/pull/19504) | +| [systemtap](https://sourceware.org/systemtap/) ([tracing](tracing.md))| N/A | + +## Required + +| Dependency | Releases | Version used | Minimum required | Runtime | +| --- | --- | --- | --- | --- | +| [Boost](../depends/packages/boost.mk) | [link](https://www.boost.org/users/download/) | [1.77.0](https://github.com/bitcoin/bitcoin/pull/24383) | [1.64.0](https://github.com/bitcoin/bitcoin/pull/22320) | No | +| [libevent](../depends/packages/libevent.mk) | [link](https://github.com/libevent/libevent/releases) | [2.1.12-stable](https://github.com/bitcoin/bitcoin/pull/21991) | [2.1.8](https://github.com/bitcoin/bitcoin/pull/24681) | No | +| glibc | [link](https://www.gnu.org/software/libc/) | N/A | [2.18](https://github.com/bitcoin/bitcoin/pull/23511) | Yes | +| Linux Kernel | [link](https://www.kernel.org/) | N/A | 3.2.0 | Yes | + +## Optional + +### GUI +| Dependency | Releases | Version used | Minimum required | Runtime | +| --- | --- | --- | --- | --- | +| [Fontconfig](../depends/packages/fontconfig.mk) | [link](https://www.freedesktop.org/wiki/Software/fontconfig/) | [2.12.6](https://github.com/bitcoin/bitcoin/pull/23495) | 2.6 | Yes | +| [FreeType](../depends/packages/freetype.mk) | [link](https://freetype.org) | [2.11.0](https://github.com/bitcoin/bitcoin/commit/01544dd78ccc0b0474571da854e27adef97137fb) | 2.3.0 | Yes | +| [qrencode](../depends/packages/qrencode.mk) | [link](https://fukuchi.org/works/qrencode/) | [3.4.4](https://github.com/bitcoin/bitcoin/pull/6373) | | No | +| [Qt](../depends/packages/qt.mk) | [link](https://download.qt.io/official_releases/qt/) | [5.15.3](https://github.com/bitcoin/bitcoin/pull/24668) | [5.11.3](https://github.com/bitcoin/bitcoin/pull/24132) | No | + +### Networking +| Dependency | Releases | Version used | Minimum required | Runtime | +| --- | --- | --- | --- | --- | +| [libnatpmp](../depends/packages/libnatpmp.mk) | [link](https://github.com/miniupnp/libnatpmp/) | commit [4536032...](https://github.com/bitcoin/bitcoin/pull/21209) | | No | +| [MiniUPnPc](../depends/packages/miniupnpc.mk) | [link](https://miniupnp.tuxfamily.org/) | [2.2.2](https://github.com/bitcoin/bitcoin/pull/20421) | 1.9 | No | + +### Notifications +| Dependency | Releases | Version used | Minimum required | Runtime | +| --- | --- | --- | --- | --- | +| [ZeroMQ](../depends/packages/zeromq.mk) | [link](https://github.com/zeromq/libzmq/releases) | [4.3.4](https://github.com/bitcoin/bitcoin/pull/23956) | 4.0.0 | No | + +### Wallet +| Dependency | Releases | Version used | Minimum required | Runtime | +| --- | --- | --- | --- | --- | +| [Berkeley DB](../depends/packages/bdb.mk) (legacy wallet) | [link](https://www.oracle.com/technetwork/database/database-technologies/berkeleydb/downloads/index.html) | 4.8.30 | 4.8.x | No | +| [SQLite](../depends/packages/sqlite.mk) | [link](https://sqlite.org) | [3.32.1](https://github.com/bitcoin/bitcoin/pull/19077) | [3.7.17](https://github.com/bitcoin/bitcoin/pull/19077) | No | diff --git a/doc/descriptors.md b/doc/descriptors.md index 3bbb626a42..ab2face4f0 100644 --- a/doc/descriptors.md +++ b/doc/descriptors.md @@ -33,6 +33,7 @@ Output descriptors currently support: - Pay-to-taproot outputs (P2TR), through the `tr` function. - Multisig scripts, through the `multi` function. - Multisig scripts where the public keys are sorted lexicographically, through the `sortedmulti` function. +- Multisig scripts inside taproot script trees, through the `multi_a` (and `sortedmulti_a`) function. - Any type of supported address through the `addr` function. - Raw hex scripts through the `raw` function. - Public keys (compressed and uncompressed) in hex notation, or BIP32 extended pubkeys with derivation paths. @@ -56,6 +57,7 @@ Output descriptors currently support: - `wsh(multi(1,xpub661MyMwAqRbcFW31YEwpkMuc5THy2PSt5bDMsktWQcFF8syAmRUapSCGu8ED9W6oDMSgv6Zz8idoc4a6mr8BDzTJY47LJhkJ8UB7WEGuduB/1/0/*,xpub69H7F5d8KSRgmmdJg2KhpAK8SR3DjMwAdkxj3ZuxV27CprR9LgpeyGmXUbC6wb7ERfvrnKZjXoUmmDznezpbZb7ap6r1D3tgFxHmwMkQTPH/0/0/*))` describes a set of *1-of-2* P2WSH multisig outputs where the first multisig key is the *1/0/`i`* child of the first specified xpub and the second multisig key is the *0/0/`i`* child of the second specified xpub, and `i` is any number in a configurable range (`0-1000` by default). - `wsh(sortedmulti(1,xpub661MyMwAqRbcFW31YEwpkMuc5THy2PSt5bDMsktWQcFF8syAmRUapSCGu8ED9W6oDMSgv6Zz8idoc4a6mr8BDzTJY47LJhkJ8UB7WEGuduB/1/0/*,xpub69H7F5d8KSRgmmdJg2KhpAK8SR3DjMwAdkxj3ZuxV27CprR9LgpeyGmXUbC6wb7ERfvrnKZjXoUmmDznezpbZb7ap6r1D3tgFxHmwMkQTPH/0/0/*))` describes a set of *1-of-2* P2WSH multisig outputs where one multisig key is the *1/0/`i`* child of the first specified xpub and the other multisig key is the *0/0/`i`* child of the second specified xpub, and `i` is any number in a configurable range (`0-1000` by default). The order of public keys in the resulting witnessScripts is determined by the lexicographic order of the public keys at that index. - `tr(c6047f9441ed7d6d3045406e95c07cd85c778e4b8cef3ca7abac09b95c709ee5,{pk(fff97bd5755eeea420453a14355235d382f6472f8568a18b2f057a1460297556),pk(e493dbf1c10d80f3581e4904930b1404cc6c13900ee0758474fa94abe8c4cd13)})` describes a P2TR output with the `c6...` x-only pubkey as internal key, and two script paths. +- `tr(c6047f9441ed7d6d3045406e95c07cd85c778e4b8cef3ca7abac09b95c709ee5,sortedmulti_a(2,2f8bde4d1a07209355b4a7250a5c5128e88b84bddc619ab7cba8d569b240efe4,5cbdf0646e5db4eaa398f365f2ea7a0e3d419b7e0330e39ce92bddedcac4f9bc))` describes a P2TR output with the `c6...` x-only pubkey as internal key, and a single `multi_a` script that needs 2 signatures with 2 specified x-only keys, which will be sorted lexicographically. ## Reference @@ -68,8 +70,10 @@ Descriptors consist of several types of expressions. The top level expression is - `pkh(KEY)` (not inside `tr`): P2PKH output for the given public key (use `addr` if you only know the pubkey hash). - `wpkh(KEY)` (top level or inside `sh` only): P2WPKH output for the given compressed pubkey. - `combo(KEY)` (top level only): an alias for the collection of `pk(KEY)` and `pkh(KEY)`. If the key is compressed, it also includes `wpkh(KEY)` and `sh(wpkh(KEY))`. -- `multi(k,KEY_1,KEY_2,...,KEY_n)` (not inside `tr`): k-of-n multisig script. +- `multi(k,KEY_1,KEY_2,...,KEY_n)` (not inside `tr`): k-of-n multisig script using OP_CHECKMULTISIG. - `sortedmulti(k,KEY_1,KEY_2,...,KEY_n)` (not inside `tr`): k-of-n multisig script with keys sorted lexicographically in the resulting script. +- `multi_a(k,KEY_1,KEY_2,...,KEY_N)` (only inside `tr`): k-of-n multisig script using OP_CHECKSIG, OP_CHECKSIGADD, and OP_NUMEQUAL. +- `sortedmulti_a(k,KEY_1,KEY_2,...,KEY_N)` (only inside `tr`): similar to `multi_a`, but the (x-only) public keys in it will be sorted lexicographically. - `tr(KEY)` or `tr(KEY,TREE)` (top level only): P2TR output with the specified key as internal key, and optionally a tree of script paths. - `addr(ADDR)` (top level only): the script which ADDR expands to. - `raw(HEX)` (top level only): the script whose hex encoding is HEX. @@ -90,12 +94,12 @@ Descriptors consist of several types of expressions. The top level expression is - Optionally followed by a single `/*` or `/*'` final step to denote all (direct) unhardened or hardened children. - The usage of hardened derivation steps requires providing the private key. +(Anywhere a `'` suffix is permitted to denote hardened derivation, the suffix `h` can be used instead.) + `TREE` expressions: - any `SCRIPT` expression - An open brace `{`, a `TREE` expression, a comma `,`, a `TREE` expression, and a closing brace `}` -(Anywhere a `'` suffix is permitted to denote hardened derivation, the suffix `h` can be used instead.) - `ADDR` expressions are any type of supported address: - P2PKH addresses (base58, of the form `1...` for mainnet or `[nm]...` for testnet). Note that P2PKH addresses in descriptors cannot be used for P2PK outputs (use the `pk` function instead). - P2SH addresses (base58, of the form `3...` for mainnet or `2...` for testnet, defined in [BIP 13](https://github.com/bitcoin/bips/blob/master/bip-0013.mediawiki)). @@ -139,6 +143,47 @@ Key order does not matter for `sortedmulti()`. `sortedmulti()` behaves in the sa as `multi()` does but the keys are reordered in the resulting script such that they are lexicographically ordered as described in BIP67. +#### Basic multisig example + +For a good example of a basic M-of-N multisig between multiple participants using descriptor +wallets and PSBTs, as well as a signing flow, see [this functional test](/test/functional/wallet_multisig_descriptor_psbt.py). + +Disclaimers: It is important to note that this example serves as a quick-start and is kept basic for readability. A downside of the approach +outlined here is that each participant must maintain (and backup) two separate wallets: a signer and the corresponding multisig. +It should also be noted that privacy best-practices are not "by default" here - participants should take care to only use the signer to sign +transactions related to the multisig. Lastly, it is not recommended to use anything other than a Bitcoin Core descriptor wallet to serve as your +signer(s). Other wallets, whether hardware or software, likely impose additional checks and safeguards to prevent users from signing transactions that +could lead to loss of funds, or are deemed security hazards. Conforming to various 3rd-party checks and verifications is not in the scope of this example. + +The basic steps are: + + 1. Every participant generates an xpub. The most straightforward way is to create a new descriptor wallet which we will refer to as + the participant's signer wallet. Avoid reusing this wallet for any purpose other than signing transactions from the + corresponding multisig we are about to create. Hint: extract the wallet's xpubs using `listdescriptors` and pick the one from the + `pkh` descriptor since it's least likely to be accidentally reused (legacy addresses) + 2. Create a watch-only descriptor wallet (blank, private keys disabled). Now the multisig is created by importing the two descriptors: + `wsh(sortedmulti(<M>,XPUB1/0/*,XPUB2/0/*,…,XPUBN/0/*))` and `wsh(sortedmulti(<M>,XPUB1/1/*,XPUB2/1/*,…,XPUBN/1/*))` + (one descriptor w/ `0` for receiving addresses and another w/ `1` for change). Every participant does this + 3. A receiving address is generated for the multisig. As a check to ensure step 2 was done correctly, every participant + should verify they get the same addresses + 4. Funds are sent to the resulting address + 5. A sending transaction from the multisig is created using `walletcreatefundedpsbt` (anyone can initiate this). It is simple to do + this in the GUI by going to the `Send` tab in the multisig wallet and creating an unsigned transaction (PSBT) + 6. At least `M` participants check the PSBT with their multisig using `decodepsbt` to verify the transaction is OK before signing it. + 7. (If OK) the participant signs the PSBT with their signer wallet using `walletprocesspsbt`. It is simple to do this in the GUI by + loading the PSBT from file and signing it + 8. The signed PSBTs are collected with `combinepsbt`, finalized w/ `finalizepsbt`, and then the resulting transaction is broadcasted + to the network. Note that any wallet (eg one of the signers or multisig) is capable of doing this. + 9. Checks that balances are correct after the transaction has been included in a block + +You may prefer a daisy chained signing flow where each participant signs the PSBT one after another until +the PSBT has been signed `M` times and is "complete." For the most part, the steps above remain the same, except (6, 7) +change slightly from signing the original PSBT in parallel to signing it in series. `combinepsbt` is not necessary with +this signing flow and the last (`m`th) signer can just broadcast the PSBT after signing. Note that a parallel signing flow may be +preferable in cases where there are more signers. This signing flow is also included in the test / Python example. +[The test](/test/functional/wallet_multisig_descriptor_psbt.py) is meant to be documentation as much as it is a functional test, so +it is kept as simple and readable as possible. + ### BIP32 derived keys and chains Most modern wallet software and hardware uses keys that are derived using diff --git a/doc/design/assumeutxo.md b/doc/design/assumeutxo.md new file mode 100644 index 0000000000..2726cf779b --- /dev/null +++ b/doc/design/assumeutxo.md @@ -0,0 +1,138 @@ +# assumeutxo + +Assumeutxo is a feature that allows fast bootstrapping of a validating bitcoind +instance with a very similar security model to assumevalid. + +The RPC commands `dumptxoutset` and `loadtxoutset` are used to respectively generate +and load UTXO snapshots. The utility script `./contrib/devtools/utxo_snapshot.sh` may +be of use. + +## General background + +- [assumeutxo proposal](https://github.com/jamesob/assumeutxo-docs/tree/2019-04-proposal/proposal) +- [Github issue](https://github.com/bitcoin/bitcoin/issues/15605) +- [draft PR](https://github.com/bitcoin/bitcoin/pull/15606) + +## Design notes + +- A new block index `nStatus` flag is introduced, `BLOCK_ASSUMED_VALID`, to mark block + index entries that are required to be assumed-valid by a chainstate created + from a UTXO snapshot. This flag is mostly used as a way to modify certain + CheckBlockIndex() logic to account for index entries that are pending validation by a + chainstate running asynchronously in the background. We also use this flag to control + which index entries are added to setBlockIndexCandidates during LoadBlockIndex(). + +- Indexing implementations via BaseIndex can no longer assume that indexation happens + sequentially, since background validation chainstates can submit BlockConnected + events out of order with the active chain. + +- The concept of UTXO snapshots is treated as an implementation detail that lives + behind the ChainstateManager interface. The external presentation of the changes + required to facilitate the use of UTXO snapshots is the understanding that there are + now certain regions of the chain that can be temporarily assumed to be valid (using + the nStatus flag mentioned above). In certain cases, e.g. wallet rescanning, this is + very similar to dealing with a pruned chain. + + Logic outside ChainstateManager should try not to know about snapshots, instead + preferring to work in terms of more general states like assumed-valid. + + +## Chainstate phases + +Chainstate within the system goes through a number of phases when UTXO snapshots are +used, as managed by `ChainstateManager`. At various points there can be multiple +`CChainState` objects in existence to facilitate both maintaining the network tip and +performing historical validation of the assumed-valid chain. + +It is worth noting that though there are multiple separate chainstates, those +chainstates share use of a common block index (i.e. they hold the same `BlockManager` +reference). + +The subheadings below outline the phases and the corresponding changes to chainstate +data. + +### "Normal" operation via initial block download + +`ChainstateManager` manages a single CChainState object, for which +`m_snapshot_blockhash` is null. This chainstate is (maybe obviously) +considered active. This is the "traditional" mode of operation for bitcoind. + +| | | +| ---------- | ----------- | +| number of chainstates | 1 | +| active chainstate | ibd | + +### User loads a UTXO snapshot via `loadtxoutset` RPC + +`ChainstateManager` initializes a new chainstate (see `ActivateSnapshot()`) to load the +snapshot contents into. During snapshot load and validation (see +`PopulateAndValidateSnapshot()`), the new chainstate is not considered active and the +original chainstate remains in use as active. + +| | | +| ---------- | ----------- | +| number of chainstates | 2 | +| active chainstate | ibd | + +Once the snapshot chainstate is loaded and validated, it is promoted to active +chainstate and a sync to tip begins. A new chainstate directory is created in the +datadir for the snapshot chainstate called +`chainstate_[SHA256 blockhash of snapshot base block]`. + +| | | +| ---------- | ----------- | +| number of chainstates | 2 | +| active chainstate | snapshot | + +The snapshot begins to sync to tip from its base block, technically in parallel with +the original chainstate, but it is given priority during block download and is +allocated most of the cache (see `MaybeRebalanceCaches()` and usages) as our chief +consideration is getting to network tip. + +**Failure consideration:** if shutdown happens at any point during this phase, both +chainstates will be detected during the next init and the process will resume. + +### Snapshot chainstate hits network tip + +Once the snapshot chainstate leaves IBD, caches are rebalanced +(via `MaybeRebalanceCaches()` in `ActivateBestChain()`) and more cache is given +to the background chainstate, which is responsible for doing full validation of the +assumed-valid parts of the chain. + +**Note:** at this point, ValidationInterface callbacks will be coming in from both +chainstates. Considerations here must be made for indexing, which may no longer be happening +sequentially. + +### Background chainstate hits snapshot base block + +Once the tip of the background chainstate hits the base block of the snapshot +chainstate, we stop use of the background chainstate by setting `m_stop_use` (not yet +committed - see #15606), in `CompleteSnapshotValidation()`, which is checked in +`ActivateBestChain()`). We hash the background chainstate's UTXO set contents and +ensure it matches the compiled value in `CMainParams::m_assumeutxo_data`. + +The background chainstate data lingers on disk until shutdown, when in +`ChainstateManager::Reset()`, the background chainstate is cleaned up with +`ValidatedSnapshotShutdownCleanup()`, which renames the `chainstate_[hash]` datadir as +`chainstate`. + +| | | +| ---------- | ----------- | +| number of chainstates | 2 (ibd has `m_stop_use=true`) | +| active chainstate | snapshot | + +**Failure consideration:** if bitcoind unexpectedly halts after `m_stop_use` is set on +the background chainstate but before `CompleteSnapshotValidation()` can finish, the +need to complete snapshot validation will be detected on subsequent init by +`ChainstateManager::CheckForUncleanShutdown()`. + +### Bitcoind restarts sometime after snapshot validation has completed + +When bitcoind initializes again, what began as the snapshot chainstate is now +indistinguishable from a chainstate that has been built from the traditional IBD +process, and will be initialized as such. + +| | | +| ---------- | ----------- | +| number of chainstates | 1 | +| active chainstate | ibd | diff --git a/doc/design/libraries.md b/doc/design/libraries.md new file mode 100644 index 0000000000..75f8d60ba0 --- /dev/null +++ b/doc/design/libraries.md @@ -0,0 +1,104 @@ +# Libraries + +| Name | Description | +|--------------------------|-------------| +| *libbitcoin_cli* | RPC client functionality used by *bitcoin-cli* executable | +| *libbitcoin_common* | Home for common functionality shared by different executables and libraries. Similar to *libbitcoin_util*, but higher-level (see [Dependencies](#dependencies)). | +| *libbitcoin_consensus* | Stable, backwards-compatible consensus functionality used by *libbitcoin_node* and *libbitcoin_wallet* and also exposed as a [shared library](../shared-libraries.md). | +| *libbitcoinconsensus* | Shared library build of static *libbitcoin_consensus* library | +| *libbitcoin_kernel* | Consensus engine and support library used for validation by *libbitcoin_node* and also exposed as a [shared library](../shared-libraries.md). | +| *libbitcoinqt* | GUI functionality used by *bitcoin-qt* and *bitcoin-gui* executables | +| *libbitcoin_ipc* | IPC functionality used by *bitcoin-node*, *bitcoin-wallet*, *bitcoin-gui* executables to communicate when [`--enable-multiprocess`](multiprocess.md) is used. | +| *libbitcoin_node* | P2P and RPC server functionality used by *bitcoind* and *bitcoin-qt* executables. | +| *libbitcoin_util* | Home for common functionality shared by different executables and libraries. Similar to *libbitcoin_common*, but lower-level (see [Dependencies](#dependencies)). | +| *libbitcoin_wallet* | Wallet functionality used by *bitcoind* and *bitcoin-wallet* executables. | +| *libbitcoin_wallet_tool* | Lower-level wallet functionality used by *bitcoin-wallet* executable. | +| *libbitcoin_zmq* | [ZeroMQ](../zmq.md) functionality used by *bitcoind* and *bitcoin-qt* executables. | + +## Conventions + +- Most libraries are internal libraries and have APIs which are completely unstable! There are few or no restrictions on backwards compatibility or rules about external dependencies. Exceptions are *libbitcoin_consensus* and *libbitcoin_kernel* which have external interfaces documented at [../shared-libraries.md](../shared-libraries.md). + +- Generally each library should have a corresponding source directory and namespace. Source code organization is a work in progress, so it is true that some namespaces are applied inconsistently, and if you look at [`libbitcoin_*_SOURCES`](../../src/Makefile.am) lists you can see that many libraries pull in files from outside their source directory. But when working with libraries, it is good to follow a consistent pattern like: + + - *libbitcoin_node* code lives in `src/node/` in the `node::` namespace + - *libbitcoin_wallet* code lives in `src/wallet/` in the `wallet::` namespace + - *libbitcoin_ipc* code lives in `src/ipc/` in the `ipc::` namespace + - *libbitcoin_util* code lives in `src/util/` in the `util::` namespace + - *libbitcoin_consensus* code lives in `src/consensus/` in the `Consensus::` namespace + +## Dependencies + +- Libraries should minimize what other libraries they depend on, and only reference symbols following the arrows shown in the dependency graph below: + +<table><tr><td> + +```mermaid + +%%{ init : { "flowchart" : { "curve" : "linear" }}}%% + +graph TD; + +bitcoin-cli[bitcoin-cli]-->libbitcoin_cli; + +bitcoind[bitcoind]-->libbitcoin_node; +bitcoind[bitcoind]-->libbitcoin_wallet; + +bitcoin-qt[bitcoin-qt]-->libbitcoin_node; +bitcoin-qt[bitcoin-qt]-->libbitcoinqt; +bitcoin-qt[bitcoin-qt]-->libbitcoin_wallet; + +bitcoin-wallet[bitcoin-wallet]-->libbitcoin_wallet; +bitcoin-wallet[bitcoin-wallet]-->libbitcoin_wallet_tool; + +libbitcoin_cli-->libbitcoin_common; +libbitcoin_cli-->libbitcoin_util; + +libbitcoin_common-->libbitcoin_util; +libbitcoin_common-->libbitcoin_consensus; + +libbitcoin_kernel-->libbitcoin_consensus; +libbitcoin_kernel-->libbitcoin_util; + +libbitcoin_node-->libbitcoin_common; +libbitcoin_node-->libbitcoin_consensus; +libbitcoin_node-->libbitcoin_kernel; +libbitcoin_node-->libbitcoin_util; + +libbitcoinqt-->libbitcoin_common; +libbitcoinqt-->libbitcoin_util; + +libbitcoin_wallet-->libbitcoin_common; +libbitcoin_wallet-->libbitcoin_util; + +libbitcoin_wallet_tool-->libbitcoin_util; +libbitcoin_wallet_tool-->libbitcoin_wallet; + +classDef bold stroke-width:2px, font-weight:bold, font-size: smaller; +class bitcoin-qt,bitcoind,bitcoin-cli,bitcoin-wallet bold +``` +</td></tr><tr><td> + +**Dependency graph**. Arrows show linker symbol dependencies. *Consensus* lib depends on nothing. *Util* lib is depended on by everything. *Kernel* lib depends only on consensus and util. + +</td></tr></table> + +- The graph shows what _linker symbols_ (functions and variables) from each library other libraries can call and reference directly, but it is not a call graph. For example, there is no arrow connecting *libbitcoin_wallet* and *libbitcoin_node* libraries, because these libraries are intended to be modular and not depend on each other's internal implementation details. But wallet code still is still able to call node code indirectly through the `interfaces::Chain` abstract class in [`interfaces/chain.h`](../../src/interfaces/chain.h) and node code calls wallet code through the `interfaces::ChainClient` and `interfaces::Chain::Notifications` abstract classes in the same file. In general, defining abstract classes in [`src/interfaces/`](../../src/interfaces/) can be a convenient way of avoiding unwanted direct dependencies or circular dependencies between libraries. + +- *libbitcoin_consensus* should be a standalone dependency that any library can depend on, and it should not depend on any other libraries itself. + +- *libbitcoin_util* should also be a standalone dependency that any library can depend on, and it should not depend on other internal libraries. + +- *libbitcoin_common* should serve a similar function as *libbitcoin_util* and be a place for miscellaneous code used by various daemon, GUI, and CLI applications and libraries to live. It should not depend on anything other than *libbitcoin_util* and *libbitcoin_consensus*. The boundary between _util_ and _common_ is a little fuzzy but historically _util_ has been used for more generic, lower-level things like parsing hex, and _common_ has been used for bitcoin-specific, higher-level things like parsing base58. The difference between util and common is mostly important because *libbitcoin_kernel* is not supposed to depend on *libbitcoin_common*, only *libbitcoin_util*. In general, if it is ever unclear whether it is better to add code to *util* or *common*, it is probably better to add it to *common* unless it is very generically useful or useful particularly to include in the kernel. + + +- *libbitcoin_kernel* should only depend on *libbitcoin_util* and *libbitcoin_consensus*. + +- The only thing that should depend on *libbitcoin_kernel* internally should be *libbitcoin_node*. GUI and wallet libraries *libbitcoinqt* and *libbitcoin_wallet* in particular should not depend on *libbitcoin_kernel* and the unneeded functionality it would pull in, like block validation. To the extent that GUI and wallet code need scripting and signing functionality, they should be get able it from *libbitcoin_consensus*, *libbitcoin_common*, and *libbitcoin_util*, instead of *libbitcoin_kernel*. + +- GUI, node, and wallet code internal implementations should all be independent of each other, and the *libbitcoinqt*, *libbitcoin_node*, *libbitcoin_wallet* libraries should never reference each other's symbols. They should only call each other through [`src/interfaces/`](`../../src/interfaces/`) abstract interfaces. + +## Work in progress + +- Validation code is moving from *libbitcoin_node* to *libbitcoin_kernel* as part of [The libbitcoinkernel Project #24303](https://github.com/bitcoin/bitcoin/issues/24303) +- Source code organization is discussed in general in [Library source code organization #15732](https://github.com/bitcoin/bitcoin/issues/15732) diff --git a/doc/multiprocess.md b/doc/design/multiprocess.md index e3f389a6d3..e3f389a6d3 100644 --- a/doc/multiprocess.md +++ b/doc/design/multiprocess.md diff --git a/doc/developer-notes.md b/doc/developer-notes.md index 583c50a763..23f975df34 100644 --- a/doc/developer-notes.md +++ b/doc/developer-notes.md @@ -12,10 +12,12 @@ Developer Notes - [Generating Documentation](#generating-documentation) - [Development tips and tricks](#development-tips-and-tricks) - [Compiling for debugging](#compiling-for-debugging) + - [Show sources in debugging](#show-sources-in-debugging) - [Compiling for gprof profiling](#compiling-for-gprof-profiling) - [`debug.log`](#debuglog) - [Signet, testnet, and regtest modes](#signet-testnet-and-regtest-modes) - [DEBUG_LOCKORDER](#debug_lockorder) + - [DEBUG_LOCKCONTENTION](#debug_lockcontention) - [Valgrind suppressions file](#valgrind-suppressions-file) - [Compiling for test coverage](#compiling-for-test-coverage) - [Performance profiling with perf](#performance-profiling-with-perf) @@ -30,6 +32,7 @@ Developer Notes - [C++ data structures](#c-data-structures) - [Strings and formatting](#strings-and-formatting) - [Shadowing](#shadowing) + - [Lifetimebound](#lifetimebound) - [Threads and synchronization](#threads-and-synchronization) - [Scripts](#scripts) - [Shebang](#shebang) @@ -89,8 +92,15 @@ code. - Class member variables have a `m_` prefix. - Global variables have a `g_` prefix. - Constant names are all uppercase, and use `_` to separate words. + - Enumerator constants may be `snake_case`, `PascalCase` or `ALL_CAPS`. + This is a more tolerant policy than the [C++ Core + Guidelines](https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#Renum-caps), + which recommend using `snake_case`. Please use what seems appropriate. - Class names, function names, and method names are UpperCamelCase - (PascalCase). Do not prefix class names with `C`. + (PascalCase). Do not prefix class names with `C`. See [Internal interface + naming style](#internal-interface-naming-style) for an exception to this + convention. + - Test suite naming convention: The Boost test suite in file `src/test/foo_tests.cpp` should be named `foo_tests`. Test suite names must be unique. @@ -100,6 +110,28 @@ code. - `nullptr` is preferred over `NULL` or `(void*)0`. - `static_assert` is preferred over `assert` where possible. Generally; compile-time checking is preferred over run-time checking. +For function calls a namespace should be specified explicitly, unless such functions have been declared within it. +Otherwise, [argument-dependent lookup](https://en.cppreference.com/w/cpp/language/adl), also known as ADL, could be +triggered that makes code harder to maintain and reason about: +```c++ +#include <filesystem> + +namespace fs { +class path : public std::filesystem::path +{ +}; +// The intention is to disallow this function. +bool exists(const fs::path& p) = delete; +} // namespace fs + +int main() +{ + //fs::path p; // error + std::filesystem::path p; // compiled + exists(p); // ADL being used for unqualified name lookup +} +``` + Block style example: ```c++ int g_count = 0; @@ -132,11 +164,78 @@ public: } // namespace foo ``` +Coding Style (C++ functions and methods) +-------------------- + +- When ordering function parameters, place input parameters first, then any + in-out parameters, followed by any output parameters. + +- *Rationale*: API consistency. + +- Prefer returning values directly to using in-out or output parameters. Use + `std::optional` where helpful for returning values. + +- *Rationale*: Less error-prone (no need for assumptions about what the output + is initialized to on failure), easier to read, and often the same or better + performance. + +- Generally, use `std::optional` to represent optional by-value inputs (and + instead of a magic default value, if there is no real default). Non-optional + input parameters should usually be values or const references, while + non-optional in-out and output parameters should usually be references, as + they cannot be null. + +Coding Style (C++ named arguments) +------------------------------ + +When passing named arguments, use a format that clang-tidy understands. The +argument names can otherwise not be verified by clang-tidy. + +For example: + +```c++ +void function(Addrman& addrman, bool clear); + +int main() +{ + function(g_addrman, /*clear=*/false); +} +``` + +### Running clang-tidy + +To run clang-tidy on Ubuntu/Debian, install the dependencies: + +```sh +apt install clang-tidy bear clang +``` + +Then, pass clang as compiler to configure, and use bear to produce the `compile_commands.json`: + +```sh +./autogen.sh && ./configure CC=clang CXX=clang++ +make clean && bear make -j $(nproc) # For bear 2.x +make clean && bear -- make -j $(nproc) # For bear 3.x +``` + +To run clang-tidy on all source files: + +```sh +( cd ./src/ && run-clang-tidy -j $(nproc) ) +``` + +To run clang-tidy on the changed source lines: + +```sh +git diff | ( cd ./src/ && clang-tidy-diff -p2 -j $(nproc) ) +``` + Coding Style (Python) --------------------- Refer to [/test/functional/README.md#style-guidelines](/test/functional/README.md#style-guidelines). + Coding Style (Doxygen-compatible comments) ------------------------------------------ @@ -249,6 +348,35 @@ Development tips and tricks Run configure with `--enable-debug` to add additional compiler flags that produce better debugging builds. +### Show sources in debugging + +If you have ccache enabled, absolute paths are stripped from debug information +with the -fdebug-prefix-map and -fmacro-prefix-map options (if supported by the +compiler). This might break source file detection in case you move binaries +after compilation, debug from the directory other than the project root or use +an IDE that only supports absolute paths for debugging. + +There are a few possible fixes: + +1. Configure source file mapping. + +For `gdb` create or append to `.gdbinit` file: +``` +set substitute-path ./src /path/to/project/root/src +``` + +For `lldb` create or append to `.lldbinit` file: +``` +settings set target.source-map ./src /path/to/project/root/src +``` + +2. Add a symlink to the `./src` directory: +``` +ln -s /path/to/project/root/src src +``` + +3. Use `debugedit` to modify debug information in the binary. + ### Compiling for gprof profiling Run configure with the `--enable-gprof` option, then make. @@ -282,6 +410,21 @@ configure option adds `-DDEBUG_LOCKORDER` to the compiler flags. This inserts run-time checks to keep track of which locks are held and adds warnings to the `debug.log` file if inconsistencies are detected. +### DEBUG_LOCKCONTENTION + +Defining `DEBUG_LOCKCONTENTION` adds a "lock" logging category to the logging +RPC that, when enabled, logs the location and duration of each lock contention +to the `debug.log` file. + +The `--enable-debug` configure option adds `-DDEBUG_LOCKCONTENTION` to the +compiler flags. You may also enable it manually for a non-debug build by running +configure with `-DDEBUG_LOCKCONTENTION` added to your CPPFLAGS, +i.e. `CPPFLAGS="-DDEBUG_LOCKCONTENTION"`, then build and run bitcoind. + +You can then use the `-debug=lock` configuration option at bitcoind startup or +`bitcoin-cli logging '["lock"]'` at runtime to turn on lock contention logging. +It can be toggled off again with `bitcoin-cli logging [] '["lock"]'`. + ### Assertions and Checks The util file `src/util/check.h` offers helpers to protect against coding and @@ -297,7 +440,7 @@ other input. failure, it will throw an exception, which can be caught to recover from the error. - For example, a nullptr dereference or any other logic bug in RPC code - means that the RPC code is faulty and can not be executed. However, the + means that the RPC code is faulty and cannot be executed. However, the logic bug can be shown to the user and the program can continue to run. * `Assume` should be used to document assumptions when program execution can safely continue even if the assumption is violated. In debug builds it @@ -345,7 +488,7 @@ make cov Profiling is a good way to get a precise idea of where time is being spent in code. One tool for doing profiling on Linux platforms is called -[`perf`](http://www.brendangregg.com/perf.html), and has been integrated into +[`perf`](https://www.brendangregg.com/perf.html), and has been integrated into the functional test framework. Perf can observe a running process and sample (at some frequency) where its execution is. @@ -563,10 +706,6 @@ Wallet - Make sure that no crashes happen with run-time option `-disablewallet`. -- Include `db_cxx.h` (BerkeleyDB header) only when `ENABLE_WALLET` is set. - - - *Rationale*: Otherwise compilation of the disable-wallet build will fail in environments without BerkeleyDB. - General C++ ------------- @@ -669,19 +808,19 @@ Foo(vec); ```cpp enum class Tabs { - INFO, - CONSOLE, - GRAPH, - PEERS + info, + console, + network_graph, + peers }; int GetInt(Tabs tab) { switch (tab) { - case Tabs::INFO: return 0; - case Tabs::CONSOLE: return 1; - case Tabs::GRAPH: return 2; - case Tabs::PEERS: return 3; + case Tabs::info: return 0; + case Tabs::console: return 1; + case Tabs::network_graph: return 2; + case Tabs::peers: return 3; } // no default case, so the compiler can warn about missing cases assert(false); } @@ -692,11 +831,6 @@ int GetInt(Tabs tab) Strings and formatting ------------------------ -- Be careful of `LogPrint` versus `LogPrintf`. `LogPrint` takes a `category` argument, `LogPrintf` does not. - - - *Rationale*: Confusion of these can result in runtime exceptions due to - formatting mismatch, and it is easy to get wrong because of subtly similar naming. - - Use `std::string`, avoid C string manipulation functions. - *Rationale*: C++ string handling is marginally safer, less scope for @@ -769,22 +903,37 @@ from using a different variable with the same name), please name variables so that their names do not shadow variables defined in the source code. When using nested cycles, do not name the inner cycle variable the same as in -the upper cycle, etc. +the outer cycle, etc. + +Lifetimebound +-------------- + +The [Clang `lifetimebound` +attribute](https://clang.llvm.org/docs/AttributeReference.html#lifetimebound) +can be used to tell the compiler that a lifetime is bound to an object and +potentially see a compile-time warning if the object has a shorter lifetime from +the invalid use of a temporary. You can use the attribute by adding a `LIFETIMEBOUND` +annotation defined in `src/attributes.h`; please grep the codebase for examples. Threads and synchronization ---------------------------- -- Prefer `Mutex` type to `RecursiveMutex` one +- Prefer `Mutex` type to `RecursiveMutex` one. - Consistently use [Clang Thread Safety Analysis](https://clang.llvm.org/docs/ThreadSafetyAnalysis.html) annotations to - get compile-time warnings about potential race conditions in code. Combine annotations in function declarations with - run-time asserts in function definitions: + get compile-time warnings about potential race conditions or deadlocks in code. - In functions that are declared separately from where they are defined, the thread safety annotations should be added exclusively to the function declaration. Annotations on the definition could lead to false positives (lack of compile failure) at call sites between the two. + - Prefer locks that are in a class rather than global, and that are + internal to a class (private or protected) rather than public. + + - Combine annotations in function declarations with run-time asserts in + function definitions: + ```C++ // txmempool.h class CTxMemPool @@ -808,21 +957,37 @@ void CTxMemPool::UpdateTransactionsFromBlock(...) ```C++ // validation.h -class ChainstateManager +class CChainState { +protected: + ... + Mutex m_chainstate_mutex; + ... public: ... - bool ProcessNewBlock(...) LOCKS_EXCLUDED(::cs_main); + bool ActivateBestChain( + BlockValidationState& state, + std::shared_ptr<const CBlock> pblock = nullptr) + EXCLUSIVE_LOCKS_REQUIRED(!m_chainstate_mutex) + LOCKS_EXCLUDED(::cs_main); + ... + bool PreciousBlock(BlockValidationState& state, CBlockIndex* pindex) + EXCLUSIVE_LOCKS_REQUIRED(!m_chainstate_mutex) + LOCKS_EXCLUDED(::cs_main); ... } // validation.cpp -bool ChainstateManager::ProcessNewBlock(...) +bool CChainState::PreciousBlock(BlockValidationState& state, CBlockIndex* pindex) { + AssertLockNotHeld(m_chainstate_mutex); AssertLockNotHeld(::cs_main); - ... - LOCK(::cs_main); - ... + { + LOCK(cs_main); + ... + } + + return ActivateBestChain(state, std::shared_ptr<const CBlock>()); } ``` @@ -853,6 +1018,8 @@ TRY_LOCK(cs_vNodes, lockNodes); Scripts -------------------------- +Write scripts in Python rather than bash, when possible. + ### Shebang - Use `#!/usr/bin/env bash` instead of obsolete `#!/bin/bash`. @@ -959,37 +1126,40 @@ Subtrees Several parts of the repository are subtrees of software maintained elsewhere. -Some of these are maintained by active developers of Bitcoin Core, in which case changes should probably go -directly upstream without being PRed directly against the project. They will be merged back in the next -subtree merge. +Some of these are maintained by active developers of Bitcoin Core, in which case +changes should go directly upstream without being PRed directly against the project. +They will be merged back in the next subtree merge. -Others are external projects without a tight relationship with our project. Changes to these should also -be sent upstream, but bugfixes may also be prudent to PR against Bitcoin Core so that they can be integrated -quickly. Cosmetic changes should be purely taken upstream. +Others are external projects without a tight relationship with our project. Changes +to these should also be sent upstream, but bugfixes may also be prudent to PR against +a Bitcoin Core subtree, so that they can be integrated quickly. Cosmetic changes +should be taken upstream. -There is a tool in `test/lint/git-subtree-check.sh` ([instructions](../test/lint#git-subtree-checksh)) to check a subtree directory for consistency with -its upstream repository. +There is a tool in `test/lint/git-subtree-check.sh` ([instructions](../test/lint#git-subtree-checksh)) +to check a subtree directory for consistency with its upstream repository. Current subtrees include: - src/leveldb - - Upstream at https://github.com/google/leveldb ; Maintained by Google, but - open important PRs to Core to avoid delay. + - Subtree at https://github.com/bitcoin-core/leveldb-subtree ; maintained by Core contributors. + - Upstream at https://github.com/google/leveldb ; maintained by Google. Open + important PRs to the subtree to avoid delay. - **Note**: Follow the instructions in [Upgrading LevelDB](#upgrading-leveldb) when merging upstream changes to the LevelDB subtree. - src/crc32c - Used by leveldb for hardware acceleration of CRC32C checksums for data integrity. - - Upstream at https://github.com/google/crc32c ; Maintained by Google. + - Subtree at https://github.com/bitcoin-core/crc32c-subtree ; maintained by Core contributors. + - Upstream at https://github.com/google/crc32c ; maintained by Google. - src/secp256k1 - - Upstream at https://github.com/bitcoin-core/secp256k1/ ; actively maintained by Core contributors. + - Upstream at https://github.com/bitcoin-core/secp256k1/ ; maintained by Core contributors. - src/crypto/ctaes - - Upstream at https://github.com/bitcoin-core/ctaes ; actively maintained by Core contributors. + - Upstream at https://github.com/bitcoin-core/ctaes ; maintained by Core contributors. -- src/univalue - - Upstream at https://github.com/bitcoin-core/univalue ; actively maintained by Core contributors, deviates from upstream https://github.com/jgarzik/univalue +- src/minisketch + - Upstream at https://github.com/sipa/minisketch ; maintained by Core contributors. Upgrading LevelDB --------------------- @@ -1119,7 +1289,10 @@ A few guidelines for introducing and reviewing new RPC interfaces: - *Rationale*: Consistency with the existing interface. -- Argument naming: use snake case `fee_delta` (and not, e.g. camel case `feeDelta`) +- Argument and field naming: please consider whether there is already a naming + style or spelling convention in the API for the type of object in question + (`blockhash`, for example), and if so, try to use that. If not, use snake case + `fee_delta` (and not, e.g. `feedelta` or camel case `feeDelta`). - *Rationale*: Consistency with the existing interface. @@ -1158,7 +1331,7 @@ A few guidelines for introducing and reviewing new RPC interfaces: - Don't forget to fill in the argument names correctly in the RPC command table. - - *Rationale*: If not, the call can not be used with name-based arguments. + - *Rationale*: If not, the call cannot be used with name-based arguments. - Add every non-string RPC argument `(method, idx, name)` to the table `vRPCConvertParams` in `rpc/client.cpp`. @@ -1213,6 +1386,12 @@ A few guidelines for introducing and reviewing new RPC interfaces: - *Rationale*: User-facing consistency. +- Use `fs::path::u8string()` and `fs::u8path()` functions when converting path + to JSON strings, not `fs::PathToString` and `fs::PathFromString` + + - *Rationale*: JSON strings are Unicode strings, not byte strings, and + RFC8259 requires JSON to be encoded as UTF-8. + Internal interface guidelines ----------------------------- @@ -1279,22 +1458,9 @@ communication: virtual boost::signals2::scoped_connection connectTipChanged(TipChangedFn fn) = 0; ``` -- For consistency and friendliness to code generation tools, interface method - input and inout parameters should be ordered first and output parameters - should come last. +- Interface methods should not be overloaded. - Example: - - ```c++ - // Good: error output param is last - virtual bool broadcastTransaction(const CTransactionRef& tx, CAmount max_fee, std::string& error) = 0; - - // Bad: error output param is between input params - virtual bool broadcastTransaction(const CTransactionRef& tx, std::string& error, CAmount max_fee) = 0; - ``` - -- For friendliness to code generation tools, interface methods should not be - overloaded: + *Rationale*: consistency and friendliness to code generation tools. Example: @@ -1308,10 +1474,13 @@ communication: virtual bool disconnect(NodeId id) = 0; ``` -- For consistency and friendliness to code generation tools, interface method - names should be `lowerCamelCase` and standalone function names should be +### Internal interface naming style + +- Interface method names should be `lowerCamelCase` and standalone function names should be `UpperCamelCase`. + *Rationale*: consistency and friendliness to code generation tools. + Examples: ```c++ diff --git a/doc/fuzzing.md b/doc/fuzzing.md index 6605749557..9abfbc9213 100644 --- a/doc/fuzzing.md +++ b/doc/fuzzing.md @@ -16,6 +16,13 @@ $ FUZZ=process_message src/test/fuzz/fuzz # abort fuzzing using ctrl-c ``` +There is also a runner script to execute all fuzz targets. Refer to +`./test/fuzz/test_runner.py --help` for more details. + +## Overview of Bitcoin Core fuzzing + +[Google](https://github.com/google/fuzzing/) has a good overview of fuzzing in general, with contributions from key architects of some of the most-used fuzzers. [This paper](https://agroce.github.io/bitcoin_report.pdf) includes an external overview of the status of Bitcoin Core fuzzing, as of summer 2021. [John Regehr](https://blog.regehr.org/archives/1687) provides good advice on writing code that assists fuzzers in finding bugs, which is useful for developers to keep in mind. + ## Fuzzing harnesses and output [`process_message`](https://github.com/bitcoin/bitcoin/blob/master/src/test/fuzz/process_message.cpp) is a fuzzing harness for the [`ProcessMessage(...)` function (`net_processing`)](https://github.com/bitcoin/bitcoin/blob/master/src/net_processing.cpp). The available fuzzing harnesses are found in [`src/test/fuzz/`](https://github.com/bitcoin/bitcoin/tree/master/src/test/fuzz). @@ -64,6 +71,15 @@ block^@M-^?M-^?M-^?M-^?M-^?nM-^?M-^? In this case the fuzzer managed to create a `block` message which when passed to `ProcessMessage(...)` increased coverage. +It is possible to specify `bitcoind` arguments to the `fuzz` executable. +Depending on the test, they may be ignored or consumed and alter the behavior +of the test. Just make sure to use double-dash to distinguish them from the +fuzzer's own arguments: + +```sh +$ FUZZ=address_deserialize_v2 src/test/fuzz/fuzz -runs=1 fuzz_seed_corpus/address_deserialize_v2 --checkaddrman=5 --printtoconsole=1 +``` + ## Fuzzing corpora The project's collection of seed corpora is found in the [`bitcoin-core/qa-assets`](https://github.com/bitcoin-core/qa-assets) repo. @@ -251,6 +267,73 @@ $ honggfuzz/honggfuzz --exit_upon_crash --quiet --timeout 4 -n 1 -Q \ -debug ``` +# Fuzzing Bitcoin Core using Eclipser (v1.x) + +## Quickstart guide + +To quickly get started fuzzing Bitcoin Core using [Eclipser v1.x](https://github.com/SoftSec-KAIST/Eclipser/tree/v1.x): + +```sh +$ git clone https://github.com/bitcoin/bitcoin +$ cd bitcoin/ +$ sudo vim /etc/apt/sources.list # Uncomment the lines starting with 'deb-src'. +$ sudo apt-get update +$ sudo apt-get build-dep qemu +$ sudo apt-get install libtool libtool-bin wget automake autoconf bison gdb +``` + +At this point, you must install the .NET core. The process differs, depending on your Linux distribution. +See [this link](https://docs.microsoft.com/en-us/dotnet/core/install/linux) for details. +On ubuntu 20.04, the following should work: + +```sh +$ wget -q https://packages.microsoft.com/config/ubuntu/20.04/packages-microsoft-prod.deb +$ sudo dpkg -i packages-microsoft-prod.deb +$ rm packages-microsoft-prod.deb +$ sudo apt-get update +$ sudo apt-get install -y dotnet-sdk-2.1 +``` + +You will also want to make sure Python is installed as `python` for the Eclipser install to succeed. + +```sh +$ git clone https://github.com/SoftSec-KAIST/Eclipser.git +$ cd Eclipser +$ git checkout v1.x +$ make +$ cd .. +$ ./autogen.sh +$ ./configure --enable-fuzz +$ make +$ mkdir -p outputs/ +$ FUZZ=bech32 dotnet Eclipser/build/Eclipser.dll fuzz -p src/test/fuzz/fuzz -t 36000 -o outputs --src stdin +``` + +This will perform 10 hours of fuzzing. + +To make further use of the inputs generated by Eclipser, you +must first decode them: + +```sh +$ dotnet Eclipser/build/Eclipser.dll decode -i outputs/testcase -o decoded_outputs +``` +This will place raw inputs in the directory `decoded_outputs/decoded_stdins`. Crashes are in the `outputs/crashes` directory, and must +be decoded in the same way. + +Fuzzing with Eclipser will likely be much more effective if using an existing corpus: + +```sh +$ git clone https://github.com/bitcoin-core/qa-assets +$ FUZZ=bech32 dotnet Eclipser/build/Eclipser.dll fuzz -p src/test/fuzz/fuzz -t 36000 -i qa-assets/fuzz_seed_corpus/bech32 outputs --src stdin +``` + +Note that fuzzing with Eclipser on certain targets (those that create 'full nodes', e.g. `process_message*`) will, +for now, slowly fill `/tmp/` with improperly cleaned-up files, which will cause spurious crashes. +See [this proposed patch](https://github.com/bitcoin/bitcoin/pull/22472) for more information. + +Read the [Eclipser documentation for v1.x](https://github.com/SoftSec-KAIST/Eclipser/tree/v1.x) for more details on using Eclipser. + + # OSS-Fuzz Bitcoin Core participates in Google's [OSS-Fuzz](https://github.com/google/oss-fuzz/tree/master/projects/bitcoin-core) diff --git a/doc/i2p.md b/doc/i2p.md index 5f631c11ca..39f65c4e5f 100644 --- a/doc/i2p.md +++ b/doc/i2p.md @@ -65,13 +65,9 @@ logging` for more information. -onlynet=i2p ``` -Make outgoing connections only to I2P addresses. Incoming connections are not -affected by this option. It can be specified multiple times to allow multiple -network types, e.g. onlynet=ipv4, onlynet=ipv6, onlynet=onion, onlynet=i2p. - -Warning: if you use -onlynet with values other than onion, and the -onion or --proxy option is set, then outgoing onion connections will still be made; use --noonion or -onion=0 to disable outbound onion connections in this case. +Make automatic outbound connections only to I2P addresses. Inbound and manual +connections are not affected by this option. It can be specified multiple times +to allow multiple networks, e.g. onlynet=onion, onlynet=i2p. I2P support was added to Bitcoin Core in version 22.0 and there may be fewer I2P peers than Tor or IP ones. Therefore, using I2P alone without other networks may @@ -84,15 +80,15 @@ phase when syncing up a new node can be very slow. This phase can be sped up by using other networks, for instance `onlynet=onion`, at the same time. In general, a node can be run with both onion and I2P hidden services (or -any/all of IPv4/IPv6/onion/I2P), which can provide a potential fallback if one -of the networks has issues. +any/all of IPv4/IPv6/onion/I2P/CJDNS), which can provide a potential fallback if +one of the networks has issues. ## I2P-related information in Bitcoin Core There are several ways to see your I2P address in Bitcoin Core: -- in the debug log (grep for `AddLocal`, the I2P address ends in `.b32.i2p`) -- in the output of the `getnetworkinfo` RPC in the "localaddresses" section -- in the output of `bitcoin-cli -netinfo` peer connections dashboard +- in the "Local addresses" output of CLI `-netinfo` +- in the "localaddresses" output of RPC `getnetworkinfo` +- in the debug log (grep for `AddLocal`; the I2P address ends in `.b32.i2p`) To see which I2P peers your node is connected to, use `bitcoin-cli -netinfo 4` or the `getpeerinfo` RPC (e.g. `bitcoin-cli getpeerinfo`). diff --git a/doc/managing-wallets.md b/doc/managing-wallets.md index aab6d131bd..6c1e13c503 100644 --- a/doc/managing-wallets.md +++ b/doc/managing-wallets.md @@ -12,11 +12,9 @@ In the GUI, the `Create a new wallet` button is displayed on the main screen whe The following command, for example, creates a descriptor wallet. More information about this command may be found by running `bitcoin-cli help createwallet`. ``` -$ bitcoin-cli -named createwallet wallet_name="wallet-01" descriptors=true +$ bitcoin-cli createwallet "wallet-01" ``` -The `descriptors` parameter can be omitted if the intention is to create a legacy wallet. For now, the default type is the legacy wallet, but that is expected to change in a future release. - By default, wallets are created in the `wallets` folder of the data directory, which varies by operating system, as shown below. The user can change the default by using the `-datadir` or `-walletdir` initialization parameters. | Operating System | Default wallet directory | @@ -54,7 +52,7 @@ Only the wallet's private key is encrypted. All other wallet information, such a The wallet's private key can also be encrypted in the `createwallet` command via the `passphrase` argument: ``` -$ bitcoin-cli -named createwallet wallet_name="wallet-01" descriptors=true passphrase="passphrase" +$ bitcoin-cli -named createwallet wallet_name="wallet-01" passphrase="passphrase" ``` Note that if the passphrase is lost, all the coins in the wallet will also be lost forever. diff --git a/doc/multisig-tutorial.md b/doc/multisig-tutorial.md new file mode 100644 index 0000000000..1d2b3244bc --- /dev/null +++ b/doc/multisig-tutorial.md @@ -0,0 +1,241 @@ +# 1. Multisig Tutorial + +Currently, it is possible to create a multisig wallet using Bitcoin Core only. + +Although there is already a brief explanation about the multisig in the [Descriptors documentation](https://github.com/bitcoin/bitcoin/blob/master/doc/descriptors.md#multisig), this tutorial proposes to use the signet (instead of regtest), bringing the reader closer to a real environment and explaining some functions in more detail. + +This tutorial uses [jq](https://github.com/stedolan/jq) JSON processor to process the results from RPC and stores the relevant values in bash variables. This makes the tutorial reproducible and easier to follow step by step. + +Before starting this tutorial, start the bitcoin node on the signet network. + +```bash +./src/bitcoind -signet -daemon +``` + +This tutorial also uses the default WPKH derivation path to get the xpubs and does not conform to [BIP 45](https://github.com/bitcoin/bips/blob/master/bip-0045.mediawiki) or [BIP 87](https://github.com/bitcoin/bips/blob/master/bip-0087.mediawiki). + +At the time of writing, there is no way to extract a specific path from wallets in Bitcoin Core. For this, an external signer/xpub can be used. + +[PR #22341](https://github.com/bitcoin/bitcoin/pull/22341), which is still under development, introduces a new wallet RPC `getxpub`. It takes a BIP32 path as an argument and returns the xpub, along with the master key fingerprint. + +## 1.1 Basic Multisig Workflow + +### 1.1 Create the Descriptor Wallets + +For a 2-of-3 multisig, create 3 descriptor wallets. It is important that they are of the descriptor type in order to retrieve the wallet descriptors. These wallets contain HD seed and private keys, which will be used to sign the PSBTs and derive the xpub. + +These three wallets should not be used directly for privacy reasons (public key reuse). They should only be used to sign transactions for the (watch-only) multisig wallet. + +```bash +for ((n=1;n<=3;n++)) +do + ./src/bitcoin-cli -signet createwallet "participant_${n}" +done +``` + +Extract the xpub of each wallet. To do this, the `listdescriptors` RPC is used. By default, Bitcoin Core single-sig wallets are created using path `m/44'/1'/0'` for PKH, `m/84'/1'/0'` for WPKH, `m/49'/1'/0'` for P2WPKH-nested-in-P2SH and `m/86'/1'/0'` for P2TR based accounts. Each of them uses the chain 0 for external addresses and chain 1 for internal ones, as shown in the example below. + +``` +wpkh([1004658e/84'/1'/0']tpubDCBEcmVKbfC9KfdydyLbJ2gfNL88grZu1XcWSW9ytTM6fitvaRmVyr8Ddf7SjZ2ZfMx9RicjYAXhuh3fmLiVLPodPEqnQQURUfrBKiiVZc8/0/*)#g8l47ngv + +wpkh([1004658e/84'/1'/0']tpubDCBEcmVKbfC9KfdydyLbJ2gfNL88grZu1XcWSW9ytTM6fitvaRmVyr8Ddf7SjZ2ZfMx9RicjYAXhuh3fmLiVLPodPEqnQQURUfrBKiiVZc8/1/*)#en65rxc5 +``` + +The suffix (after #) is the checksum. Descriptors can optionally be suffixed with a checksum to protect against typos or copy-paste errors. +All RPCs in Bitcoin Core will include the checksum in their output. + +```bash +declare -A xpubs + +for ((n=1;n<=3;n++)) +do + xpubs["internal_xpub_${n}"]=$(./src/bitcoin-cli -signet -rpcwallet="participant_${n}" listdescriptors | jq '.descriptors | [.[] | select(.desc | startswith("wpkh") and contains("/1/*"))][0] | .desc' | grep -Po '(?<=\().*(?=\))') + + xpubs["external_xpub_${n}"]=$(./src/bitcoin-cli -signet -rpcwallet="participant_${n}" listdescriptors | jq '.descriptors | [.[] | select(.desc | startswith("wpkh") and contains("/0/*") )][0] | .desc' | grep -Po '(?<=\().*(?=\))') +done +``` + +`jq` is used to extract the xpub from the `wpkh` descriptor. + +The following command can be used to verify if the xpub was generated correctly. + +```bash +for x in "${!xpubs[@]}"; do printf "[%s]=%s\n" "$x" "${xpubs[$x]}" ; done +``` + +As previously mentioned, this step extracts the `m/84'/1'/0'` account instead of the path defined in [BIP 45](https://github.com/bitcoin/bips/blob/master/bip-0045.mediawiki) or [BIP 87](https://github.com/bitcoin/bips/blob/master/bip-0087.mediawiki), since there is no way to extract a specific path in Bitcoin Core at the time of writing. + +### 1.2 Define the Multisig Descriptors + +Define the external and internal multisig descriptors, add the checksum and then, join both in a JSON array. + +```bash +external_desc="wsh(sortedmulti(2,${xpubs["external_xpub_1"]},${xpubs["external_xpub_2"]},${xpubs["external_xpub_3"]}))" +internal_desc="wsh(sortedmulti(2,${xpubs["internal_xpub_1"]},${xpubs["internal_xpub_2"]},${xpubs["internal_xpub_3"]}))" + +external_desc_sum=$(./src/bitcoin-cli -signet getdescriptorinfo $external_desc | jq '.descriptor') +internal_desc_sum=$(./src/bitcoin-cli -signet getdescriptorinfo $internal_desc | jq '.descriptor') + +multisig_ext_desc="{\"desc\": $external_desc_sum, \"active\": true, \"internal\": false, \"timestamp\": \"now\"}" +multisig_int_desc="{\"desc\": $internal_desc_sum, \"active\": true, \"internal\": true, \"timestamp\": \"now\"}" + +multisig_desc="[$multisig_ext_desc, $multisig_int_desc]" +``` + +`external_desc` and `internal_desc` specify the output type (`wsh`, in this case) and the xpubs involved. They also use BIP 67 (`sortedmulti`), so the wallet can be recreated without worrying about the order of xpubs. Conceptually, descriptors describe a list of scriptPubKey (along with information for spending from it) [[source](https://github.com/bitcoin/bitcoin/issues/21199#issuecomment-780772418)]. + +Note that at least two descriptors are usually used, one for internal derivation paths and external ones. There are discussions about eliminating this redundancy, as can been seen in the issue [#17190](https://github.com/bitcoin/bitcoin/issues/17190). + +After creating the descriptors, it is necessary to add the checksum, which is required by the `importdescriptors` RPC. + +The checksum for a descriptor without one can be computed using the `getdescriptorinfo` RPC. The response has the `descriptor` field, which is the descriptor with the checksum added. + +There are other fields that can be added to the descriptors: + +* `active`: Sets the descriptor to be the active one for the corresponding output type (`wsh`, in this case). +* `internal`: Indicates whether matching outputs should be treated as something other than incoming payments (e.g. change). +* `timestamp`: Sets the time from which to start rescanning the blockchain for the descriptor, in UNIX epoch time. + +Documentation for these and other parameters can be found by typing `./src/bitcoin-cli help importdescriptors`. + +`multisig_desc` concatenates external and internal descriptors in a JSON array and then it will be used to create the multisig wallet. + +### 1.3 Create the Multisig Wallet + +To create the multisig wallet, first create an empty one (no keys, HD seed and private keys disabled). + +Then import the descriptors created in the previous step using the `importdescriptors` RPC. + +After that, `getwalletinfo` can be used to check if the wallet was created successfully. + +```bash +./src/bitcoin-cli -signet -named createwallet wallet_name="multisig_wallet_01" disable_private_keys=true blank=true + +./src/bitcoin-cli -signet -rpcwallet="multisig_wallet_01" importdescriptors "$multisig_desc" + +./src/bitcoin-cli -signet -rpcwallet="multisig_wallet_01" getwalletinfo +``` + +Once the wallets have already been created and this tutorial needs to be repeated or resumed, it is not necessary to recreate them, just load them with the command below: + +```bash +for ((n=1;n<=3;n++)); do ./src/bitcoin-cli -signet loadwallet "participant_${n}"; done +``` + +### 1.4 Fund the wallet + +The wallet can receive signet coins by generating a new address and passing it as parameters to `getcoins.py` script. + +This script will print a captcha in dot-matrix to the terminal, using unicode Braille characters. After solving the captcha, the coins will be sent directly to the address or wallet (according to the parameters). + +The url used by the script can also be accessed directly. At time of writing, the url is [`https://signetfaucet.com`](https://signetfaucet.com). + +Coins received by the wallet must have at least 1 confirmation before they can be spent. It is necessary to wait for a new block to be mined before continuing. + +```bash +receiving_address=$(./src/bitcoin-cli -signet -rpcwallet="multisig_wallet_01" getnewaddress) + +./contrib/signet/getcoins.py -c ./src/bitcoin-cli -a $receiving_address +``` + +To copy the receiving address onto the clipboard, use the following command. This can be useful when getting coins via the signet faucet mentioned above. + +```bash +echo -n "$receiving_address" | xclip -sel clip +``` + +The `getbalances` RPC may be used to check the balance. Coins with `trusted` status can be spent. + +```bash +./src/bitcoin-cli -signet -rpcwallet="multisig_wallet_01" getbalances +``` + +### 1.5 Create a PSBT + +Unlike singlesig wallets, multisig wallets cannot create and sign transactions directly because they require the signatures of the co-signers. Instead they create a Partially Signed Bitcoin Transaction (PSBT). + +PSBT is a data format that allows wallets and other tools to exchange information about a Bitcoin transaction and the signatures necessary to complete it. [[source](https://bitcoinops.org/en/topics/psbt/)] + +The current PSBT version (v0) is defined in [BIP 174](https://github.com/bitcoin/bips/blob/master/bip-0174.mediawiki). + +For simplicity, the destination address is taken from the `participant_1` wallet in the code above, but it can be any valid bitcoin address. + +The `walletcreatefundedpsbt` RPC is used to create and fund a transaction in the PSBT format. It is the first step in creating the PSBT. + +```bash +balance=$(./src/bitcoin-cli -signet -rpcwallet="multisig_wallet_01" getbalance) + +amount=$(echo "$balance * 0.8" | bc -l | sed -e 's/^\./0./' -e 's/^-\./-0./') + +destination_addr=$(./src/bitcoin-cli -signet -rpcwallet="participant_1" getnewaddress) + +funded_psbt=$(./src/bitcoin-cli -signet -named -rpcwallet="multisig_wallet_01" walletcreatefundedpsbt outputs="{\"$destination_addr\": $amount}" | jq -r '.psbt') +``` + +There is also the `createpsbt` RPC, which serves the same purpose, but it has no access to the wallet or to the UTXO set. It is functionally the same as `createrawtransaction` and just drops the raw transaction into an otherwise blank PSBT. [[source](https://bitcointalk.org/index.php?topic=5131043.msg50573609#msg50573609)] In most cases, `walletcreatefundedpsbt` solves the problem. + +The `send` RPC can also return a PSBT if more signatures are needed to sign the transaction. + +### 1.6 Decode or Analyze the PSBT + +Optionally, the PSBT can be decoded to a JSON format using `decodepsbt` RPC. + +The `analyzepsbt` RPC analyzes and provides information about the current status of a PSBT and its inputs, e.g. missing signatures. + +```bash +./src/bitcoin-cli -signet decodepsbt $funded_psbt + +./src/bitcoin-cli -signet analyzepsbt $funded_psbt +``` + +### 1.7 Update the PSBT + +In the code above, two PSBTs are created. One signed by `participant_1` wallet and other, by the `participant_2` wallet. + +The `walletprocesspsbt` is used by the wallet to sign a PSBT. + +```bash +psbt_1=$(./src/bitcoin-cli -signet -rpcwallet="participant_1" walletprocesspsbt $funded_psbt | jq '.psbt') + +psbt_2=$(./src/bitcoin-cli -signet -rpcwallet="participant_2" walletprocesspsbt $funded_psbt | jq '.psbt') +``` + +### 1.8 Combine the PSBT + +The PSBT, if signed separately by the co-signers, must be combined into one transaction before being finalized. This is done by `combinepsbt` RPC. + +```bash +combined_psbt=$(./src/bitcoin-cli -signet combinepsbt "[$psbt_1, $psbt_2]") +``` + +There is an RPC called `joinpsbts`, but it has a different purpose than `combinepsbt`. `joinpsbts` joins the inputs from multiple distinct PSBTs into one PSBT. + +In the example above, the PSBTs are the same, but signed by different participants. If the user tries to merge them using `joinpsbts`, the error `Input txid:pos exists in multiple PSBTs` is returned. To be able to merge different PSBTs into one, they must have different inputs and outputs. + +### 1.9 Finalize and Broadcast the PSBT + +The `finalizepsbt` RPC is used to produce a network serialized transaction which can be broadcast with `sendrawtransaction`. + +It checks that all inputs have complete scriptSigs and scriptWitnesses and, if so, encodes them into network serialized transactions. + +```bash +finalized_psbt_hex=$(./src/bitcoin-cli -signet finalizepsbt $combined_psbt | jq -r '.hex') + +./src/bitcoin-cli -signet sendrawtransaction $finalized_psbt_hex +``` + +### 1.10 Alternative Workflow (PSBT sequential signatures) + +Instead of each wallet signing the original PSBT and combining them later, the wallets can also sign the PSBTs sequentially. This is less scalable than the previously presented parallel workflow, but it works. + +After that, the rest of the process is the same: the PSBT is finalized and transmitted to the network. + +```bash +psbt_1=$(./src/bitcoin-cli -signet -rpcwallet="participant_1" walletprocesspsbt $funded_psbt | jq -r '.psbt') + +psbt_2=$(./src/bitcoin-cli -signet -rpcwallet="participant_2" walletprocesspsbt $psbt_1 | jq -r '.psbt') + +finalized_psbt_hex=$(./src/bitcoin-cli -signet finalizepsbt $psbt_2 | jq -r '.hex') + +./src/bitcoin-cli -signet sendrawtransaction $finalized_psbt_hex +``` diff --git a/doc/p2p-bad-ports.md b/doc/p2p-bad-ports.md new file mode 100644 index 0000000000..4f717f97a2 --- /dev/null +++ b/doc/p2p-bad-ports.md @@ -0,0 +1,114 @@ +When Bitcoin Core automatically opens outgoing P2P connections, it chooses +a peer (address and port) from its list of potential peers. This list is +populated with unchecked data gossiped over the P2P network by other peers. + +A malicious actor may gossip an address:port where no Bitcoin node is listening, +or one where a service is listening that is not related to the Bitcoin network. +As a result, this service may occasionally get connection attempts from Bitcoin +nodes. + +"Bad" ports are ones used by services which are usually not open to the public +and usually require authentication. A connection attempt (by Bitcoin Core, +trying to connect because it thinks there is a Bitcoin node on that +address:port) to such service may be considered a malicious action by an +ultra-paranoid administrator. An example for such a port is 22 (ssh). On the +other hand, connection attempts to public services that usually do not require +authentication are unlikely to be considered a malicious action, +e.g. port 80 (http). + +Below is a list of "bad" ports which Bitcoin Core avoids when choosing a peer to +connect to. If a node is listening on such a port, it will likely receive fewer +incoming connections. + + 1: tcpmux + 7: echo + 9: discard + 11: systat + 13: daytime + 15: netstat + 17: qotd + 19: chargen + 20: ftp data + 21: ftp access + 22: ssh + 23: telnet + 25: smtp + 37: time + 42: name + 43: nicname + 53: domain + 69: tftp + 77: priv-rjs + 79: finger + 87: ttylink + 95: supdup + 101: hostname + 102: iso-tsap + 103: gppitnp + 104: acr-nema + 109: pop2 + 110: pop3 + 111: sunrpc + 113: auth + 115: sftp + 117: uucp-path + 119: nntp + 123: NTP + 135: loc-srv /epmap + 137: netbios + 139: netbios + 143: imap2 + 161: snmp + 179: BGP + 389: ldap + 427: SLP (Also used by Apple Filing Protocol) + 465: smtp+ssl + 512: print / exec + 513: login + 514: shell + 515: printer + 526: tempo + 530: courier + 531: chat + 532: netnews + 540: uucp + 548: AFP (Apple Filing Protocol) + 554: rtsp + 556: remotefs + 563: nntp+ssl + 587: smtp (rfc6409) + 601: syslog-conn (rfc3195) + 636: ldap+ssl + 989: ftps-data + 990: ftps + 993: ldap+ssl + 995: pop3+ssl + 1719: h323gatestat + 1720: h323hostcall + 1723: pptp + 2049: nfs + 3659: apple-sasl / PasswordServer + 4045: lockd + 5060: sip + 5061: sips + 6000: X11 + 6566: sane-port + 6665: Alternate IRC + 6666: Alternate IRC + 6667: Standard IRC + 6668: Alternate IRC + 6669: Alternate IRC + 6697: IRC + TLS + 10080: Amanda + +For further information see: + +[pull/23306](https://github.com/bitcoin/bitcoin/pull/23306#issuecomment-947516736) + +[pull/23542](https://github.com/bitcoin/bitcoin/pull/23542) + +[fetch.spec.whatwg.org](https://fetch.spec.whatwg.org/#port-blocking) + +[chromium.googlesource.com](https://chromium.googlesource.com/chromium/src.git/+/refs/heads/main/net/base/port_util.cc) + +[hg.mozilla.org](https://hg.mozilla.org/mozilla-central/file/tip/netwerk/base/nsIOService.cpp) diff --git a/doc/policy/README.md b/doc/policy/README.md new file mode 100644 index 0000000000..6e8686365d --- /dev/null +++ b/doc/policy/README.md @@ -0,0 +1,15 @@ +# Transaction Relay Policy + +**Policy** (Mempool or Transaction Relay Policy) is the node's set of validation rules, in addition +to consensus, enforced for unconfirmed transactions before submitting them to the mempool. These +rules are local to the node and configurable (e.g. `-minrelaytxfee`, `-limitancestorsize`, +`-incrementalRelayFee`). Policy may include restrictions on the transaction itself, the transaction +in relation to the current chain tip, and the transaction in relation to the node's mempool +contents. Policy is *not* applied to transactions in blocks. + +This documentation is not an exhaustive list of all policy rules. + +- [Mempool Limits](mempool-limits.md) +- [Mempool Replacements](mempool-replacements.md) +- [Packages](packages.md) + diff --git a/doc/policy/mempool-limits.md b/doc/policy/mempool-limits.md new file mode 100644 index 0000000000..73ab017f1b --- /dev/null +++ b/doc/policy/mempool-limits.md @@ -0,0 +1,65 @@ +# Mempool Limits + +## Definitions + +Given any two transactions Tx0 and Tx1 where Tx1 spends an output of Tx0, +Tx0 is a *parent* of Tx1 and Tx1 is a *child* of Tx0. + +A transaction's *ancestors* include, recursively, its parents, the parents of its parents, etc. +A transaction's *descendants* include, recursively, its children, the children of its children, etc. + +A mempool entry's *ancestor count* is the total number of in-mempool (unconfirmed) transactions in +its ancestor set, including itself. +A mempool entry's *descendant count* is the total number of in-mempool (unconfirmed) transactions in +its descendant set, including itself. + +A mempool entry's *ancestor size* is the aggregated virtual size of in-mempool (unconfirmed) +transactions in its ancestor set, including itself. +A mempool entry's *descendant size* is the aggregated virtual size of in-mempool (unconfirmed) +transactions in its descendant set, including itself. + +Transactions submitted to the mempool must not exceed the ancestor and descendant limits (aka +mempool *package limits*) set by the node (see `-limitancestorcount`, `-limitancestorsize`, +`-limitdescendantcount`, `-limitdescendantsize`). + +## Exemptions + +### CPFP Carve Out + +**CPFP Carve Out** if a transaction candidate for submission to the +mempool would cause some mempool entry to exceed its descendant limits, an exemption is made if all +of the following conditions are met: + +1. The candidate transaction is no more than 10,000 virtual bytes. + +2. The candidate transaction has an ancestor count of 2 (itself and exactly 1 ancestor). + +3. The in-mempool transaction's descendant count, including the candidate transaction, would only + exceed the limit by 1. + +*Rationale*: this rule was introduced to prevent pinning by domination of a transaction's descendant +limits in two-party contract protocols such as LN. Also see the [mailing list +post](https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2018-November/016518.html). + +This rule was introduced in [PR #15681](https://github.com/bitcoin/bitcoin/pull/15681). + +### Single-Conflict RBF Carve Out + +When a candidate transaction for submission to the mempool would replace mempool entries, it may +also decrease the descendant count of other mempool entries. Since ancestor/descendant limits are +calculated prior to removing the would-be-replaced transactions, they may be overestimated. + +An exemption is given for a candidate transaction that would replace mempool transactions and meets +all of the following conditions: + +1. The candidate transaction has exactly 1 directly conflicting transaction. + +2. The candidate transaction does not spend any unconfirmed inputs that are not also spent by the + directly conflicting transaction. + +The following discounts are given to account for the would-be-replaced transaction(s): + +1. The descendant count limit is temporarily increased by 1. + +2. The descendant size limit temporarily is increased by the virtual size of the to-be-replaced + directly conflicting transaction. diff --git a/doc/policy/mempool-replacements.md b/doc/policy/mempool-replacements.md new file mode 100644 index 0000000000..18f08daf88 --- /dev/null +++ b/doc/policy/mempool-replacements.md @@ -0,0 +1,76 @@ +# Mempool Replacements + +## Current Replace-by-Fee Policy + +A transaction conflicts with an in-mempool transaction ("directly conflicting transaction") if they +spend one or more of the same inputs. A transaction may conflict with multiple in-mempool +transactions. + +A transaction ("replacement transaction") may replace its directly conflicting transactions and +their in-mempool descendants (together, "original transactions") if, in addition to passing all +other consensus and policy rules, each of the following conditions are met: + +1. The directly conflicting transactions all signal replaceability explicitly. A transaction is + signaling replaceability if any of its inputs have an nSequence number less than (0xffffffff - 1). + + *Rationale*: See [BIP125 + explanation](https://github.com/bitcoin/bips/blob/master/bip-0125.mediawiki#motivation). + +2. The replacement transaction only include an unconfirmed input if that input was included in + one of the directly conflicting transactions. An unconfirmed input spends an output from a + currently-unconfirmed transaction. + + *Rationale*: When RBF was originally implemented, the mempool did not keep track of + ancestor feerates yet. This rule was suggested as a temporary restriction. + +3. The replacement transaction pays an absolute fee of at least the sum paid by the original + transactions. + + *Rationale*: Only requiring the replacement transaction to have a higher feerate could allow an + attacker to bypass node minimum relay feerate requirements and cause the network to repeatedly + relay slightly smaller replacement transactions without adding any more fees. Additionally, if + any of the original transactions would be included in the next block assembled by an economically + rational miner, a replacement policy allowing the replacement transaction to decrease the absolute + fees in the next block would be incentive-incompatible. + +4. The additional fees (difference between absolute fee paid by the replacement transaction and the + sum paid by the original transactions) pays for the replacement transaction's bandwidth at or + above the rate set by the node's incremental relay feerate. For example, if the incremental relay + feerate is 1 satoshi/vB and the replacement transaction is 500 virtual bytes total, then the + replacement pays a fee at least 500 satoshis higher than the sum of the original transactions. + + *Rationale*: Try to prevent DoS attacks where an attacker causes the network to repeatedly relay + transactions each paying a tiny additional amount in fees, e.g. just 1 satoshi. + +5. The number of original transactions does not exceed 100. More precisely, the sum of all + directly conflicting transactions' descendant counts (number of transactions inclusive of itself + and its descendants) must not exceed 100; it is possible that this overestimates the true number + of original transactions. + + *Rationale*: Try to prevent DoS attacks where an attacker is able to easily occupy and flush out + significant portions of the node's mempool using replacements with multiple directly conflicting + transactions, each with large descendant sets. + +6. The replacement transaction's feerate is greater than the feerates of all directly conflicting + transactions. + + *Rationale*: This rule was originally intended to ensure that the replacement transaction is + preferable for block-inclusion, compared to what would be removed from the mempool. This rule + predates ancestor feerate-based transaction selection. + +This set of rules is similar but distinct from BIP125. + +## History + +* Opt-in full replace-by-fee (without inherited signaling) honoured in mempool and mining as of + **v0.12.0** ([PR 6871](https://github.com/bitcoin/bitcoin/pull/6871)). + +* [BIP125](https://github.com/bitcoin/bips/blob/master/bip-0125.mediawiki) defined based on + Bitcoin Core implementation. + +* The incremental relay feerate used to calculate the required additional fees is distinct from + `minRelayTxFee` and configurable using `-incrementalrelayfee` + ([PR #9380](https://github.com/bitcoin/bitcoin/pull/9380)). + +* RBF enabled by default in the wallet GUI as of **v0.18.1** ([PR + #11605](https://github.com/bitcoin/bitcoin/pull/11605)). diff --git a/doc/policy/packages.md b/doc/policy/packages.md new file mode 100644 index 0000000000..f2a3d6517e --- /dev/null +++ b/doc/policy/packages.md @@ -0,0 +1,119 @@ +# Package Mempool Accept + +## Definitions + +A **package** is an ordered list of transactions, representable by a connected Directed Acyclic +Graph (a directed edge exists between a transaction that spends the output of another transaction). + +For every transaction `t` in a **topologically sorted** package, if any of its parents are present +in the package, they appear somewhere in the list before `t`. + +A **child-with-unconfirmed-parents** package is a topologically sorted package that consists of +exactly one child and all of its unconfirmed parents (no other transactions may be present). +The last transaction in the package is the child, and its package can be canonically defined based +on the current state: each of its inputs must be available in the UTXO set as of the current chain +tip or some preceding transaction in the package. + +## Package Mempool Acceptance Rules + +The following rules are enforced for all packages: + +* Packages cannot exceed `MAX_PACKAGE_COUNT=25` count and `MAX_PACKAGE_SIZE=101KvB` total size + (#20833) + + - *Rationale*: This is already enforced as mempool ancestor/descendant limits. If + transactions in a package are all related, exceeding this limit would mean that the package + can either be split up or it wouldn't pass individual mempool policy. + + - Note that, if these mempool limits change, package limits should be reconsidered. Users may + also configure their mempool limits differently. + +* Packages must be topologically sorted. (#20833) + +* Packages cannot have conflicting transactions, i.e. no two transactions in a package can spend + the same inputs. Packages cannot have duplicate transactions. (#20833) + +* No transaction in a package can conflict with a mempool transaction. BIP125 Replace By Fee is + currently disabled for packages. (#20833) + + - Package RBF may be enabled in the future. + +* When packages are evaluated against ancestor/descendant limits, the union of all transactions' + descendants and ancestors is considered. (#21800) + + - *Rationale*: This is essentially a "worst case" heuristic intended for packages that are + heavily connected, i.e. some transaction in the package is the ancestor or descendant of all + the other transactions. + +The following rules are only enforced for packages to be submitted to the mempool (not enforced for +test accepts): + +* Packages must be child-with-unconfirmed-parents packages. This also means packages must contain at + least 2 transactions. (#22674) + + - *Rationale*: This allows for fee-bumping by CPFP. Allowing multiple parents makes it possible + to fee-bump a batch of transactions. Restricting packages to a defined topology is easier to + reason about and simplifies the validation logic greatly. + + - Warning: Batched fee-bumping may be unsafe for some use cases. Users and application developers + should take caution if utilizing multi-parent packages. + +* Transactions in the package that have the same txid as another transaction already in the mempool + will be removed from the package prior to submission ("deduplication"). + + - *Rationale*: Node operators are free to set their mempool policies however they please, nodes + may receive transactions in different orders, and malicious counterparties may try to take + advantage of policy differences to pin or delay propagation of transactions. As such, it's + possible for some package transaction(s) to already be in the mempool, and there is no need to + repeat validation for those transactions or double-count them in fees. + + - *Rationale*: We want to prevent potential censorship vectors. We should not reject entire + packages because we already have one of the transactions. Also, if an attacker first broadcasts + a competing package or transaction with a mutated witness, even though the two + same-txid-different-witness transactions are conflicting and cannot replace each other, the + honest package should still be considered for acceptance. + +### Package Fees and Feerate + +*Package Feerate* is the total modified fees (base fees + any fee delta from +`prioritisetransaction`) divided by the total virtual size of all transactions in the package. +If any transactions in the package are already in the mempool, they are not submitted again +("deduplicated") and are thus excluded from this calculation. + +To meet the two feerate requirements of a mempool, i.e., the pre-configured minimum relay feerate +(`minRelayTxFee`) and the dynamic mempool minimum feerate, the total package feerate is used instead +of the individual feerate. The individual transactions are allowed to be below the feerate +requirements if the package meets the feerate requirements. For example, the parent(s) in the +package can pay no fees but be paid for by the child. + +*Rationale*: This can be thought of as "CPFP within a package," solving the issue of a parent not +meeting minimum fees on its own. This would allow contracting applications to adjust their fees at +broadcast time instead of overshooting or risking becoming stuck or pinned. + +*Rationale*: It would be incorrect to use the fees of transactions that are already in the mempool, as +we do not want a transaction's fees to be double-counted. + +Implementation Note: Transactions within a package are always validated individually first, and +package validation is used for the transactions that failed. Since package feerate is only +calculated using transactions that are not in the mempool, this implementation detail affects the +outcome of package validation. + +*Rationale*: Packages are intended for incentive-compatible fee-bumping: transaction B is a +"legitimate" fee-bump for transaction A only if B is a descendant of A and has a *higher* feerate +than A. We want to prevent "parents pay for children" behavior; fees of parents should not help +their children, since the parents can be mined without the child. More generally, if transaction A +is not needed in order for transaction B to be mined, A's fees cannot help B. In a +child-with-parents package, simply excluding any parent transactions that meet feerate requirements +individually is sufficient to ensure this. + +*Rationale*: We must not allow a low-feerate child to prevent its parent from being accepted; fees +of children should not negatively impact their parents, since they are not necessary for the parents +to be mined. More generally, if transaction B is not needed in order for transaction A to be mined, +B's fees cannot harm A. In a child-with-parents package, simply validating parents individually +first is sufficient to ensure this. + +*Rationale*: As a principle, we want to avoid accidentally restricting policy in order to be +backward-compatible for users and applications that rely on p2p transaction relay. Concretely, +package validation should not prevent the acceptance of a transaction that would otherwise be +policy-valid on its own. By always accepting a transaction that passes individual validation before +trying package validation, we prevent any unintentional restriction of policy. diff --git a/doc/productivity.md b/doc/productivity.md index a01c6f545d..e9b7bc270c 100644 --- a/doc/productivity.md +++ b/doc/productivity.md @@ -9,6 +9,7 @@ Table of Contents * [Disable features with `./configure`](#disable-features-with-configure) * [Make use of your threads with `make -j`](#make-use-of-your-threads-with-make--j) * [Only build what you need](#only-build-what-you-need) + * [Compile on multiple machines](#compile-on-multiple-machines) * [Multiple working directories with `git worktrees`](#multiple-working-directories-with-git-worktrees) * [Interactive "dummy rebases" for fixups and execs with `git merge-base`](#interactive-dummy-rebases-for-fixups-and-execs-with-git-merge-base) * [Writing code](#writing-code) @@ -81,6 +82,10 @@ make -C src bitcoin_bench (You can and should combine this with `-j`, as above, for a parallel build.) +### Compile on multiple machines + +If you have more than one computer at your disposal, you can use [distcc](https://www.distcc.org) to speed up compilation. This is easiest when all computers run the same operating system and compiler version. + ### Multiple working directories with `git worktrees` If you work with multiple branches or multiple copies of the repository, you should try `git worktrees`. diff --git a/doc/psbt.md b/doc/psbt.md index c411b31d5d..0f31cb8eba 100644 --- a/doc/psbt.md +++ b/doc/psbt.md @@ -92,6 +92,9 @@ hardware implementations will typically implement multiple roles simultaneously. #### Multisig with multiple Bitcoin Core instances +For a quick start see [Basic M-of-N multisig example using descriptor wallets and PSBTs](./descriptors.md#basic-multisig-example). +If you are using legacy wallets feel free to continue with the example provided here. + Alice, Bob, and Carol want to create a 2-of-3 multisig address. They're all using Bitcoin Core. We assume their wallets only contain the multisig funds. In case they also have a personal wallet, this can be accomplished through the diff --git a/doc/release-notes-15936.md b/doc/release-notes-15936.md new file mode 100644 index 0000000000..90c0413b9a --- /dev/null +++ b/doc/release-notes-15936.md @@ -0,0 +1,15 @@ +GUI changes +----------- + +Configuration changes made in the bitcoin GUI (such as the pruning setting, +proxy settings, UPNP preferences) are now saved to `<datadir>/settings.json` +file rather than to the Qt settings backend (windows registry or unix desktop +config files), so these settings will now apply to bitcoind, instead of being +ignored. + +Also, the interaction between GUI settings and `bitcoin.conf` settings is +simplified. Settings from `bitcoin.conf` are now displayed normally in the GUI +settings dialog, instead of in a separate warning message ("Options set in this +dialog are overridden by the configuration file: -setting=value"). And these +settings can now be edited because `settings.json` values take precedence over +`bitcoin.conf` values. diff --git a/doc/release-notes-24098.md b/doc/release-notes-24098.md new file mode 100644 index 0000000000..79e047e9a5 --- /dev/null +++ b/doc/release-notes-24098.md @@ -0,0 +1,22 @@ +Notable changes +=============== + +Updated REST APIs +----------------- + +- The `/headers/` and `/blockfilterheaders/` endpoints have been updated to use + a query parameter instead of path parameter to specify the result count. The + count parameter is now optional, and defaults to 5 for both endpoints. The old + endpoints are still functional, and have no documented behaviour change. + + For `/headers`, use + `GET /rest/headers/<BLOCK-HASH>.<bin|hex|json>?count=<COUNT=5>` + instead of + `GET /rest/headers/<COUNT>/<BLOCK-HASH>.<bin|hex|json>` (deprecated) + + For `/blockfilterheaders/`, use + `GET /rest/blockfilterheaders/<FILTERTYPE>/<BLOCK-HASH>.<bin|hex|json>?count=<COUNT=5>` + instead of + `GET /rest/blockfilterheaders/<FILTERTYPE>/<COUNT>/<BLOCK-HASH>.<bin|hex|json>` (deprecated) + + (#24098) diff --git a/doc/release-notes-24118.md b/doc/release-notes-24118.md new file mode 100644 index 0000000000..16f23c7d00 --- /dev/null +++ b/doc/release-notes-24118.md @@ -0,0 +1,10 @@ +New RPCs +-------- + +- The `sendall` RPC spends specific UTXOs to one or more recipients + without creating change. By default, the `sendall` RPC will spend + every UTXO in the wallet. `sendall` is useful to empty wallets or to + create a changeless payment from select UTXOs. When creating a payment + from a specific amount for which the recipient incurs the transaction + fee, continue to use the `subtractfeefromamount` option via the + `send`, `sendtoaddress`, or `sendmany` RPCs. (#24118) diff --git a/doc/release-notes-24198.md b/doc/release-notes-24198.md new file mode 100644 index 0000000000..e41b2a8e26 --- /dev/null +++ b/doc/release-notes-24198.md @@ -0,0 +1,6 @@ +Updated RPCs +------------ + +- The `listtransactions`, `gettransaction`, and `listsinceblock` + RPC methods now include a wtxid field (hash of serialized transaction, + including witness data) for each transaction.
\ No newline at end of file diff --git a/doc/release-notes-24494.md b/doc/release-notes-24494.md new file mode 100644 index 0000000000..afbb926433 --- /dev/null +++ b/doc/release-notes-24494.md @@ -0,0 +1,2 @@ +To help prevent fingerprinting transactions created by the Bitcoin Core wallet, change output +amounts are now randomized. (#24494) diff --git a/doc/release-notes-empty-template.md b/doc/release-notes-empty-template.md new file mode 100644 index 0000000000..8462714898 --- /dev/null +++ b/doc/release-notes-empty-template.md @@ -0,0 +1,99 @@ +*The release notes draft is a temporary file that can be added to by anyone. See +[/doc/developer-notes.md#release-notes](/doc/developer-notes.md#release-notes) +for the process.* + +*version* Release Notes Draft +=============================== + +Bitcoin Core version *version* is now available from: + + <https://bitcoincore.org/bin/bitcoin-core-*version*/> + +This release includes new features, various bug fixes and performance +improvements, as well as updated translations. + +Please report bugs using the issue tracker at GitHub: + + <https://github.com/bitcoin/bitcoin/issues> + +To receive security and update notifications, please subscribe to: + + <https://bitcoincore.org/en/list/announcements/join/> + +How to Upgrade +============== + +If you are running an older version, shut it down. Wait until it has completely +shut down (which might take a few minutes in some cases), then run the +installer (on Windows) or just copy over `/Applications/Bitcoin-Qt` (on Mac) +or `bitcoind`/`bitcoin-qt` (on Linux). + +Upgrading directly from a version of Bitcoin Core that has reached its EOL is +possible, but it might take some time if the data directory needs to be migrated. Old +wallet versions of Bitcoin Core are generally supported. + +Compatibility +============== + +Bitcoin Core is supported and extensively tested on operating systems +using the Linux kernel, macOS 10.15+, and Windows 7 and newer. Bitcoin +Core should also work on most other Unix-like systems but is not as +frequently tested on them. It is not recommended to use Bitcoin Core on +unsupported systems. + +Notable changes +=============== + +P2P and network changes +----------------------- + +Updated RPCs +------------ + + +Changes to wallet related RPCs can be found in the Wallet section below. + +New RPCs +-------- + +Build System +------------ + +Updated settings +---------------- + + +Changes to GUI or wallet related settings can be found in the GUI or Wallet section below. + +New settings +------------ + +Tools and Utilities +------------------- + +Wallet +------ + +GUI changes +----------- + +Low-level changes +================= + +RPC +--- + +Tests +----- + +*version* change log +==================== + +Credits +======= + +Thanks to everyone who directly contributed to this release: + + +As well as to everyone that helped with translations on +[Transifex](https://www.transifex.com/bitcoin/bitcoin/). diff --git a/doc/release-notes.md b/doc/release-notes.md index 6815408ca7..35f0713879 100644 --- a/doc/release-notes.md +++ b/doc/release-notes.md @@ -1,17 +1,7 @@ -*After branching off for a major version release of Bitcoin Core, use this -template to create the initial release notes draft.* - *The release notes draft is a temporary file that can be added to by anyone. See [/doc/developer-notes.md#release-notes](/doc/developer-notes.md#release-notes) for the process.* -*Create the draft, named* "*version* Release Notes Draft" -*(e.g. "22.0 Release Notes Draft"), as a collaborative wiki in:* - -https://github.com/bitcoin-core/bitcoin-devwiki/wiki/ - -*Before the final release, move the notes back to this git repository.* - *version* Release Notes Draft =============================== @@ -46,7 +36,7 @@ Compatibility ============== Bitcoin Core is supported and extensively tested on operating systems -using the Linux kernel, macOS 10.14+, and Windows 7 and newer. Bitcoin +using the Linux kernel, macOS 10.15+, and Windows 7 and newer. Bitcoin Core should also work on most other Unix-like systems but is not as frequently tested on them. It is not recommended to use Bitcoin Core on unsupported systems. @@ -57,43 +47,54 @@ Notable changes P2P and network changes ----------------------- -- A bitcoind node will no longer rumour addresses to inbound peers by default. - They will become eligible for address gossip after sending an ADDR, ADDRV2, - or GETADDR message. (#21528) - Updated RPCs ------------ +- The `-deprecatedrpc=softforks` configuration option has been removed. The + RPC `getblockchaininfo` no longer returns the `softforks` field, which was + previously deprecated in 23.0. (#23508) Information on soft fork status is + now only available via the `getdeploymentinfo` RPC. + +- The `deprecatedrpc=exclude_coinbase` configuration option has been removed. + The `receivedby` RPCs (`listreceivedbyaddress`, `listreceivedbylabel`, + `getreceivedbyaddress` and `getreceivedbylabel`) now always return results + accounting for received coins from coinbase outputs, without an option to + change that behaviour. Excluding coinbases was previously deprecated in 23.0. + (#25171) + +- The `deprecatedrpc=fees` configuration option has been removed. The top-level + fee fields `fee`, `modifiedfee`, `ancestorfees` and `descendantfees` are no + longer returned by RPCs `getmempoolentry`, `getrawmempool(verbose=true)`, + `getmempoolancestors(verbose=true)` and `getmempooldescendants(verbose=true)`. + The same fee fields can be accessed through the `fees` object in the result. + The top-level fee fields were previously deprecated in 23.0. (#25204) + +Changes to wallet related RPCs can be found in the Wallet section below. + New RPCs -------- Build System ------------ -Files ------ +Updated settings +---------------- + -* On startup, the list of banned hosts and networks (via `setban` RPC) in - `banlist.dat` is ignored and only `banlist.json` is considered. Bitcoin Core - version 22.x is the only version that can read `banlist.dat` and also write - it to `banlist.json`. If `banlist.json` already exists, version 22.x will not - try to translate the `banlist.dat` into json. After an upgrade, `listbanned` - can be used to double check the parsed entries. (#22570) +Changes to GUI or wallet related settings can be found in the GUI or Wallet section below. New settings ------------ -Updated settings ----------------- - Tools and Utilities ------------------- -- Update `-getinfo` to return data in a user-friendly format that also reduces vertical space. (#21832) - Wallet ------ +- RPC `getreceivedbylabel` now returns an error, "Label not found + in wallet" (-4), if the label is not in the address book. (#25122) + GUI changes ----------- @@ -103,16 +104,11 @@ Low-level changes RPC --- -- `getblockchaininfo` now returns a new `time` field, that provides the chain tip time. (#22407) - Tests ----- -- For the `regtest` network the activation heights of several softforks were - changed. - * BIP 34 (blockheight in coinbase) from 500 to 2 (#16333) - * BIP 66 (DERSIG) from 1251 to 102 (#22632) - * BIP 65 (CLTV) from 1351 to 111 (#21862) +*version* change log +==================== Credits ======= diff --git a/doc/release-notes/release-notes-22.0.md b/doc/release-notes/release-notes-22.0.md new file mode 100644 index 0000000000..972c91aa6f --- /dev/null +++ b/doc/release-notes/release-notes-22.0.md @@ -0,0 +1,1163 @@ +22.0 Release Notes +================== + +Bitcoin Core version 22.0 is now available from: + + <https://bitcoincore.org/bin/bitcoin-core-22.0/> + +This release includes new features, various bug fixes and performance +improvements, as well as updated translations. + +Please report bugs using the issue tracker at GitHub: + + <https://github.com/bitcoin/bitcoin/issues> + +To receive security and update notifications, please subscribe to: + + <https://bitcoincore.org/en/list/announcements/join/> + +How to Upgrade +============== + +If you are running an older version, shut it down. Wait until it has completely +shut down (which might take a few minutes in some cases), then run the +installer (on Windows) or just copy over `/Applications/Bitcoin-Qt` (on Mac) +or `bitcoind`/`bitcoin-qt` (on Linux). + +Upgrading directly from a version of Bitcoin Core that has reached its EOL is +possible, but it might take some time if the data directory needs to be migrated. Old +wallet versions of Bitcoin Core are generally supported. + +Compatibility +============== + +Bitcoin Core is supported and extensively tested on operating systems +using the Linux kernel, macOS 10.14+, and Windows 7 and newer. Bitcoin +Core should also work on most other Unix-like systems but is not as +frequently tested on them. It is not recommended to use Bitcoin Core on +unsupported systems. + +From Bitcoin Core 22.0 onwards, macOS versions earlier than 10.14 are no longer supported. + +Notable changes +=============== + +P2P and network changes +----------------------- +- Added support for running Bitcoin Core as an + [I2P (Invisible Internet Project)](https://en.wikipedia.org/wiki/I2P) service + and connect to such services. See [i2p.md](https://github.com/bitcoin/bitcoin/blob/22.x/doc/i2p.md) for details. (#20685) +- This release removes support for Tor version 2 hidden services in favor of Tor + v3 only, as the Tor network [dropped support for Tor + v2](https://blog.torproject.org/v2-deprecation-timeline) with the release of + Tor version 0.4.6. Henceforth, Bitcoin Core ignores Tor v2 addresses; it + neither rumors them over the network to other peers, nor stores them in memory + or to `peers.dat`. (#22050) + +- Added NAT-PMP port mapping support via + [`libnatpmp`](https://miniupnp.tuxfamily.org/libnatpmp.html). (#18077) + +New and Updated RPCs +-------------------- + +- Due to [BIP 350](https://github.com/bitcoin/bips/blob/master/bip-0350.mediawiki) + being implemented, behavior for all RPCs that accept addresses is changed when + a native witness version 1 (or higher) is passed. These now require a Bech32m + encoding instead of a Bech32 one, and Bech32m encoding will be used for such + addresses in RPC output as well. No version 1 addresses should be created + for mainnet until consensus rules are adopted that give them meaning + (as will happen through [BIP 341](https://github.com/bitcoin/bips/blob/master/bip-0341.mediawiki)). + Once that happens, Bech32m is expected to be used for them, so this shouldn't + affect any production systems, but may be observed on other networks where such + addresses already have meaning (like signet). (#20861) + +- The `getpeerinfo` RPC returns two new boolean fields, `bip152_hb_to` and + `bip152_hb_from`, that respectively indicate whether we selected a peer to be + in compact blocks high-bandwidth mode or whether a peer selected us as a + compact blocks high-bandwidth peer. High-bandwidth peers send new block + announcements via a `cmpctblock` message rather than the usual inv/headers + announcements. See BIP 152 for more details. (#19776) + +- `getpeerinfo` no longer returns the following fields: `addnode`, `banscore`, + and `whitelisted`, which were previously deprecated in 0.21. Instead of + `addnode`, the `connection_type` field returns manual. Instead of + `whitelisted`, the `permissions` field indicates if the peer has special + privileges. The `banscore` field has simply been removed. (#20755) + +- The following RPCs: `gettxout`, `getrawtransaction`, `decoderawtransaction`, + `decodescript`, `gettransaction`, and REST endpoints: `/rest/tx`, + `/rest/getutxos`, `/rest/block` deprecated the following fields (which are no + longer returned in the responses by default): `addresses`, `reqSigs`. + The `-deprecatedrpc=addresses` flag must be passed for these fields to be + included in the RPC response. This flag/option will be available only for this major release, after which + the deprecation will be removed entirely. Note that these fields are attributes of + the `scriptPubKey` object returned in the RPC response. However, in the response + of `decodescript` these fields are top-level attributes, and included again as attributes + of the `scriptPubKey` object. (#20286) + +- When creating a hex-encoded bitcoin transaction using the `bitcoin-tx` utility + with the `-json` option set, the following fields: `addresses`, `reqSigs` are no longer + returned in the tx output of the response. (#20286) + +- The `listbanned` RPC now returns two new numeric fields: `ban_duration` and `time_remaining`. + Respectively, these new fields indicate the duration of a ban and the time remaining until a ban expires, + both in seconds. Additionally, the `ban_created` field is repositioned to come before `banned_until`. (#21602) + +- The `setban` RPC can ban onion addresses again. This fixes a regression introduced in version 0.21.0. (#20852) + +- The `getnodeaddresses` RPC now returns a "network" field indicating the + network type (ipv4, ipv6, onion, or i2p) for each address. (#21594) + +- `getnodeaddresses` now also accepts a "network" argument (ipv4, ipv6, onion, + or i2p) to return only addresses of the specified network. (#21843) + +- The `testmempoolaccept` RPC now accepts multiple transactions (still experimental at the moment, + API may be unstable). This is intended for testing transaction packages with dependency + relationships; it is not recommended for batch-validating independent transactions. In addition to + mempool policy, package policies apply: the list cannot contain more than 25 transactions or have a + total size exceeding 101K virtual bytes, and cannot conflict with (spend the same inputs as) each other or + the mempool, even if it would be a valid BIP125 replace-by-fee. There are some known limitations to + the accuracy of the test accept: it's possible for `testmempoolaccept` to return "allowed"=True for a + group of transactions, but "too-long-mempool-chain" if they are actually submitted. (#20833) + +- `addmultisigaddress` and `createmultisig` now support up to 20 keys for + Segwit addresses. (#20867) + +Changes to Wallet or GUI related RPCs can be found in the GUI or Wallet section below. + +Build System +------------ + +- Release binaries are now produced using the new `guix`-based build system. + The [/doc/release-process.md](/doc/release-process.md) document has been updated accordingly. + +Files +----- + +- The list of banned hosts and networks (via `setban` RPC) is now saved on disk + in JSON format in `banlist.json` instead of `banlist.dat`. `banlist.dat` is + only read on startup if `banlist.json` is not present. Changes are only written to the new + `banlist.json`. A future version of Bitcoin Core may completely ignore + `banlist.dat`. (#20966) + +New settings +------------ + +- The `-natpmp` option has been added to use NAT-PMP to map the listening port. + If both UPnP and NAT-PMP are enabled, a successful allocation from UPnP + prevails over one from NAT-PMP. (#18077) + +Updated settings +---------------- + +Changes to Wallet or GUI related settings can be found in the GUI or Wallet section below. + +- Passing an invalid `-rpcauth` argument now cause bitcoind to fail to start. (#20461) + +Tools and Utilities +------------------- + +- A new CLI `-addrinfo` command returns the number of addresses known to the + node per network type (including Tor v2 versus v3) and total. This can be + useful to see if the node knows enough addresses in a network to use options + like `-onlynet=<network>` or to upgrade to this release of Bitcoin Core 22.0 + that supports Tor v3 only. (#21595) + +- A new `-rpcwaittimeout` argument to `bitcoin-cli` sets the timeout + in seconds to use with `-rpcwait`. If the timeout expires, + `bitcoin-cli` will report a failure. (#21056) + +Wallet +------ + +- External signers such as hardware wallets can now be used through the new RPC methods `enumeratesigners` and `displayaddress`. Support is also added to the `send` RPC call. This feature is experimental. See [external-signer.md](https://github.com/bitcoin/bitcoin/blob/22.x/doc/external-signer.md) for details. (#16546) + +- A new `listdescriptors` RPC is available to inspect the contents of descriptor-enabled wallets. + The RPC returns public versions of all imported descriptors, including their timestamp and flags. + For ranged descriptors, it also returns the range boundaries and the next index to generate addresses from. (#20226) + +- The `bumpfee` RPC is not available with wallets that have private keys + disabled. `psbtbumpfee` can be used instead. (#20891) + +- The `fundrawtransaction`, `send` and `walletcreatefundedpsbt` RPCs now support an `include_unsafe` option + that when `true` allows using unsafe inputs to fund the transaction. + Note that the resulting transaction may become invalid if one of the unsafe inputs disappears. + If that happens, the transaction must be funded with different inputs and republished. (#21359) + +- We now support up to 20 keys in `multi()` and `sortedmulti()` descriptors + under `wsh()`. (#20867) + +- Taproot descriptors can be imported into the wallet only after activation has occurred on the network (e.g. mainnet, testnet, signet) in use. See [descriptors.md](https://github.com/bitcoin/bitcoin/blob/22.x/doc/descriptors.md) for supported descriptors. + +GUI changes +----------- + +- External signers such as hardware wallets can now be used. These require an external tool such as [HWI](https://github.com/bitcoin-core/HWI) to be installed and configured under Options -> Wallet. When creating a new wallet a new option "External signer" will appear in the dialog. If the device is detected, its name is suggested as the wallet name. The watch-only keys are then automatically imported. Receive addresses can be verified on the device. The send dialog will automatically use the connected device. This feature is experimental and the UI may freeze for a few seconds when performing these actions. + +Low-level changes +================= + +RPC +--- + +- The RPC server can process a limited number of simultaneous RPC requests. + Previously, if this limit was exceeded, the RPC server would respond with + [status code 500 (`HTTP_INTERNAL_SERVER_ERROR`)](https://en.wikipedia.org/wiki/List_of_HTTP_status_codes#5xx_server_errors). + Now it returns status code 503 (`HTTP_SERVICE_UNAVAILABLE`). (#18335) + +- Error codes have been updated to be more accurate for the following error cases (#18466): + - `signmessage` now returns RPC_INVALID_ADDRESS_OR_KEY (-5) if the + passed address is invalid. Previously returned RPC_TYPE_ERROR (-3). + - `verifymessage` now returns RPC_INVALID_ADDRESS_OR_KEY (-5) if the + passed address is invalid. Previously returned RPC_TYPE_ERROR (-3). + - `verifymessage` now returns RPC_TYPE_ERROR (-3) if the passed signature + is malformed. Previously returned RPC_INVALID_ADDRESS_OR_KEY (-5). + +Tests +----- + +22.0 change log +=============== + +A detailed list of changes in this version follows. To keep the list to a manageable length, small refactors and typo fixes are not included, and similar changes are sometimes condensed into one line. + +### Consensus +- bitcoin/bitcoin#19438 Introduce deploymentstatus (ajtowns) +- bitcoin/bitcoin#20207 Follow-up extra comments on taproot code and tests (sipa) +- bitcoin/bitcoin#21330 Deal with missing data in signature hashes more consistently (sipa) + +### Policy +- bitcoin/bitcoin#18766 Disable fee estimation in blocksonly mode (by removing the fee estimates global) (darosior) +- bitcoin/bitcoin#20497 Add `MAX_STANDARD_SCRIPTSIG_SIZE` to policy (sanket1729) +- bitcoin/bitcoin#20611 Move `TX_MAX_STANDARD_VERSION` to policy (MarcoFalke) + +### Mining +- bitcoin/bitcoin#19937, bitcoin/bitcoin#20923 Signet mining utility (ajtowns) + +### Block and transaction handling +- bitcoin/bitcoin#14501 Fix possible data race when committing block files (luke-jr) +- bitcoin/bitcoin#15946 Allow maintaining the blockfilterindex when using prune (jonasschnelli) +- bitcoin/bitcoin#18710 Add local thread pool to CCheckQueue (hebasto) +- bitcoin/bitcoin#19521 Coinstats Index (fjahr) +- bitcoin/bitcoin#19806 UTXO snapshot activation (jamesob) +- bitcoin/bitcoin#19905 Remove dead CheckForkWarningConditionsOnNewFork (MarcoFalke) +- bitcoin/bitcoin#19935 Move SaltedHashers to separate file and add some new ones (achow101) +- bitcoin/bitcoin#20054 Remove confusing and useless "unexpected version" warning (MarcoFalke) +- bitcoin/bitcoin#20519 Handle rename failure in `DumpMempool(…)` by using the `RenameOver(…)` return value (practicalswift) +- bitcoin/bitcoin#20749, bitcoin/bitcoin#20750, bitcoin/bitcoin#21055, bitcoin/bitcoin#21270, bitcoin/bitcoin#21525, bitcoin/bitcoin#21391, bitcoin/bitcoin#21767, bitcoin/bitcoin#21866 Prune `g_chainman` usage (dongcarl) +- bitcoin/bitcoin#20833 rpc/validation: enable packages through testmempoolaccept (glozow) +- bitcoin/bitcoin#20834 Locks and docs in ATMP and CheckInputsFromMempoolAndCache (glozow) +- bitcoin/bitcoin#20854 Remove unnecessary try-block (amitiuttarwar) +- bitcoin/bitcoin#20868 Remove redundant check on pindex (jarolrod) +- bitcoin/bitcoin#20921 Don't try to invalidate genesis block in CChainState::InvalidateBlock (theStack) +- bitcoin/bitcoin#20972 Locks: Annotate CTxMemPool::check to require `cs_main` (dongcarl) +- bitcoin/bitcoin#21009 Remove RewindBlockIndex logic (dhruv) +- bitcoin/bitcoin#21025 Guard chainman chainstates with `cs_main` (dongcarl) +- bitcoin/bitcoin#21202 Two small clang lock annotation improvements (amitiuttarwar) +- bitcoin/bitcoin#21523 Run VerifyDB on all chainstates (jamesob) +- bitcoin/bitcoin#21573 Update libsecp256k1 subtree to latest master (sipa) +- bitcoin/bitcoin#21582, bitcoin/bitcoin#21584, bitcoin/bitcoin#21585 Fix assumeutxo crashes (MarcoFalke) +- bitcoin/bitcoin#21681 Fix ActivateSnapshot to use hardcoded nChainTx (jamesob) +- bitcoin/bitcoin#21796 index: Avoid async shutdown on init error (MarcoFalke) +- bitcoin/bitcoin#21946 Document and test lack of inherited signaling in RBF policy (ariard) +- bitcoin/bitcoin#22084 Package testmempoolaccept followups (glozow) +- bitcoin/bitcoin#22102 Remove `Warning:` from warning message printed for unknown new rules (prayank23) +- bitcoin/bitcoin#22112 Force port 0 in I2P (vasild) +- bitcoin/bitcoin#22135 CRegTestParams: Use `args` instead of `gArgs` (kiminuo) +- bitcoin/bitcoin#22146 Reject invalid coin height and output index when loading assumeutxo (MarcoFalke) +- bitcoin/bitcoin#22253 Distinguish between same tx and same-nonwitness-data tx in mempool (glozow) +- bitcoin/bitcoin#22261 Two small fixes to node broadcast logic (jnewbery) +- bitcoin/bitcoin#22415 Make `m_mempool` optional in CChainState (jamesob) +- bitcoin/bitcoin#22499 Update assumed chain params (sriramdvt) +- bitcoin/bitcoin#22589 net, doc: update I2P hardcoded seeds and docs for 22.0 (jonatack) + +### P2P protocol and network code +- bitcoin/bitcoin#18077 Add NAT-PMP port forwarding support (hebasto) +- bitcoin/bitcoin#18722 addrman: improve performance by using more suitable containers (vasild) +- bitcoin/bitcoin#18819 Replace `cs_feeFilter` with simple std::atomic (MarcoFalke) +- bitcoin/bitcoin#19203 Add regression fuzz harness for CVE-2017-18350. Add FuzzedSocket (practicalswift) +- bitcoin/bitcoin#19288 fuzz: Add fuzzing harness for TorController (practicalswift) +- bitcoin/bitcoin#19415 Make DNS lookup mockable, add fuzzing harness (practicalswift) +- bitcoin/bitcoin#19509 Per-Peer Message Capture (troygiorshev) +- bitcoin/bitcoin#19763 Don't try to relay to the address' originator (vasild) +- bitcoin/bitcoin#19771 Replace enum CConnMan::NumConnections with enum class ConnectionDirection (luke-jr) +- bitcoin/bitcoin#19776 net, rpc: expose high bandwidth mode state via getpeerinfo (theStack) +- bitcoin/bitcoin#19832 Put disconnecting logs into BCLog::NET category (hebasto) +- bitcoin/bitcoin#19858 Periodically make block-relay connections and sync headers (sdaftuar) +- bitcoin/bitcoin#19884 No delay in adding fixed seeds if -dnsseed=0 and peers.dat is empty (dhruv) +- bitcoin/bitcoin#20079 Treat handshake misbehavior like unknown message (MarcoFalke) +- bitcoin/bitcoin#20138 Assume that SetCommonVersion is called at most once per peer (MarcoFalke) +- bitcoin/bitcoin#20162 p2p: declare Announcement::m_state as uint8_t, add getter/setter (jonatack) +- bitcoin/bitcoin#20197 Protect onions in AttemptToEvictConnection(), add eviction protection test coverage (jonatack) +- bitcoin/bitcoin#20210 assert `CNode::m_inbound_onion` is inbound in ctor, add getter, unit tests (jonatack) +- bitcoin/bitcoin#20228 addrman: Make addrman a top-level component (jnewbery) +- bitcoin/bitcoin#20234 Don't bind on 0.0.0.0 if binds are restricted to Tor (vasild) +- bitcoin/bitcoin#20477 Add unit testing of node eviction logic (practicalswift) +- bitcoin/bitcoin#20516 Well-defined CAddress disk serialization, and addrv2 anchors.dat (sipa) +- bitcoin/bitcoin#20557 addrman: Fix new table bucketing during unserialization (jnewbery) +- bitcoin/bitcoin#20561 Periodically clear `m_addr_known` (sdaftuar) +- bitcoin/bitcoin#20599 net processing: Tolerate sendheaders and sendcmpct messages before verack (jnewbery) +- bitcoin/bitcoin#20616 Check CJDNS address is valid (lontivero) +- bitcoin/bitcoin#20617 Remove `m_is_manual_connection` from CNodeState (ariard) +- bitcoin/bitcoin#20624 net processing: Remove nStartingHeight check from block relay (jnewbery) +- bitcoin/bitcoin#20651 Make p2p recv buffer timeout 20 minutes for all peers (jnewbery) +- bitcoin/bitcoin#20661 Only select from addrv2-capable peers for torv3 address relay (sipa) +- bitcoin/bitcoin#20685 Add I2P support using I2P SAM (vasild) +- bitcoin/bitcoin#20690 Clean up logging of outbound connection type (sdaftuar) +- bitcoin/bitcoin#20721 Move ping data to `net_processing` (jnewbery) +- bitcoin/bitcoin#20724 Cleanup of -debug=net log messages (ajtowns) +- bitcoin/bitcoin#20747 net processing: Remove dropmessagestest (jnewbery) +- bitcoin/bitcoin#20764 cli -netinfo peer connections dashboard updates 🎄 ✨ (jonatack) +- bitcoin/bitcoin#20788 add RAII socket and use it instead of bare SOCKET (vasild) +- bitcoin/bitcoin#20791 remove unused legacyWhitelisted in AcceptConnection() (jonatack) +- bitcoin/bitcoin#20816 Move RecordBytesSent() call out of `cs_vSend` lock (jnewbery) +- bitcoin/bitcoin#20845 Log to net debug in MaybeDiscourageAndDisconnect except for noban and manual peers (MarcoFalke) +- bitcoin/bitcoin#20864 Move SocketSendData lock annotation to header (MarcoFalke) +- bitcoin/bitcoin#20965 net, rpc: return `NET_UNROUTABLE` as `not_publicly_routable`, automate helps (jonatack) +- bitcoin/bitcoin#20966 banman: save the banlist in a JSON format on disk (vasild) +- bitcoin/bitcoin#21015 Make all of `net_processing` (and some of net) use std::chrono types (dhruv) +- bitcoin/bitcoin#21029 bitcoin-cli: Correct docs (no "generatenewaddress" exists) (luke-jr) +- bitcoin/bitcoin#21148 Split orphan handling from `net_processing` into txorphanage (ajtowns) +- bitcoin/bitcoin#21162 Net Processing: Move RelayTransaction() into PeerManager (jnewbery) +- bitcoin/bitcoin#21167 make `CNode::m_inbound_onion` public, initialize explicitly (jonatack) +- bitcoin/bitcoin#21186 net/net processing: Move addr data into `net_processing` (jnewbery) +- bitcoin/bitcoin#21187 Net processing: Only call PushAddress() from `net_processing` (jnewbery) +- bitcoin/bitcoin#21198 Address outstanding review comments from PR20721 (jnewbery) +- bitcoin/bitcoin#21222 log: Clarify log message when file does not exist (MarcoFalke) +- bitcoin/bitcoin#21235 Clarify disconnect log message in ProcessGetBlockData, remove send bool (MarcoFalke) +- bitcoin/bitcoin#21236 Net processing: Extract `addr` send functionality into MaybeSendAddr() (jnewbery) +- bitcoin/bitcoin#21261 update inbound eviction protection for multiple networks, add I2P peers (jonatack) +- bitcoin/bitcoin#21328 net, refactor: pass uint16 CService::port as uint16 (jonatack) +- bitcoin/bitcoin#21387 Refactor sock to add I2P fuzz and unit tests (vasild) +- bitcoin/bitcoin#21395 Net processing: Remove unused CNodeState.address member (jnewbery) +- bitcoin/bitcoin#21407 i2p: limit the size of incoming messages (vasild) +- bitcoin/bitcoin#21506 p2p, refactor: make NetPermissionFlags an enum class (jonatack) +- bitcoin/bitcoin#21509 Don't send FEEFILTER in blocksonly mode (mzumsande) +- bitcoin/bitcoin#21560 Add Tor v3 hardcoded seeds (laanwj) +- bitcoin/bitcoin#21563 Restrict period when `cs_vNodes` mutex is locked (hebasto) +- bitcoin/bitcoin#21564 Avoid calling getnameinfo when formatting IPv4 addresses in CNetAddr::ToStringIP (practicalswift) +- bitcoin/bitcoin#21631 i2p: always check the return value of Sock::Wait() (vasild) +- bitcoin/bitcoin#21644 p2p, bugfix: use NetPermissions::HasFlag() in CConnman::Bind() (jonatack) +- bitcoin/bitcoin#21659 flag relevant Sock methods with [[nodiscard]] (vasild) +- bitcoin/bitcoin#21750 remove unnecessary check of `CNode::cs_vSend` (vasild) +- bitcoin/bitcoin#21756 Avoid calling `getnameinfo` when formatting IPv6 addresses in `CNetAddr::ToStringIP` (practicalswift) +- bitcoin/bitcoin#21775 Limit `m_block_inv_mutex` (MarcoFalke) +- bitcoin/bitcoin#21825 Add I2P hardcoded seeds (jonatack) +- bitcoin/bitcoin#21843 p2p, rpc: enable GetAddr, GetAddresses, and getnodeaddresses by network (jonatack) +- bitcoin/bitcoin#21845 net processing: Don't require locking `cs_main` before calling RelayTransactions() (jnewbery) +- bitcoin/bitcoin#21872 Sanitize message type for logging (laanwj) +- bitcoin/bitcoin#21914 Use stronger AddLocal() for our I2P address (vasild) +- bitcoin/bitcoin#21985 Return IPv6 scope id in `CNetAddr::ToStringIP()` (laanwj) +- bitcoin/bitcoin#21992 Remove -feefilter option (amadeuszpawlik) +- bitcoin/bitcoin#21996 Pass strings to NetPermissions::TryParse functions by const ref (jonatack) +- bitcoin/bitcoin#22013 ignore block-relay-only peers when skipping DNS seed (ajtowns) +- bitcoin/bitcoin#22050 Remove tor v2 support (jonatack) +- bitcoin/bitcoin#22096 AddrFetch - don't disconnect on self-announcements (mzumsande) +- bitcoin/bitcoin#22141 net processing: Remove hash and fValidatedHeaders from QueuedBlock (jnewbery) +- bitcoin/bitcoin#22144 Randomize message processing peer order (sipa) +- bitcoin/bitcoin#22147 Protect last outbound HB compact block peer (sdaftuar) +- bitcoin/bitcoin#22179 Torv2 removal followups (vasild) +- bitcoin/bitcoin#22211 Relay I2P addresses even if not reachable (by us) (vasild) +- bitcoin/bitcoin#22284 Performance improvements to ProtectEvictionCandidatesByRatio() (jonatack) +- bitcoin/bitcoin#22387 Rate limit the processing of rumoured addresses (sipa) +- bitcoin/bitcoin#22455 addrman: detect on-disk corrupted nNew and nTried during unserialization (vasild) + +### Wallet +- bitcoin/bitcoin#15710 Catch `ios_base::failure` specifically (Bushstar) +- bitcoin/bitcoin#16546 External signer support - Wallet Box edition (Sjors) +- bitcoin/bitcoin#17331 Use effective values throughout coin selection (achow101) +- bitcoin/bitcoin#18418 Increase `OUTPUT_GROUP_MAX_ENTRIES` to 100 (fjahr) +- bitcoin/bitcoin#18842 Mark replaced tx to not be in the mempool anymore (MarcoFalke) +- bitcoin/bitcoin#19136 Add `parent_desc` to `getaddressinfo` (achow101) +- bitcoin/bitcoin#19137 wallettool: Add dump and createfromdump commands (achow101) +- bitcoin/bitcoin#19651 `importdescriptor`s update existing (S3RK) +- bitcoin/bitcoin#20040 Refactor OutputGroups to handle fees and spending eligibility on grouping (achow101) +- bitcoin/bitcoin#20202 Make BDB support optional (achow101) +- bitcoin/bitcoin#20226, bitcoin/bitcoin#21277, - bitcoin/bitcoin#21063 Add `listdescriptors` command (S3RK) +- bitcoin/bitcoin#20267 Disable and fix tests for when BDB is not compiled (achow101) +- bitcoin/bitcoin#20275 List all wallets in non-SQLite and non-BDB builds (ryanofsky) +- bitcoin/bitcoin#20365 wallettool: Add parameter to create descriptors wallet (S3RK) +- bitcoin/bitcoin#20403 `upgradewallet` fixes, improvements, test coverage (jonatack) +- bitcoin/bitcoin#20448 `unloadwallet`: Allow specifying `wallet_name` param matching RPC endpoint wallet (luke-jr) +- bitcoin/bitcoin#20536 Error with "Transaction too large" if the funded tx will end up being too large after signing (achow101) +- bitcoin/bitcoin#20687 Add missing check for -descriptors wallet tool option (MarcoFalke) +- bitcoin/bitcoin#20952 Add BerkeleyDB version sanity check at init time (laanwj) +- bitcoin/bitcoin#21127 Load flags before everything else (Sjors) +- bitcoin/bitcoin#21141 Add new format string placeholders for walletnotify (maayank) +- bitcoin/bitcoin#21238 A few descriptor improvements to prepare for Taproot support (sipa) +- bitcoin/bitcoin#21302 `createwallet` examples for descriptor wallets (S3RK) +- bitcoin/bitcoin#21329 descriptor wallet: Cache last hardened xpub and use in normalized descriptors (achow101) +- bitcoin/bitcoin#21365 Basic Taproot signing support for descriptor wallets (sipa) +- bitcoin/bitcoin#21417 Misc external signer improvement and HWI 2 support (Sjors) +- bitcoin/bitcoin#21467 Move external signer out of wallet module (Sjors) +- bitcoin/bitcoin#21572 Fix wrong wallet RPC context set after #21366 (ryanofsky) +- bitcoin/bitcoin#21574 Drop JSONRPCRequest constructors after #21366 (ryanofsky) +- bitcoin/bitcoin#21666 Miscellaneous external signer changes (fanquake) +- bitcoin/bitcoin#21759 Document coin selection code (glozow) +- bitcoin/bitcoin#21786 Ensure sat/vB feerates are in range (mantissa of 3) (jonatack) +- bitcoin/bitcoin#21944 Fix issues when `walletdir` is root directory (prayank23) +- bitcoin/bitcoin#22042 Replace size/weight estimate tuple with struct for named fields (instagibbs) +- bitcoin/bitcoin#22051 Basic Taproot derivation support for descriptors (sipa) +- bitcoin/bitcoin#22154 Add OutputType::BECH32M and related wallet support for fetching bech32m addresses (achow101) +- bitcoin/bitcoin#22156 Allow tr() import only when Taproot is active (achow101) +- bitcoin/bitcoin#22166 Add support for inferring tr() descriptors (sipa) +- bitcoin/bitcoin#22173 Do not load external signers wallets when unsupported (achow101) +- bitcoin/bitcoin#22308 Add missing BlockUntilSyncedToCurrentChain (MarcoFalke) +- bitcoin/bitcoin#22334 Do not spam about non-existent spk managers (S3RK) +- bitcoin/bitcoin#22379 Erase spkmans rather than setting to nullptr (achow101) +- bitcoin/bitcoin#22421 Make IsSegWitOutput return true for taproot outputs (sipa) +- bitcoin/bitcoin#22461 Change ScriptPubKeyMan::Upgrade default to True (achow101) +- bitcoin/bitcoin#22492 Reorder locks in dumpwallet to avoid lock order assertion (achow101) +- bitcoin/bitcoin#22686 Use GetSelectionAmount in ApproximateBestSubset (achow101) + +### RPC and other APIs +- bitcoin/bitcoin#18335, bitcoin/bitcoin#21484 cli: Print useful error if bitcoind rpc work queue exceeded (LarryRuane) +- bitcoin/bitcoin#18466 Fix invalid parameter error codes for `{sign,verify}message` RPCs (theStack) +- bitcoin/bitcoin#18772 Calculate fees in `getblock` using BlockUndo data (robot-visions) +- bitcoin/bitcoin#19033 http: Release work queue after event base finish (promag) +- bitcoin/bitcoin#19055 Add MuHash3072 implementation (fjahr) +- bitcoin/bitcoin#19145 Add `hash_type` MUHASH for gettxoutsetinfo (fjahr) +- bitcoin/bitcoin#19847 Avoid duplicate set lookup in `gettxoutproof` (promag) +- bitcoin/bitcoin#20286 Deprecate `addresses` and `reqSigs` from RPC outputs (mjdietzx) +- bitcoin/bitcoin#20459 Fail to return undocumented return values (MarcoFalke) +- bitcoin/bitcoin#20461 Validate `-rpcauth` arguments (promag) +- bitcoin/bitcoin#20556 Properly document return values (`submitblock`, `gettxout`, `getblocktemplate`, `scantxoutset`) (MarcoFalke) +- bitcoin/bitcoin#20755 Remove deprecated fields from `getpeerinfo` (amitiuttarwar) +- bitcoin/bitcoin#20832 Better error messages for invalid addresses (eilx2) +- bitcoin/bitcoin#20867 Support up to 20 keys for multisig under Segwit context (darosior) +- bitcoin/bitcoin#20877 cli: `-netinfo` user help and argument parsing improvements (jonatack) +- bitcoin/bitcoin#20891 Remove deprecated bumpfee behavior (achow101) +- bitcoin/bitcoin#20916 Return wtxid from `testmempoolaccept` (MarcoFalke) +- bitcoin/bitcoin#20917 Add missing signet mentions in network name lists (theStack) +- bitcoin/bitcoin#20941 Document `RPC_TRANSACTION_ALREADY_IN_CHAIN` exception (jarolrod) +- bitcoin/bitcoin#20944 Return total fee in `getmempoolinfo` (MarcoFalke) +- bitcoin/bitcoin#20964 Add specific error code for "wallet already loaded" (laanwj) +- bitcoin/bitcoin#21053 Document {previous,next}blockhash as optional (theStack) +- bitcoin/bitcoin#21056 Add a `-rpcwaittimeout` parameter to limit time spent waiting (cdecker) +- bitcoin/bitcoin#21192 cli: Treat high detail levels as maximum in `-netinfo` (laanwj) +- bitcoin/bitcoin#21311 Document optional fields for `getchaintxstats` result (theStack) +- bitcoin/bitcoin#21359 `include_unsafe` option for fundrawtransaction (t-bast) +- bitcoin/bitcoin#21426 Remove `scantxoutset` EXPERIMENTAL warning (jonatack) +- bitcoin/bitcoin#21544 Missing doc updates for bumpfee psbt update (MarcoFalke) +- bitcoin/bitcoin#21594 Add `network` field to `getnodeaddresses` (jonatack) +- bitcoin/bitcoin#21595, bitcoin/bitcoin#21753 cli: Create `-addrinfo` (jonatack) +- bitcoin/bitcoin#21602 Add additional ban time fields to `listbanned` (jarolrod) +- bitcoin/bitcoin#21679 Keep default argument value in correct type (promag) +- bitcoin/bitcoin#21718 Improve error message for `getblock` invalid datatype (klementtan) +- bitcoin/bitcoin#21913 RPCHelpMan fixes (kallewoof) +- bitcoin/bitcoin#22021 `bumpfee`/`psbtbumpfee` fixes and updates (jonatack) +- bitcoin/bitcoin#22043 `addpeeraddress` test coverage, code simplify/constness (jonatack) +- bitcoin/bitcoin#22327 cli: Avoid truncating `-rpcwaittimeout` (MarcoFalke) + +### GUI +- bitcoin/bitcoin#18948 Call setParent() in the parent's context (hebasto) +- bitcoin/bitcoin#20482 Add depends qt fix for ARM macs (jonasschnelli) +- bitcoin/bitcoin#21836 scripted-diff: Replace three dots with ellipsis in the ui strings (hebasto) +- bitcoin/bitcoin#21935 Enable external signer support for GUI builds (Sjors) +- bitcoin/bitcoin#22133 Make QWindowsVistaStylePlugin available again (regression) (hebasto) +- bitcoin-core/gui#4 UI external signer support (e.g. hardware wallet) (Sjors) +- bitcoin-core/gui#13 Hide peer detail view if multiple are selected (promag) +- bitcoin-core/gui#18 Add peertablesortproxy module (hebasto) +- bitcoin-core/gui#21 Improve pruning tooltip (fluffypony, BitcoinErrorLog) +- bitcoin-core/gui#72 Log static plugins meta data and used style (hebasto) +- bitcoin-core/gui#79 Embed monospaced font (hebasto) +- bitcoin-core/gui#85 Remove unused "What's This" button in dialogs on Windows OS (hebasto) +- bitcoin-core/gui#115 Replace "Hide tray icon" option with positive "Show tray icon" one (hebasto) +- bitcoin-core/gui#118 Remove BDB version from the Information tab (hebasto) +- bitcoin-core/gui#121 Early subscribe core signals in transaction table model (promag) +- bitcoin-core/gui#123 Do not accept command while executing another one (hebasto) +- bitcoin-core/gui#125 Enable changing the autoprune block space size in intro dialog (luke-jr) +- bitcoin-core/gui#138 Unlock encrypted wallet "OK" button bugfix (mjdietzx) +- bitcoin-core/gui#139 doc: Improve gui/src/qt README.md (jarolrod) +- bitcoin-core/gui#154 Support macOS Dark mode (goums, Uplab) +- bitcoin-core/gui#162 Add network to peers window and peer details (jonatack) +- bitcoin-core/gui#163, bitcoin-core/gui#180 Peer details: replace Direction with Connection Type (jonatack) +- bitcoin-core/gui#164 Handle peer addition/removal in a right way (hebasto) +- bitcoin-core/gui#165 Save QSplitter state in QSettings (hebasto) +- bitcoin-core/gui#173 Follow Qt docs when implementing rowCount and columnCount (hebasto) +- bitcoin-core/gui#179 Add Type column to peers window, update peer details name/tooltip (jonatack) +- bitcoin-core/gui#186 Add information to "Confirm fee bump" window (prayank23) +- bitcoin-core/gui#189 Drop workaround for QTBUG-42503 which was fixed in Qt 5.5.0 (prusnak) +- bitcoin-core/gui#194 Save/restore RPCConsole geometry only for window (hebasto) +- bitcoin-core/gui#202 Fix right panel toggle in peers tab (RandyMcMillan) +- bitcoin-core/gui#203 Display plain "Inbound" in peer details (jonatack) +- bitcoin-core/gui#204 Drop buggy TableViewLastColumnResizingFixer class (hebasto) +- bitcoin-core/gui#205, bitcoin-core/gui#229 Save/restore TransactionView and recentRequestsView tables column sizes (hebasto) +- bitcoin-core/gui#206 Display fRelayTxes and `bip152_highbandwidth_{to, from}` in peer details (jonatack) +- bitcoin-core/gui#213 Add Copy Address Action to Payment Requests (jarolrod) +- bitcoin-core/gui#214 Disable requests context menu actions when appropriate (jarolrod) +- bitcoin-core/gui#217 Make warning label look clickable (jarolrod) +- bitcoin-core/gui#219 Prevent the main window popup menu (hebasto) +- bitcoin-core/gui#220 Do not translate file extensions (hebasto) +- bitcoin-core/gui#221 RPCConsole translatable string fixes and improvements (jonatack) +- bitcoin-core/gui#226 Add "Last Block" and "Last Tx" rows to peer details area (jonatack) +- bitcoin-core/gui#233 qt test: Don't bind to regtest port (achow101) +- bitcoin-core/gui#243 Fix issue when disabling the auto-enabled blank wallet checkbox (jarolrod) +- bitcoin-core/gui#246 Revert "qt: Use "fusion" style on macOS Big Sur with old Qt" (hebasto) +- bitcoin-core/gui#248 For values of "Bytes transferred" and "Bytes/s" with 1000-based prefix names use 1000-based divisor instead of 1024-based (wodry) +- bitcoin-core/gui#251 Improve URI/file handling message (hebasto) +- bitcoin-core/gui#256 Save/restore column sizes of the tables in the Peers tab (hebasto) +- bitcoin-core/gui#260 Handle exceptions isntead of crash (hebasto) +- bitcoin-core/gui#263 Revamp context menus (hebasto) +- bitcoin-core/gui#271 Don't clear console prompt when font resizing (jarolrod) +- bitcoin-core/gui#275 Support runtime appearance adjustment on macOS (hebasto) +- bitcoin-core/gui#276 Elide long strings in their middle in the Peers tab (hebasto) +- bitcoin-core/gui#281 Set shortcuts for console's resize buttons (jarolrod) +- bitcoin-core/gui#293 Enable wordWrap for Services (RandyMcMillan) +- bitcoin-core/gui#296 Do not use QObject::tr plural syntax for numbers with a unit symbol (hebasto) +- bitcoin-core/gui#297 Avoid unnecessary translations (hebasto) +- bitcoin-core/gui#298 Peertableview alternating row colors (RandyMcMillan) +- bitcoin-core/gui#300 Remove progress bar on modal overlay (brunoerg) +- bitcoin-core/gui#309 Add access to the Peers tab from the network icon (hebasto) +- bitcoin-core/gui#311 Peers Window rename 'Peer id' to 'Peer' (jarolrod) +- bitcoin-core/gui#313 Optimize string concatenation by default (hebasto) +- bitcoin-core/gui#325 Align numbers in the "Peer Id" column to the right (hebasto) +- bitcoin-core/gui#329 Make console buttons look clickable (jarolrod) +- bitcoin-core/gui#330 Allow prompt icon to be colorized (jarolrod) +- bitcoin-core/gui#331 Make RPC console welcome message translation-friendly (hebasto) +- bitcoin-core/gui#332 Replace disambiguation strings with translator comments (hebasto) +- bitcoin-core/gui#335 test: Use QSignalSpy instead of QEventLoop (jarolrod) +- bitcoin-core/gui#343 Improve the GUI responsiveness when progress dialogs are used (hebasto) +- bitcoin-core/gui#361 Fix GUI segfault caused by bitcoin/bitcoin#22216 (ryanofsky) +- bitcoin-core/gui#362 Add keyboard shortcuts to context menus (luke-jr) +- bitcoin-core/gui#366 Dark Mode fixes/portability (luke-jr) +- bitcoin-core/gui#375 Emit dataChanged signal to dynamically re-sort Peers table (hebasto) +- bitcoin-core/gui#393 Fix regression in "Encrypt Wallet" menu item (hebasto) +- bitcoin-core/gui#396 Ensure external signer option remains disabled without signers (achow101) +- bitcoin-core/gui#406 Handle new added plurals in `bitcoin_en.ts` (hebasto) + +### Build system +- bitcoin/bitcoin#17227 Add Android packaging support (icota) +- bitcoin/bitcoin#17920 guix: Build support for macOS (dongcarl) +- bitcoin/bitcoin#18298 Fix Qt processing of configure script for depends with DEBUG=1 (hebasto) +- bitcoin/bitcoin#19160 multiprocess: Add basic spawn and IPC support (ryanofsky) +- bitcoin/bitcoin#19504 Bump minimum python version to 3.6 (ajtowns) +- bitcoin/bitcoin#19522 fix building libconsensus with reduced exports for Darwin targets (fanquake) +- bitcoin/bitcoin#19683 Pin clang search paths for darwin host (dongcarl) +- bitcoin/bitcoin#19764 Split boost into build/host packages + bump + cleanup (dongcarl) +- bitcoin/bitcoin#19817 libtapi 1100.0.11 (fanquake) +- bitcoin/bitcoin#19846 enable unused member function diagnostic (Zero-1729) +- bitcoin/bitcoin#19867 Document and cleanup Qt hacks (fanquake) +- bitcoin/bitcoin#20046 Set `CMAKE_INSTALL_RPATH` for native packages (ryanofsky) +- bitcoin/bitcoin#20223 Drop the leading 0 from the version number (achow101) +- bitcoin/bitcoin#20333 Remove `native_biplist` dependency (fanquake) +- bitcoin/bitcoin#20353 configure: Support -fdebug-prefix-map and -fmacro-prefix-map (ajtowns) +- bitcoin/bitcoin#20359 Various config.site.in improvements and linting (dongcarl) +- bitcoin/bitcoin#20413 Require C++17 compiler (MarcoFalke) +- bitcoin/bitcoin#20419 Set minimum supported macOS to 10.14 (fanquake) +- bitcoin/bitcoin#20421 miniupnpc 2.2.2 (fanquake) +- bitcoin/bitcoin#20422 Mac deployment unification (fanquake) +- bitcoin/bitcoin#20424 Update univalue subtree (MarcoFalke) +- bitcoin/bitcoin#20449 Fix Windows installer build (achow101) +- bitcoin/bitcoin#20468 Warn when generating man pages for binaries built from a dirty branch (tylerchambers) +- bitcoin/bitcoin#20469 Avoid secp256k1.h include from system (dergoegge) +- bitcoin/bitcoin#20470 Replace genisoimage with xorriso (dongcarl) +- bitcoin/bitcoin#20471 Use C++17 in depends (fanquake) +- bitcoin/bitcoin#20496 Drop unneeded macOS framework dependencies (hebasto) +- bitcoin/bitcoin#20520 Do not force Precompiled Headers (PCH) for building Qt on Linux (hebasto) +- bitcoin/bitcoin#20549 Support make src/bitcoin-node and src/bitcoin-gui (promag) +- bitcoin/bitcoin#20565 Ensure PIC build for bdb on Android (BlockMechanic) +- bitcoin/bitcoin#20594 Fix getauxval calls in randomenv.cpp (jonasschnelli) +- bitcoin/bitcoin#20603 Update crc32c subtree (MarcoFalke) +- bitcoin/bitcoin#20609 configure: output notice that test binary is disabled by fuzzing (apoelstra) +- bitcoin/bitcoin#20619 guix: Quality of life improvements (dongcarl) +- bitcoin/bitcoin#20629 Improve id string robustness (dongcarl) +- bitcoin/bitcoin#20641 Use Qt top-level build facilities (hebasto) +- bitcoin/bitcoin#20650 Drop workaround for a fixed bug in Qt build system (hebasto) +- bitcoin/bitcoin#20673 Use more legible qmake commands in qt package (hebasto) +- bitcoin/bitcoin#20684 Define .INTERMEDIATE target once only (hebasto) +- bitcoin/bitcoin#20720 more robustly check for fcf-protection support (fanquake) +- bitcoin/bitcoin#20734 Make platform-specific targets available for proper platform builds only (hebasto) +- bitcoin/bitcoin#20936 build fuzz tests by default (danben) +- bitcoin/bitcoin#20937 guix: Make nsis reproducible by respecting SOURCE-DATE-EPOCH (dongcarl) +- bitcoin/bitcoin#20938 fix linking against -latomic when building for riscv (fanquake) +- bitcoin/bitcoin#20939 fix `RELOC_SECTION` security check for bitcoin-util (fanquake) +- bitcoin/bitcoin#20963 gitian-linux: Build binaries for 64-bit POWER (continued) (laanwj) +- bitcoin/bitcoin#21036 gitian: Bump descriptors to focal for 22.0 (fanquake) +- bitcoin/bitcoin#21045 Adds switch to enable/disable randomized base address in MSVC builds (EthanHeilman) +- bitcoin/bitcoin#21065 make macOS HOST in download-osx generic (fanquake) +- bitcoin/bitcoin#21078 guix: only download sources for hosts being built (fanquake) +- bitcoin/bitcoin#21116 Disable --disable-fuzz-binary for gitian/guix builds (hebasto) +- bitcoin/bitcoin#21182 remove mostly pointless `BOOST_PROCESS` macro (fanquake) +- bitcoin/bitcoin#21205 actually fail when Boost is missing (fanquake) +- bitcoin/bitcoin#21209 use newer source for libnatpmp (fanquake) +- bitcoin/bitcoin#21226 Fix fuzz binary compilation under windows (danben) +- bitcoin/bitcoin#21231 Add /opt/homebrew to path to look for boost libraries (fyquah) +- bitcoin/bitcoin#21239 guix: Add codesignature attachment support for osx+win (dongcarl) +- bitcoin/bitcoin#21250 Make `HAVE_O_CLOEXEC` available outside LevelDB (bugfix) (theStack) +- bitcoin/bitcoin#21272 guix: Passthrough `SDK_PATH` into container (dongcarl) +- bitcoin/bitcoin#21274 assumptions: Assume C++17 (fanquake) +- bitcoin/bitcoin#21286 Bump minimum Qt version to 5.9.5 (hebasto) +- bitcoin/bitcoin#21298 guix: Bump time-machine, glibc, and linux-headers (dongcarl) +- bitcoin/bitcoin#21304 guix: Add guix-clean script + establish gc-root for container profiles (dongcarl) +- bitcoin/bitcoin#21320 fix libnatpmp macos cross compile (fanquake) +- bitcoin/bitcoin#21321 guix: Add curl to required tool list (hebasto) +- bitcoin/bitcoin#21333 set Unicode true for NSIS installer (fanquake) +- bitcoin/bitcoin#21339 Make `AM_CONDITIONAL([ENABLE_EXTERNAL_SIGNER])` unconditional (hebasto) +- bitcoin/bitcoin#21349 Fix fuzz-cuckoocache cross-compiling with DEBUG=1 (hebasto) +- bitcoin/bitcoin#21354 build, doc: Drop no longer required packages from macOS cross-compiling dependencies (hebasto) +- bitcoin/bitcoin#21363 build, qt: Improve Qt static plugins/libs check code (hebasto) +- bitcoin/bitcoin#21375 guix: Misc feedback-based fixes + hier restructuring (dongcarl) +- bitcoin/bitcoin#21376 Qt 5.12.10 (fanquake) +- bitcoin/bitcoin#21382 Clean remnants of QTBUG-34748 fix (hebasto) +- bitcoin/bitcoin#21400 Fix regression introduced in #21363 (hebasto) +- bitcoin/bitcoin#21403 set --build when configuring packages in depends (fanquake) +- bitcoin/bitcoin#21421 don't try and use -fstack-clash-protection on Windows (fanquake) +- bitcoin/bitcoin#21423 Cleanups and follow ups after bumping Qt to 5.12.10 (hebasto) +- bitcoin/bitcoin#21427 Fix `id_string` invocations (dongcarl) +- bitcoin/bitcoin#21430 Add -Werror=implicit-fallthrough compile flag (hebasto) +- bitcoin/bitcoin#21457 Split libtapi and clang out of `native_cctools` (fanquake) +- bitcoin/bitcoin#21462 guix: Add guix-{attest,verify} scripts (dongcarl) +- bitcoin/bitcoin#21495 build, qt: Fix static builds on macOS Big Sur (hebasto) +- bitcoin/bitcoin#21497 Do not opt-in unused CoreWLAN stuff in depends for macOS (hebasto) +- bitcoin/bitcoin#21543 Enable safe warnings for msvc builds (hebasto) +- bitcoin/bitcoin#21565 Make `bitcoin_qt.m4` more generic (fanquake) +- bitcoin/bitcoin#21610 remove -Wdeprecated-register from NOWARN flags (fanquake) +- bitcoin/bitcoin#21613 enable -Wdocumentation (fanquake) +- bitcoin/bitcoin#21629 Fix configuring when building depends with `NO_BDB=1` (fanquake) +- bitcoin/bitcoin#21654 build, qt: Make Qt rcc output always deterministic (hebasto) +- bitcoin/bitcoin#21655 build, qt: No longer need to set `QT_RCC_TEST=1` for determinism (hebasto) +- bitcoin/bitcoin#21658 fix make deploy for arm64-darwin (sgulls) +- bitcoin/bitcoin#21694 Use XLIFF file to provide more context to Transifex translators (hebasto) +- bitcoin/bitcoin#21708, bitcoin/bitcoin#21593 Drop pointless sed commands (hebasto) +- bitcoin/bitcoin#21731 Update msvc build to use Qt5.12.10 binaries (sipsorcery) +- bitcoin/bitcoin#21733 Re-add command to install vcpkg (dplusplus1024) +- bitcoin/bitcoin#21793 Use `-isysroot` over `--sysroot` on macOS (fanquake) +- bitcoin/bitcoin#21869 Add missing `-D_LIBCPP_DEBUG=1` to debug flags (MarcoFalke) +- bitcoin/bitcoin#21889 macho: check for control flow instrumentation (fanquake) +- bitcoin/bitcoin#21920 Improve macro for testing -latomic requirement (MarcoFalke) +- bitcoin/bitcoin#21991 libevent 2.1.12-stable (fanquake) +- bitcoin/bitcoin#22054 Bump Qt version to 5.12.11 (hebasto) +- bitcoin/bitcoin#22063 Use Qt archive of the same version as the compiled binaries (hebasto) +- bitcoin/bitcoin#22070 Don't use cf-protection when targeting arm-apple-darwin (fanquake) +- bitcoin/bitcoin#22071 Latest config.guess and config.sub (fanquake) +- bitcoin/bitcoin#22075 guix: Misc leftover usability improvements (dongcarl) +- bitcoin/bitcoin#22123 Fix qt.mk for mac arm64 (promag) +- bitcoin/bitcoin#22174 build, qt: Fix libraries linking order for Linux hosts (hebasto) +- bitcoin/bitcoin#22182 guix: Overhaul how guix-{attest,verify} works and hierarchy (dongcarl) +- bitcoin/bitcoin#22186 build, qt: Fix compiling qt package in depends with GCC 11 (hebasto) +- bitcoin/bitcoin#22199 macdeploy: minor fixups and simplifications (fanquake) +- bitcoin/bitcoin#22230 Fix MSVC linker /SubSystem option for bitcoin-qt.exe (hebasto) +- bitcoin/bitcoin#22234 Mark print-% target as phony (dgoncharov) +- bitcoin/bitcoin#22238 improve detection of eBPF support (fanquake) +- bitcoin/bitcoin#22258 Disable deprecated-copy warning only when external warnings are enabled (MarcoFalke) +- bitcoin/bitcoin#22320 set minimum required Boost to 1.64.0 (fanquake) +- bitcoin/bitcoin#22348 Fix cross build for Windows with Boost Process (hebasto) +- bitcoin/bitcoin#22365 guix: Avoid relying on newer symbols by rebasing our cross toolchains on older glibcs (dongcarl) +- bitcoin/bitcoin#22381 guix: Test security-check sanity before performing them (with macOS) (fanquake) +- bitcoin/bitcoin#22405 Remove --enable-glibc-back-compat from Guix build (fanquake) +- bitcoin/bitcoin#22406 Remove --enable-determinism configure option (fanquake) +- bitcoin/bitcoin#22410 Avoid GCC 7.1 ABI change warning in guix build (sipa) +- bitcoin/bitcoin#22436 use aarch64 Clang if cross-compiling for darwin on aarch64 (fanquake) +- bitcoin/bitcoin#22465 guix: Pin kernel-header version, time-machine to upstream 1.3.0 commit (dongcarl) +- bitcoin/bitcoin#22511 guix: Silence `getent(1)` invocation, doc fixups (dongcarl) +- bitcoin/bitcoin#22531 guix: Fixes to guix-{attest,verify} (achow101) +- bitcoin/bitcoin#22642 release: Release with separate sha256sums and sig files (dongcarl) +- bitcoin/bitcoin#22685 clientversion: No suffix `#if CLIENT_VERSION_IS_RELEASE` (dongcarl) +- bitcoin/bitcoin#22713 Fix build with Boost 1.77.0 (sizeofvoid) + +### Tests and QA +- bitcoin/bitcoin#14604 Add test and refactor `feature_block.py` (sanket1729) +- bitcoin/bitcoin#17556 Change `feature_config_args.py` not to rely on strange regtest=0 behavior (ryanofsky) +- bitcoin/bitcoin#18795 wallet issue with orphaned rewards (domob1812) +- bitcoin/bitcoin#18847 compressor: Use a prevector in CompressScript serialization (jb55) +- bitcoin/bitcoin#19259 fuzz: Add fuzzing harness for LoadMempool(…) and DumpMempool(…) (practicalswift) +- bitcoin/bitcoin#19315 Allow outbound & block-relay-only connections in functional tests. (amitiuttarwar) +- bitcoin/bitcoin#19698 Apply strict verification flags for transaction tests and assert backwards compatibility (glozow) +- bitcoin/bitcoin#19801 Check for all possible `OP_CLTV` fail reasons in `feature_cltv.py` (BIP 65) (theStack) +- bitcoin/bitcoin#19893 Remove or explain syncwithvalidationinterfacequeue (MarcoFalke) +- bitcoin/bitcoin#19972 fuzz: Add fuzzing harness for node eviction logic (practicalswift) +- bitcoin/bitcoin#19982 Fix inconsistent lock order in `wallet_tests/CreateWallet` (hebasto) +- bitcoin/bitcoin#20000 Fix creation of "std::string"s with \0s (vasild) +- bitcoin/bitcoin#20047 Use `wait_for_{block,header}` helpers in `p2p_fingerprint.py` (theStack) +- bitcoin/bitcoin#20171 Add functional test `test_txid_inv_delay` (ariard) +- bitcoin/bitcoin#20189 Switch to BIP341's suggested scheme for outputs without script (sipa) +- bitcoin/bitcoin#20248 Fix length of R check in `key_signature_tests` (dgpv) +- bitcoin/bitcoin#20276, bitcoin/bitcoin#20385, bitcoin/bitcoin#20688, bitcoin/bitcoin#20692 Run various mempool tests even with wallet disabled (mjdietzx) +- bitcoin/bitcoin#20323 Create or use existing properly initialized NodeContexts (dongcarl) +- bitcoin/bitcoin#20354 Add `feature_taproot.py --previous_release` (MarcoFalke) +- bitcoin/bitcoin#20370 fuzz: Version handshake (MarcoFalke) +- bitcoin/bitcoin#20377 fuzz: Fill various small fuzzing gaps (practicalswift) +- bitcoin/bitcoin#20425 fuzz: Make CAddrMan fuzzing harness deterministic (practicalswift) +- bitcoin/bitcoin#20430 Sanitizers: Add suppression for unsigned-integer-overflow in libstdc++ (jonasschnelli) +- bitcoin/bitcoin#20437 fuzz: Avoid time-based "non-determinism" in fuzzing harnesses by using mocked GetTime() (practicalswift) +- bitcoin/bitcoin#20458 Add `is_bdb_compiled` helper (Sjors) +- bitcoin/bitcoin#20466 Fix intermittent `p2p_fingerprint` issue (MarcoFalke) +- bitcoin/bitcoin#20472 Add testing of ParseInt/ParseUInt edge cases with leading +/-/0:s (practicalswift) +- bitcoin/bitcoin#20507 sync: print proper lock order location when double lock is detected (vasild) +- bitcoin/bitcoin#20522 Fix sync issue in `disconnect_p2ps` (amitiuttarwar) +- bitcoin/bitcoin#20524 Move `MIN_VERSION_SUPPORTED` to p2p.py (jnewbery) +- bitcoin/bitcoin#20540 Fix `wallet_multiwallet` issue on windows (MarcoFalke) +- bitcoin/bitcoin#20560 fuzz: Link all targets once (MarcoFalke) +- bitcoin/bitcoin#20567 Add option to git-subtree-check to do full check, add help (laanwj) +- bitcoin/bitcoin#20569 Fix intermittent `wallet_multiwallet` issue with `got_loading_error` (MarcoFalke) +- bitcoin/bitcoin#20613 Use Popen.wait instead of RPC in `assert_start_raises_init_error` (MarcoFalke) +- bitcoin/bitcoin#20663 fuzz: Hide `script_assets_test_minimizer` (MarcoFalke) +- bitcoin/bitcoin#20674 fuzz: Call SendMessages after ProcessMessage to increase coverage (MarcoFalke) +- bitcoin/bitcoin#20683 Fix restart node race (MarcoFalke) +- bitcoin/bitcoin#20686 fuzz: replace CNode code with fuzz/util.h::ConsumeNode() (jonatack) +- bitcoin/bitcoin#20733 Inline non-member functions with body in fuzzing headers (pstratem) +- bitcoin/bitcoin#20737 Add missing assignment in `mempool_resurrect.py` (MarcoFalke) +- bitcoin/bitcoin#20745 Correct `epoll_ctl` data race suppression (hebasto) +- bitcoin/bitcoin#20748 Add race:SendZmqMessage tsan suppression (MarcoFalke) +- bitcoin/bitcoin#20760 Set correct nValue for multi-op-return policy check (MarcoFalke) +- bitcoin/bitcoin#20761 fuzz: Check that `NULL_DATA` is unspendable (MarcoFalke) +- bitcoin/bitcoin#20765 fuzz: Check that certain script TxoutType are nonstandard (mjdietzx) +- bitcoin/bitcoin#20772 fuzz: Bolster ExtractDestination(s) checks (mjdietzx) +- bitcoin/bitcoin#20789 fuzz: Rework strong and weak net enum fuzzing (MarcoFalke) +- bitcoin/bitcoin#20828 fuzz: Introduce CallOneOf helper to replace switch-case (MarcoFalke) +- bitcoin/bitcoin#20839 fuzz: Avoid extraneous copy of input data, using Span<> (MarcoFalke) +- bitcoin/bitcoin#20844 Add sanitizer suppressions for AMD EPYC CPUs (MarcoFalke) +- bitcoin/bitcoin#20857 Update documentation in `feature_csv_activation.py` (PiRK) +- bitcoin/bitcoin#20876 Replace getmempoolentry with testmempoolaccept in MiniWallet (MarcoFalke) +- bitcoin/bitcoin#20881 fuzz: net permission flags in net processing (MarcoFalke) +- bitcoin/bitcoin#20882 fuzz: Add missing muhash registration (MarcoFalke) +- bitcoin/bitcoin#20908 fuzz: Use mocktime in `process_message*` fuzz targets (MarcoFalke) +- bitcoin/bitcoin#20915 fuzz: Fail if message type is not fuzzed (MarcoFalke) +- bitcoin/bitcoin#20946 fuzz: Consolidate fuzzing TestingSetup initialization (dongcarl) +- bitcoin/bitcoin#20954 Declare `nodes` type `in test_framework.py` (kiminuo) +- bitcoin/bitcoin#20955 Fix `get_previous_releases.py` for aarch64 (MarcoFalke) +- bitcoin/bitcoin#20969 check that getblockfilter RPC fails without block filter index (theStack) +- bitcoin/bitcoin#20971 Work around libFuzzer deadlock (MarcoFalke) +- bitcoin/bitcoin#20993 Store subversion (user agent) as string in `msg_version` (theStack) +- bitcoin/bitcoin#20995 fuzz: Avoid initializing version to less than `MIN_PEER_PROTO_VERSION` (MarcoFalke) +- bitcoin/bitcoin#20998 Fix BlockToJsonVerbose benchmark (martinus) +- bitcoin/bitcoin#21003 Move MakeNoLogFileContext to `libtest_util`, and use it in bench (MarcoFalke) +- bitcoin/bitcoin#21008 Fix zmq test flakiness, improve speed (theStack) +- bitcoin/bitcoin#21023 fuzz: Disable shuffle when merge=1 (MarcoFalke) +- bitcoin/bitcoin#21037 fuzz: Avoid designated initialization (C++20) in fuzz tests (practicalswift) +- bitcoin/bitcoin#21042 doc, test: Improve `setup_clean_chain` documentation (fjahr) +- bitcoin/bitcoin#21080 fuzz: Configure check for main function (take 2) (MarcoFalke) +- bitcoin/bitcoin#21084 Fix timeout decrease in `feature_assumevalid` (brunoerg) +- bitcoin/bitcoin#21096 Re-add dead code detection (flack) +- bitcoin/bitcoin#21100 Remove unused function `xor_bytes` (theStack) +- bitcoin/bitcoin#21115 Fix Windows cross build (hebasto) +- bitcoin/bitcoin#21117 Remove `assert_blockchain_height` (MarcoFalke) +- bitcoin/bitcoin#21121 Small unit test improvements, including helper to make mempool transaction (amitiuttarwar) +- bitcoin/bitcoin#21124 Remove unnecessary assignment in bdb (brunoerg) +- bitcoin/bitcoin#21125 Change `BOOST_CHECK` to `BOOST_CHECK_EQUAL` for paths (kiminuo) +- bitcoin/bitcoin#21142, bitcoin/bitcoin#21512 fuzz: Add `tx_pool` fuzz target (MarcoFalke) +- bitcoin/bitcoin#21165 Use mocktime in `test_seed_peers` (dhruv) +- bitcoin/bitcoin#21169 fuzz: Add RPC interface fuzzing. Increase fuzzing coverage from 65% to 70% (practicalswift) +- bitcoin/bitcoin#21170 bench: Add benchmark to write json into a string (martinus) +- bitcoin/bitcoin#21178 Run `mempool_reorg.py` even with wallet disabled (DariusParvin) +- bitcoin/bitcoin#21185 fuzz: Remove expensive and redundant muhash from crypto fuzz target (MarcoFalke) +- bitcoin/bitcoin#21200 Speed up `rpc_blockchain.py` by removing miniwallet.generate() (MarcoFalke) +- bitcoin/bitcoin#21211 Move `P2WSH_OP_TRUE` to shared test library (MarcoFalke) +- bitcoin/bitcoin#21228 Avoid comparision of integers with different signs (jonasschnelli) +- bitcoin/bitcoin#21230 Fix `NODE_NETWORK_LIMITED_MIN_BLOCKS` disconnection (MarcoFalke) +- bitcoin/bitcoin#21252 Add missing wait for sync to `feature_blockfilterindex_prune` (MarcoFalke) +- bitcoin/bitcoin#21254 Avoid connecting to real network when running tests (MarcoFalke) +- bitcoin/bitcoin#21264 fuzz: Two scripted diff renames (MarcoFalke) +- bitcoin/bitcoin#21280 Bug fix in `transaction_tests` (glozow) +- bitcoin/bitcoin#21293 Replace accidentally placed bit-OR with logical-OR (hebasto) +- bitcoin/bitcoin#21297 `feature_blockfilterindex_prune.py` improvements (jonatack) +- bitcoin/bitcoin#21310 zmq test: fix sync-up by matching notification to generated block (theStack) +- bitcoin/bitcoin#21334 Additional BIP9 tests (Sjors) +- bitcoin/bitcoin#21338 Add functional test for anchors.dat (brunoerg) +- bitcoin/bitcoin#21345 Bring `p2p_leak.py` up to date (mzumsande) +- bitcoin/bitcoin#21357 Unconditionally check for fRelay field in test framework (jarolrod) +- bitcoin/bitcoin#21358 fuzz: Add missing include (`test/util/setup_common.h`) (MarcoFalke) +- bitcoin/bitcoin#21371 fuzz: fix gcc Woverloaded-virtual build warnings (jonatack) +- bitcoin/bitcoin#21373 Generate fewer blocks in `feature_nulldummy` to fix timeouts, speed up (jonatack) +- bitcoin/bitcoin#21390 Test improvements for UTXO set hash tests (fjahr) +- bitcoin/bitcoin#21410 increase `rpc_timeout` for fundrawtx `test_transaction_too_large` (jonatack) +- bitcoin/bitcoin#21411 add logging, reduce blocks, move `sync_all` in `wallet_` groups (jonatack) +- bitcoin/bitcoin#21438 Add ParseUInt8() test coverage (jonatack) +- bitcoin/bitcoin#21443 fuzz: Implement `fuzzed_dns_lookup_function` as a lambda (practicalswift) +- bitcoin/bitcoin#21445 cirrus: Use SSD cluster for speedup (MarcoFalke) +- bitcoin/bitcoin#21477 Add test for CNetAddr::ToString IPv6 address formatting (RFC 5952) (practicalswift) +- bitcoin/bitcoin#21487 fuzz: Use ConsumeWeakEnum in addrman for service flags (MarcoFalke) +- bitcoin/bitcoin#21488 Add ParseUInt16() unit test and fuzz coverage (jonatack) +- bitcoin/bitcoin#21491 test: remove duplicate assertions in util_tests (jonatack) +- bitcoin/bitcoin#21522 fuzz: Use PickValue where possible (MarcoFalke) +- bitcoin/bitcoin#21531 remove qt byteswap compattests (fanquake) +- bitcoin/bitcoin#21557 small cleanup in RPCNestedTests tests (fanquake) +- bitcoin/bitcoin#21586 Add missing suppression for signed-integer-overflow:txmempool.cpp (MarcoFalke) +- bitcoin/bitcoin#21592 Remove option to make TestChain100Setup non-deterministic (MarcoFalke) +- bitcoin/bitcoin#21597 Document `race:validation_chainstatemanager_tests` suppression (MarcoFalke) +- bitcoin/bitcoin#21599 Replace file level integer overflow suppression with function level suppression (practicalswift) +- bitcoin/bitcoin#21604 Document why no symbol names can be used for suppressions (MarcoFalke) +- bitcoin/bitcoin#21606 fuzz: Extend psbt fuzz target a bit (MarcoFalke) +- bitcoin/bitcoin#21617 fuzz: Fix uninitialized read in i2p test (MarcoFalke) +- bitcoin/bitcoin#21630 fuzz: split FuzzedSock interface and implementation (vasild) +- bitcoin/bitcoin#21634 Skip SQLite fsyncs while testing (achow101) +- bitcoin/bitcoin#21669 Remove spurious double lock tsan suppressions by bumping to clang-12 (MarcoFalke) +- bitcoin/bitcoin#21676 Use mocktime to avoid intermittent failure in `rpc_tests` (MarcoFalke) +- bitcoin/bitcoin#21677 fuzz: Avoid use of low file descriptor ids (which may be in use) in FuzzedSock (practicalswift) +- bitcoin/bitcoin#21678 Fix TestPotentialDeadLockDetected suppression (hebasto) +- bitcoin/bitcoin#21689 Remove intermittently failing and not very meaningful `BOOST_CHECK` in `cnetaddr_basic` (practicalswift) +- bitcoin/bitcoin#21691 Check that no versionbits are re-used (MarcoFalke) +- bitcoin/bitcoin#21707 Extend functional tests for addr relay (mzumsande) +- bitcoin/bitcoin#21712 Test default `include_mempool` value of gettxout (promag) +- bitcoin/bitcoin#21738 Use clang-12 for ASAN, Add missing suppression (MarcoFalke) +- bitcoin/bitcoin#21740 add new python linter to check file names and permissions (windsok) +- bitcoin/bitcoin#21749 Bump shellcheck version (hebasto) +- bitcoin/bitcoin#21754 Run `feature_cltv` with MiniWallet (MarcoFalke) +- bitcoin/bitcoin#21762 Speed up `mempool_spend_coinbase.py` (MarcoFalke) +- bitcoin/bitcoin#21773 fuzz: Ensure prevout is consensus-valid (MarcoFalke) +- bitcoin/bitcoin#21777 Fix `feature_notifications.py` intermittent issue (MarcoFalke) +- bitcoin/bitcoin#21785 Fix intermittent issue in `p2p_addr_relay.py` (MarcoFalke) +- bitcoin/bitcoin#21787 Fix off-by-ones in `rpc_fundrawtransaction` assertions (jonatack) +- bitcoin/bitcoin#21792 Fix intermittent issue in `p2p_segwit.py` (MarcoFalke) +- bitcoin/bitcoin#21795 fuzz: Terminate immediately if a fuzzing harness tries to perform a DNS lookup (belt and suspenders) (practicalswift) +- bitcoin/bitcoin#21798 fuzz: Create a block template in `tx_pool` targets (MarcoFalke) +- bitcoin/bitcoin#21804 Speed up `p2p_segwit.py` (jnewbery) +- bitcoin/bitcoin#21810 fuzz: Various RPC fuzzer follow-ups (practicalswift) +- bitcoin/bitcoin#21814 Fix `feature_config_args.py` intermittent issue (MarcoFalke) +- bitcoin/bitcoin#21821 Add missing test for empty P2WSH redeem (MarcoFalke) +- bitcoin/bitcoin#21822 Resolve bug in `interface_bitcoin_cli.py` (klementtan) +- bitcoin/bitcoin#21846 fuzz: Add `-fsanitize=integer` suppression needed for RPC fuzzer (`generateblock`) (practicalswift) +- bitcoin/bitcoin#21849 fuzz: Limit toxic test globals to their respective scope (MarcoFalke) +- bitcoin/bitcoin#21867 use MiniWallet for `p2p_blocksonly.py` (theStack) +- bitcoin/bitcoin#21873 minor fixes & improvements for files linter test (windsok) +- bitcoin/bitcoin#21874 fuzz: Add `WRITE_ALL_FUZZ_TARGETS_AND_ABORT` (MarcoFalke) +- bitcoin/bitcoin#21884 fuzz: Remove unused --enable-danger-fuzz-link-all option (MarcoFalke) +- bitcoin/bitcoin#21890 fuzz: Limit ParseISO8601DateTime fuzzing to 32-bit (MarcoFalke) +- bitcoin/bitcoin#21891 fuzz: Remove strprintf test cases that are known to fail (MarcoFalke) +- bitcoin/bitcoin#21892 fuzz: Avoid excessively large min fee rate in `tx_pool` (MarcoFalke) +- bitcoin/bitcoin#21895 Add TSA annotations to the WorkQueue class members (hebasto) +- bitcoin/bitcoin#21900 use MiniWallet for `feature_csv_activation.py` (theStack) +- bitcoin/bitcoin#21909 fuzz: Limit max insertions in timedata fuzz test (MarcoFalke) +- bitcoin/bitcoin#21922 fuzz: Avoid timeout in EncodeBase58 (MarcoFalke) +- bitcoin/bitcoin#21927 fuzz: Run const CScript member functions only once (MarcoFalke) +- bitcoin/bitcoin#21929 fuzz: Remove incorrect float round-trip serialization test (MarcoFalke) +- bitcoin/bitcoin#21936 fuzz: Terminate immediately if a fuzzing harness tries to create a TCP socket (belt and suspenders) (practicalswift) +- bitcoin/bitcoin#21941 fuzz: Call const member functions in addrman fuzz test only once (MarcoFalke) +- bitcoin/bitcoin#21945 add P2PK support to MiniWallet (theStack) +- bitcoin/bitcoin#21948 Fix off-by-one in mockscheduler test RPC (MarcoFalke) +- bitcoin/bitcoin#21953 fuzz: Add `utxo_snapshot` target (MarcoFalke) +- bitcoin/bitcoin#21970 fuzz: Add missing CheckTransaction before CheckTxInputs (MarcoFalke) +- bitcoin/bitcoin#21989 Use `COINBASE_MATURITY` in functional tests (kiminuo) +- bitcoin/bitcoin#22003 Add thread safety annotations (ajtowns) +- bitcoin/bitcoin#22004 fuzz: Speed up transaction fuzz target (MarcoFalke) +- bitcoin/bitcoin#22005 fuzz: Speed up banman fuzz target (MarcoFalke) +- bitcoin/bitcoin#22029 [fuzz] Improve transport deserialization fuzz test coverage (dhruv) +- bitcoin/bitcoin#22048 MiniWallet: introduce enum type for output mode (theStack) +- bitcoin/bitcoin#22057 use MiniWallet (P2PK mode) for `feature_dersig.py` (theStack) +- bitcoin/bitcoin#22065 Mark `CheckTxInputs` `[[nodiscard]]`. Avoid UUM in fuzzing harness `coins_view` (practicalswift) +- bitcoin/bitcoin#22069 fuzz: don't try and use fopencookie() when building for Android (fanquake) +- bitcoin/bitcoin#22082 update nanobench from release 4.0.0 to 4.3.4 (martinus) +- bitcoin/bitcoin#22086 remove BasicTestingSetup from unit tests that don't need it (fanquake) +- bitcoin/bitcoin#22089 MiniWallet: fix fee calculation for P2PK and check tx vsize (theStack) +- bitcoin/bitcoin#21107, bitcoin/bitcoin#22092 Convert documentation into type annotations (fanquake) +- bitcoin/bitcoin#22095 Additional BIP32 test vector for hardened derivation with leading zeros (kristapsk) +- bitcoin/bitcoin#22103 Fix IPv6 check on BSD systems (n-thumann) +- bitcoin/bitcoin#22118 check anchors.dat when node starts for the first time (brunoerg) +- bitcoin/bitcoin#22120 `p2p_invalid_block`: Check that a block rejected due to too-new tim… (willcl-ark) +- bitcoin/bitcoin#22153 Fix `p2p_leak.py` intermittent failure (mzumsande) +- bitcoin/bitcoin#22169 p2p, rpc, fuzz: various tiny follow-ups (jonatack) +- bitcoin/bitcoin#22176 Correct outstanding -Werror=sign-compare errors (Empact) +- bitcoin/bitcoin#22180 fuzz: Increase branch coverage of the float fuzz target (MarcoFalke) +- bitcoin/bitcoin#22187 Add `sync_blocks` in `wallet_orphanedreward.py` (domob1812) +- bitcoin/bitcoin#22201 Fix TestShell to allow running in Jupyter Notebook (josibake) +- bitcoin/bitcoin#22202 Add temporary coinstats suppressions (MarcoFalke) +- bitcoin/bitcoin#22203 Use ConnmanTestMsg from test lib in `denialofservice_tests` (MarcoFalke) +- bitcoin/bitcoin#22210 Use MiniWallet in `test_no_inherited_signaling` RBF test (MarcoFalke) +- bitcoin/bitcoin#22224 Update msvc and appveyor builds to use Qt5.12.11 binaries (sipsorcery) +- bitcoin/bitcoin#22249 Kill process group to avoid dangling processes when using `--failfast` (S3RK) +- bitcoin/bitcoin#22267 fuzz: Speed up crypto fuzz target (MarcoFalke) +- bitcoin/bitcoin#22270 Add bitcoin-util tests (+refactors) (MarcoFalke) +- bitcoin/bitcoin#22271 fuzz: Assert roundtrip equality for `CPubKey` (theStack) +- bitcoin/bitcoin#22279 fuzz: add missing ECCVerifyHandle to `base_encode_decode` (apoelstra) +- bitcoin/bitcoin#22292 bench, doc: benchmarking updates and fixups (jonatack) +- bitcoin/bitcoin#22306 Improvements to `p2p_addr_relay.py` (amitiuttarwar) +- bitcoin/bitcoin#22310 Add functional test for replacement relay fee check (ariard) +- bitcoin/bitcoin#22311 Add missing syncwithvalidationinterfacequeue in `p2p_blockfilters` (MarcoFalke) +- bitcoin/bitcoin#22313 Add missing `sync_all` to `feature_coinstatsindex` (MarcoFalke) +- bitcoin/bitcoin#22322 fuzz: Check banman roundtrip (MarcoFalke) +- bitcoin/bitcoin#22363 Use `script_util` helpers for creating P2{PKH,SH,WPKH,WSH} scripts (theStack) +- bitcoin/bitcoin#22399 fuzz: Rework CTxDestination fuzzing (MarcoFalke) +- bitcoin/bitcoin#22408 add tests for `bad-txns-prevout-null` reject reason (theStack) +- bitcoin/bitcoin#22445 fuzz: Move implementations of non-template fuzz helpers from util.h to util.cpp (sriramdvt) +- bitcoin/bitcoin#22446 Fix `wallet_listdescriptors.py` if bdb is not compiled (hebasto) +- bitcoin/bitcoin#22447 Whitelist `rpc_rawtransaction` peers to speed up tests (jonatack) +- bitcoin/bitcoin#22742 Use proper target in `do_fund_send` (S3RK) + +### Miscellaneous +- bitcoin/bitcoin#19337 sync: Detect double lock from the same thread (vasild) +- bitcoin/bitcoin#19809 log: Prefix log messages with function name and source code location if -logsourcelocations is set (practicalswift) +- bitcoin/bitcoin#19866 eBPF Linux tracepoints (jb55) +- bitcoin/bitcoin#20024 init: Fix incorrect warning "Reducing -maxconnections from N to N-1, because of system limitations" (practicalswift) +- bitcoin/bitcoin#20145 contrib: Add getcoins.py script to get coins from (signet) faucet (kallewoof) +- bitcoin/bitcoin#20255 util: Add assume() identity function (MarcoFalke) +- bitcoin/bitcoin#20288 script, doc: Contrib/seeds updates (jonatack) +- bitcoin/bitcoin#20358 src/randomenv.cpp: Fix build on uclibc (ffontaine) +- bitcoin/bitcoin#20406 util: Avoid invalid integer negation in formatmoney and valuefromamount (practicalswift) +- bitcoin/bitcoin#20434 contrib: Parse elf directly for symbol and security checks (laanwj) +- bitcoin/bitcoin#20451 lint: Run mypy over contrib/devtools (fanquake) +- bitcoin/bitcoin#20476 contrib: Add test for elf symbol-check (laanwj) +- bitcoin/bitcoin#20530 lint: Update cppcheck linter to c++17 and improve explicit usage (fjahr) +- bitcoin/bitcoin#20589 log: Clarify that failure to read/write `fee_estimates.dat` is non-fatal (MarcoFalke) +- bitcoin/bitcoin#20602 util: Allow use of c++14 chrono literals (MarcoFalke) +- bitcoin/bitcoin#20605 init: Signal-safe instant shutdown (laanwj) +- bitcoin/bitcoin#20608 contrib: Add symbol check test for PE binaries (fanquake) +- bitcoin/bitcoin#20689 contrib: Replace binary verification script verify.sh with python rewrite (theStack) +- bitcoin/bitcoin#20715 util: Add argsmanager::getcommand() and use it in bitcoin-wallet (MarcoFalke) +- bitcoin/bitcoin#20735 script: Remove outdated extract-osx-sdk.sh (hebasto) +- bitcoin/bitcoin#20817 lint: Update list of spelling linter false positives, bump to codespell 2.0.0 (theStack) +- bitcoin/bitcoin#20884 script: Improve robustness of bitcoind.service on startup (hebasto) +- bitcoin/bitcoin#20906 contrib: Embed c++11 patch in `install_db4.sh` (gruve-p) +- bitcoin/bitcoin#21004 contrib: Fix docker args conditional in gitian-build (setpill) +- bitcoin/bitcoin#21007 bitcoind: Add -daemonwait option to wait for initialization (laanwj) +- bitcoin/bitcoin#21041 log: Move "Pre-allocating up to position 0x[…] in […].dat" log message to debug category (practicalswift) +- bitcoin/bitcoin#21059 Drop boost/preprocessor dependencies (hebasto) +- bitcoin/bitcoin#21087 guix: Passthrough `BASE_CACHE` into container (dongcarl) +- bitcoin/bitcoin#21088 guix: Jump forwards in time-machine and adapt (dongcarl) +- bitcoin/bitcoin#21089 guix: Add support for powerpc64{,le} (dongcarl) +- bitcoin/bitcoin#21110 util: Remove boost `posix_time` usage from `gettime*` (fanquake) +- bitcoin/bitcoin#21111 Improve OpenRC initscript (parazyd) +- bitcoin/bitcoin#21123 code style: Add EditorConfig file (kiminuo) +- bitcoin/bitcoin#21173 util: Faster hexstr => 13% faster blocktojson (martinus) +- bitcoin/bitcoin#21221 tools: Allow argument/parameter bin packing in clang-format (jnewbery) +- bitcoin/bitcoin#21244 Move GetDataDir to ArgsManager (kiminuo) +- bitcoin/bitcoin#21255 contrib: Run test-symbol-check for risc-v (fanquake) +- bitcoin/bitcoin#21271 guix: Explicitly set umask in build container (dongcarl) +- bitcoin/bitcoin#21300 script: Add explanatory comment to tc.sh (dscotese) +- bitcoin/bitcoin#21317 util: Make assume() usable as unary expression (MarcoFalke) +- bitcoin/bitcoin#21336 Make .gitignore ignore src/test/fuzz/fuzz.exe (hebasto) +- bitcoin/bitcoin#21337 guix: Update darwin native packages dependencies (hebasto) +- bitcoin/bitcoin#21405 compat: remove memcpy -> memmove backwards compatibility alias (fanquake) +- bitcoin/bitcoin#21418 contrib: Make systemd invoke dependencies only when ready (laanwj) +- bitcoin/bitcoin#21447 Always add -daemonwait to known command line arguments (hebasto) +- bitcoin/bitcoin#21471 bugfix: Fix `bech32_encode` calls in `gen_key_io_test_vectors.py` (sipa) +- bitcoin/bitcoin#21615 script: Add trusted key for hebasto (hebasto) +- bitcoin/bitcoin#21664 contrib: Use lief for macos and windows symbol & security checks (fanquake) +- bitcoin/bitcoin#21695 contrib: Remove no longer used contrib/bitcoin-qt.pro (hebasto) +- bitcoin/bitcoin#21711 guix: Add full installation and usage documentation (dongcarl) +- bitcoin/bitcoin#21799 guix: Use `gcc-8` across the board (dongcarl) +- bitcoin/bitcoin#21802 Avoid UB in util/asmap (advance a dereferenceable iterator outside its valid range) (MarcoFalke) +- bitcoin/bitcoin#21823 script: Update reviewers (jonatack) +- bitcoin/bitcoin#21850 Remove `GetDataDir(net_specific)` function (kiminuo) +- bitcoin/bitcoin#21871 scripts: Add checks for minimum required os versions (fanquake) +- bitcoin/bitcoin#21966 Remove double serialization; use software encoder for fee estimation (sipa) +- bitcoin/bitcoin#22060 contrib: Add torv3 seed nodes for testnet, drop v2 ones (laanwj) +- bitcoin/bitcoin#22244 devtools: Correctly extract symbol versions in symbol-check (laanwj) +- bitcoin/bitcoin#22533 guix/build: Remove vestigial SKIPATTEST.TAG (dongcarl) +- bitcoin/bitcoin#22643 guix-verify: Non-zero exit code when anything fails (dongcarl) +- bitcoin/bitcoin#22654 guix: Don't include directory name in SHA256SUMS (achow101) + +### Documentation +- bitcoin/bitcoin#15451 clarify getdata limit after #14897 (HashUnlimited) +- bitcoin/bitcoin#15545 Explain why CheckBlock() is called before AcceptBlock (Sjors) +- bitcoin/bitcoin#17350 Add developer documentation to isminetype (HAOYUatHZ) +- bitcoin/bitcoin#17934 Use `CONFIG_SITE` variable instead of --prefix option (hebasto) +- bitcoin/bitcoin#18030 Coin::IsSpent() can also mean never existed (Sjors) +- bitcoin/bitcoin#18096 IsFinalTx comment about nSequence & `OP_CLTV` (nothingmuch) +- bitcoin/bitcoin#18568 Clarify developer notes about constant naming (ryanofsky) +- bitcoin/bitcoin#19961 doc: tor.md updates (jonatack) +- bitcoin/bitcoin#19968 Clarify CRollingBloomFilter size estimate (robot-dreams) +- bitcoin/bitcoin#20200 Rename CODEOWNERS to REVIEWERS (adamjonas) +- bitcoin/bitcoin#20329 docs/descriptors.md: Remove hardened marker in the path after xpub (dgpv) +- bitcoin/bitcoin#20380 Add instructions on how to fuzz the P2P layer using Honggfuzz NetDriver (practicalswift) +- bitcoin/bitcoin#20414 Remove generated manual pages from master branch (laanwj) +- bitcoin/bitcoin#20473 Document current boost dependency as 1.71.0 (laanwj) +- bitcoin/bitcoin#20512 Add bash as an OpenBSD dependency (emilengler) +- bitcoin/bitcoin#20568 Use FeeModes doc helper in estimatesmartfee (MarcoFalke) +- bitcoin/bitcoin#20577 libconsensus: add missing error code description, fix NBitcoin link (theStack) +- bitcoin/bitcoin#20587 Tidy up Tor doc (more stringent) (wodry) +- bitcoin/bitcoin#20592 Update wtxidrelay documentation per BIP339 (jonatack) +- bitcoin/bitcoin#20601 Update for FreeBSD 12.2, add GUI Build Instructions (jarolrod) +- bitcoin/bitcoin#20635 fix misleading comment about call to non-existing function (pox) +- bitcoin/bitcoin#20646 Refer to BIPs 339/155 in feature negotiation (jonatack) +- bitcoin/bitcoin#20653 Move addr relay comment in net to correct place (MarcoFalke) +- bitcoin/bitcoin#20677 Remove shouty enums in `net_processing` comments (sdaftuar) +- bitcoin/bitcoin#20741 Update 'Secure string handling' (prayank23) +- bitcoin/bitcoin#20757 tor.md and -onlynet help updates (jonatack) +- bitcoin/bitcoin#20829 Add -netinfo help (jonatack) +- bitcoin/bitcoin#20830 Update developer notes with signet (jonatack) +- bitcoin/bitcoin#20890 Add explicit macdeployqtplus dependencies install step (hebasto) +- bitcoin/bitcoin#20913 Add manual page generation for bitcoin-util (laanwj) +- bitcoin/bitcoin#20985 Add xorriso to macOS depends packages (fanquake) +- bitcoin/bitcoin#20986 Update developer notes to discourage very long lines (jnewbery) +- bitcoin/bitcoin#20987 Add instructions for generating RPC docs (ben-kaufman) +- bitcoin/bitcoin#21026 Document use of make-tag script to make tags (laanwj) +- bitcoin/bitcoin#21028 doc/bips: Add BIPs 43, 44, 49, and 84 (luke-jr) +- bitcoin/bitcoin#21049 Add release notes for listdescriptors RPC (S3RK) +- bitcoin/bitcoin#21060 More precise -debug and -debugexclude doc (wodry) +- bitcoin/bitcoin#21077 Clarify -timeout and -peertimeout config options (glozow) +- bitcoin/bitcoin#21105 Correctly identify script type (niftynei) +- bitcoin/bitcoin#21163 Guix is shipped in Debian and Ubuntu (MarcoFalke) +- bitcoin/bitcoin#21210 Rework internal and external links (MarcoFalke) +- bitcoin/bitcoin#21246 Correction for VerifyTaprootCommitment comments (roconnor-blockstream) +- bitcoin/bitcoin#21263 Clarify that squashing should happen before review (MarcoFalke) +- bitcoin/bitcoin#21323 guix, doc: Update default HOSTS value (hebasto) +- bitcoin/bitcoin#21324 Update build instructions for Fedora (hebasto) +- bitcoin/bitcoin#21343 Revamp macOS build doc (jarolrod) +- bitcoin/bitcoin#21346 install qt5 when building on macOS (fanquake) +- bitcoin/bitcoin#21384 doc: add signet to bitcoin.conf documentation (jonatack) +- bitcoin/bitcoin#21394 Improve comment about protected peers (amitiuttarwar) +- bitcoin/bitcoin#21398 Update fuzzing docs for afl-clang-lto (MarcoFalke) +- bitcoin/bitcoin#21444 net, doc: Doxygen updates and fixes in netbase.{h,cpp} (jonatack) +- bitcoin/bitcoin#21481 Tell howto install clang-format on Debian/Ubuntu (wodry) +- bitcoin/bitcoin#21567 Fix various misleading comments (glozow) +- bitcoin/bitcoin#21661 Fix name of script guix-build (Emzy) +- bitcoin/bitcoin#21672 Remove boostrap info from `GUIX_COMMON_FLAGS` doc (fanquake) +- bitcoin/bitcoin#21688 Note on SDK for macOS depends cross-compile (jarolrod) +- bitcoin/bitcoin#21709 Update reduce-memory.md and bitcoin.conf -maxconnections info (jonatack) +- bitcoin/bitcoin#21710 update helps for addnode rpc and -addnode/-maxconnections config options (jonatack) +- bitcoin/bitcoin#21752 Clarify that feerates are per virtual size (MarcoFalke) +- bitcoin/bitcoin#21811 Remove Visual Studio 2017 reference from readme (sipsorcery) +- bitcoin/bitcoin#21818 Fixup -coinstatsindex help, update bitcoin.conf and files.md (jonatack) +- bitcoin/bitcoin#21856 add OSS-Fuzz section to fuzzing.md doc (adamjonas) +- bitcoin/bitcoin#21912 Remove mention of priority estimation (MarcoFalke) +- bitcoin/bitcoin#21925 Update bips.md for 0.21.1 (MarcoFalke) +- bitcoin/bitcoin#21942 improve make with parallel jobs description (klementtan) +- bitcoin/bitcoin#21947 Fix OSS-Fuzz links (MarcoFalke) +- bitcoin/bitcoin#21988 note that brew installed qt is not supported (jarolrod) +- bitcoin/bitcoin#22056 describe in fuzzing.md how to reproduce a CI crash (jonatack) +- bitcoin/bitcoin#22080 add maxuploadtarget to bitcoin.conf example (jarolrod) +- bitcoin/bitcoin#22088 Improve note on choosing posix mingw32 (jarolrod) +- bitcoin/bitcoin#22109 Fix external links (IRC, …) (MarcoFalke) +- bitcoin/bitcoin#22121 Various validation doc fixups (MarcoFalke) +- bitcoin/bitcoin#22172 Update tor.md, release notes with removal of tor v2 support (jonatack) +- bitcoin/bitcoin#22204 Remove obsolete `okSafeMode` RPC guideline from developer notes (theStack) +- bitcoin/bitcoin#22208 Update `REVIEWERS` (practicalswift) +- bitcoin/bitcoin#22250 add basic I2P documentation (vasild) +- bitcoin/bitcoin#22296 Final merge of release notes snippets, mv to wiki (MarcoFalke) +- bitcoin/bitcoin#22335 recommend `--disable-external-signer` in OpenBSD build guide (theStack) +- bitcoin/bitcoin#22339 Document minimum required libc++ version (hebasto) +- bitcoin/bitcoin#22349 Repository IRC updates (jonatack) +- bitcoin/bitcoin#22360 Remove unused section from release process (MarcoFalke) +- bitcoin/bitcoin#22369 Add steps for Transifex to release process (jonatack) +- bitcoin/bitcoin#22393 Added info to bitcoin.conf doc (bliotti) +- bitcoin/bitcoin#22402 Install Rosetta on M1-macOS for qt in depends (hebasto) +- bitcoin/bitcoin#22432 Fix incorrect `testmempoolaccept` doc (glozow) +- bitcoin/bitcoin#22648 doc, test: improve i2p/tor docs and i2p reachable unit tests (jonatack) + +Credits +======= + +Thanks to everyone who directly contributed to this release: + +- Aaron Clauson +- Adam Jonas +- amadeuszpawlik +- Amiti Uttarwar +- Andrew Chow +- Andrew Poelstra +- Anthony Towns +- Antoine Poinsot +- Antoine Riard +- apawlik +- apitko +- Ben Carman +- Ben Woosley +- benk10 +- Bezdrighin +- Block Mechanic +- Brian Liotti +- Bruno Garcia +- Carl Dong +- Christian Decker +- coinforensics +- Cory Fields +- Dan Benjamin +- Daniel Kraft +- Darius Parvin +- Dhruv Mehta +- Dmitry Goncharov +- Dmitry Petukhov +- dplusplus1024 +- dscotese +- Duncan Dean +- Elle Mouton +- Elliott Jin +- Emil Engler +- Ethan Heilman +- eugene +- Evan Klitzke +- Fabian Jahr +- Fabrice Fontaine +- fanquake +- fdov +- flack +- Fotis Koutoupas +- Fu Yong Quah +- fyquah +- glozow +- Gregory Sanders +- Guido Vranken +- Gunar C. Gessner +- h +- HAOYUatHZ +- Hennadii Stepanov +- Igor Cota +- Ikko Ashimine +- Ivan Metlushko +- jackielove4u +- James O'Beirne +- Jarol Rodriguez +- Joel Klabo +- John Newbery +- Jon Atack +- Jonas Schnelli +- João Barbosa +- Josiah Baker +- Karl-Johan Alm +- Kiminuo +- Klement Tan +- Kristaps Kaupe +- Larry Ruane +- lisa neigut +- Lucas Ontivero +- Luke Dashjr +- Maayan Keshet +- MarcoFalke +- Martin Ankerl +- Martin Zumsande +- Michael Dietz +- Michael Polzer +- Michael Tidwell +- Niklas Gögge +- nthumann +- Oliver Gugger +- parazyd +- Patrick Strateman +- Pavol Rusnak +- Peter Bushnell +- Pierre K +- Pieter Wuille +- PiRK +- pox +- practicalswift +- Prayank +- R E Broadley +- Rafael Sadowski +- randymcmillan +- Raul Siles +- Riccardo Spagni +- Russell O'Connor +- Russell Yanofsky +- S3RK +- saibato +- Samuel Dobson +- sanket1729 +- Sawyer Billings +- Sebastian Falbesoner +- setpill +- sgulls +- sinetek +- Sjors Provoost +- Sriram +- Stephan Oeste +- Suhas Daftuar +- Sylvain Goumy +- t-bast +- Troy Giorshev +- Tushar Singla +- Tyler Chambers +- Uplab +- Vasil Dimov +- W. J. van der Laan +- willcl-ark +- William Bright +- William Casarin +- windsok +- wodry +- Yerzhan Mazhkenov +- Yuval Kogman +- Zero + +As well as to everyone that helped with translations on +[Transifex](https://www.transifex.com/bitcoin/bitcoin/). diff --git a/doc/release-notes/release-notes-23.0.md b/doc/release-notes/release-notes-23.0.md new file mode 100644 index 0000000000..b1467a0f71 --- /dev/null +++ b/doc/release-notes/release-notes-23.0.md @@ -0,0 +1,373 @@ +23.0 Release Notes +================== + +Bitcoin Core version 23.0 is now available from: + + <https://bitcoincore.org/bin/bitcoin-core-23.0/> + +This release includes new features, various bug fixes and performance +improvements, as well as updated translations. + +Please report bugs using the issue tracker at GitHub: + + <https://github.com/bitcoin/bitcoin/issues> + +To receive security and update notifications, please subscribe to: + + <https://bitcoincore.org/en/list/announcements/join/> + +How to Upgrade +============== + +If you are running an older version, shut it down. Wait until it has completely +shut down (which might take a few minutes in some cases), then run the +installer (on Windows) or just copy over `/Applications/Bitcoin-Qt` (on Mac) +or `bitcoind`/`bitcoin-qt` (on Linux). + +Upgrading directly from a version of Bitcoin Core that has reached its EOL is +possible, but it might take some time if the data directory needs to be migrated. Old +wallet versions of Bitcoin Core are generally supported. + +Compatibility +============== + +Bitcoin Core is supported and extensively tested on operating systems +using the Linux kernel, macOS 10.15+, and Windows 7 and newer. Bitcoin +Core should also work on most other Unix-like systems but is not as +frequently tested on them. It is not recommended to use Bitcoin Core on +unsupported systems. + +Notable changes +=============== + +P2P and network changes +----------------------- + +- A bitcoind node will no longer rumour addresses to inbound peers by default. + They will become eligible for address gossip after sending an ADDR, ADDRV2, + or GETADDR message. (#21528) + +- Before this release, Bitcoin Core had a strong preference to try to connect only to peers that listen on port 8333. As a result of that, Bitcoin nodes listening on non-standard ports would likely not get any Bitcoin Core peers connecting to them. This preference has been removed. (#23542) + +- Full support has been added for the CJDNS network. See the new option `-cjdnsreachable` and [doc/cjdns.md](https://github.com/bitcoin/bitcoin/tree/23.x/doc/cjdns.md) (#23077) + +Fee estimation changes +---------------------- + +- Fee estimation now takes the feerate of replacement (RBF) transactions into + account. (#22539) + +Rescan startup parameter removed +-------------------------------- + +The `-rescan` startup parameter has been removed. Wallets which require +rescanning due to corruption will still be rescanned on startup. +Otherwise, please use the `rescanblockchain` RPC to trigger a rescan. (#23123) + +Tracepoints and Userspace, Statically Defined Tracing support +------------------------------------------------------------- + +Bitcoin Core release binaries for Linux now include experimental tracepoints which +act as an interface for process-internal events. These can be used for review, +debugging, monitoring, and more. The tracepoint API is semi-stable. While the API +is tested, process internals might change between releases requiring changes to the +tracepoints. Information about the existing tracepoints can be found under +[doc/tracing.md](https://github.com/bitcoin/bitcoin/blob/23.x/doc/tracing.md) and +usage examples are provided in [contrib/tracing/](https://github.com/bitcoin/bitcoin/tree/23.x/contrib/tracing). + +Updated RPCs +------------ + +- The `validateaddress` RPC now returns an `error_locations` array for invalid + addresses, with the indices of invalid character locations in the address (if + known). For example, this will attempt to locate up to two Bech32 errors, and + return their locations if successful. Success and correctness are only guaranteed + if fewer than two substitution errors have been made. + The error message returned in the `error` field now also returns more specific + errors when decoding fails. (#16807) + +- The `-deprecatedrpc=addresses` configuration option has been removed. RPCs + `gettxout`, `getrawtransaction`, `decoderawtransaction`, `decodescript`, + `gettransaction verbose=true` and REST endpoints `/rest/tx`, `/rest/getutxos`, + `/rest/block` no longer return the `addresses` and `reqSigs` fields, which + were previously deprecated in 22.0. (#22650) +- The `getblock` RPC command now supports verbosity level 3 containing transaction inputs' + `prevout` information. The existing `/rest/block/` REST endpoint is modified to contain + this information too. Every `vin` field will contain an additional `prevout` subfield + describing the spent output. `prevout` contains the following keys: + - `generated` - true if the spent coins was a coinbase. + - `height` + - `value` + - `scriptPubKey` + +- The top-level fee fields `fee`, `modifiedfee`, `ancestorfees` and `descendantfees` + returned by RPCs `getmempoolentry`,`getrawmempool(verbose=true)`, + `getmempoolancestors(verbose=true)` and `getmempooldescendants(verbose=true)` + are deprecated and will be removed in the next major version (use + `-deprecated=fees` if needed in this version). The same fee fields can be accessed + through the `fees` object in the result. WARNING: deprecated + fields `ancestorfees` and `descendantfees` are denominated in sats, whereas all + fields in the `fees` object are denominated in BTC. (#22689) + +- Both `createmultisig` and `addmultisigaddress` now include a `warnings` + field, which will show a warning if a non-legacy address type is requested + when using uncompressed public keys. (#23113) + +Changes to wallet related RPCs can be found in the Wallet section below. + +New RPCs +-------- + +- Information on soft fork status has been moved from `getblockchaininfo` + to the new `getdeploymentinfo` RPC which allows querying soft fork status at any + block, rather than just at the chain tip. Inclusion of soft fork + status in `getblockchaininfo` can currently be restored using the + configuration `-deprecatedrpc=softforks`, but this will be removed in + a future release. Note that in either case, the `status` field + now reflects the status of the current block rather than the next + block. (#23508) + +Files +----- + +* On startup, the list of banned hosts and networks (via `setban` RPC) in + `banlist.dat` is ignored and only `banlist.json` is considered. Bitcoin Core + version 22.x is the only version that can read `banlist.dat` and also write + it to `banlist.json`. If `banlist.json` already exists, version 22.x will not + try to translate the `banlist.dat` into json. After an upgrade, `listbanned` + can be used to double check the parsed entries. (#22570) + +Updated settings +---------------- + +- In previous releases, the meaning of the command line option + `-persistmempool` (without a value provided) incorrectly disabled mempool + persistence. `-persistmempool` is now treated like other boolean options to + mean `-persistmempool=1`. Passing `-persistmempool=0`, `-persistmempool=1` + and `-nopersistmempool` is unaffected. (#23061) + +- `-maxuploadtarget` now allows human readable byte units [k|K|m|M|g|G|t|T]. + E.g. `-maxuploadtarget=500g`. No whitespace, +- or fractions allowed. + Default is `M` if no suffix provided. (#23249) + +- If `-proxy=` is given together with `-noonion` then the provided proxy will + not be set as a proxy for reaching the Tor network. So it will not be + possible to open manual connections to the Tor network for example with the + `addnode` RPC. To mimic the old behavior use `-proxy=` together with + `-onlynet=` listing all relevant networks except `onion`. (#22834) + +Tools and Utilities +------------------- + +- Update `-getinfo` to return data in a user-friendly format that also reduces vertical space. (#21832) + +- CLI `-addrinfo` now returns a single field for the number of `onion` addresses + known to the node instead of separate `torv2` and `torv3` fields, as support + for Tor V2 addresses was removed from Bitcoin Core in 22.0. (#22544) + +Wallet +------ + +- Descriptor wallets are now the default wallet type. Newly created wallets + will use descriptors unless `descriptors=false` is set during `createwallet`, or + the `Descriptor wallet` checkbox is unchecked in the GUI. + + Note that wallet RPC commands like `importmulti` and `dumpprivkey` cannot be + used with descriptor wallets, so if your client code relies on these commands + without specifying `descriptors=false` during wallet creation, you will need + to update your code. + +- Newly created descriptor wallets will contain an automatically generated `tr()` + descriptor which allows for creating single key Taproot receiving addresses. + +- `upgradewallet` will now automatically flush the keypool if upgrading + from a non-HD wallet to an HD wallet, to immediately start using the + newly-generated HD keys. (#23093) + +- a new RPC `newkeypool` has been added, which will flush (entirely + clear and refill) the keypool. (#23093) + +- `listunspent` now includes `ancestorcount`, `ancestorsize`, and + `ancestorfees` for each transaction output that is still in the mempool. + (#12677) + +- `lockunspent` now optionally takes a third parameter, `persistent`, which + causes the lock to be written persistently to the wallet database. This + allows UTXOs to remain locked even after node restarts or crashes. (#23065) + +- `receivedby` RPCs now include coinbase transactions. Previously, the + following wallet RPCs excluded coinbase transactions: `getreceivedbyaddress`, + `getreceivedbylabel`, `listreceivedbyaddress`, `listreceivedbylabel`. This + release changes this behaviour and returns results accounting for received + coins from coinbase outputs. The previous behaviour can be restored using the + configuration `-deprecatedrpc=exclude_coinbase`, but may be removed in a + future release. (#14707) + +- A new option in the same `receivedby` RPCs, `include_immature_coinbase` + (default=`false`), determines whether to account for immature coinbase + transactions. Immature coinbase transactions are coinbase transactions that + have 100 or fewer confirmations, and are not spendable. (#14707) + +GUI changes +----------- + +- UTXOs which are locked via the GUI are now stored persistently in the + wallet database, so are not lost on node shutdown or crash. (#23065) + +- The Bech32 checkbox has been replaced with a dropdown for all address types, including the new Bech32m (BIP-350) standard for Taproot enabled wallets. + +Low-level changes +================= + +RPC +--- + +- `getblockchaininfo` now returns a new `time` field, that provides the chain tip time. (#22407) + +Tests +----- + +- For the `regtest` network the activation heights of several softforks were + set to block height 1. They can be changed by the runtime setting + `-testactivationheight=name@height`. (#22818) + +Credits +======= + +Thanks to everyone who directly contributed to this release: + +- 0xb10c +- 0xree +- Aaron Clauson +- Adrian-Stefan Mares +- agroce +- aitorjs +- Alex Groce +- amadeuszpawlik +- Amiti Uttarwar +- Andrew Chow +- Andrew Poelstra +- Andrew Toth +- anouar kappitou +- Anthony Towns +- Antoine Poinsot +- Arnab Sen +- Ben Woosley +- benthecarman +- Bitcoin Hodler +- BitcoinTsunami +- brianddk +- Bruno Garcia +- CallMeMisterOwl +- Calvin Kim +- Carl Dong +- Cory Fields +- Cuong V. Nguyen +- Darius Parvin +- Dhruv Mehta +- Dimitri Deijs +- Dimitris Apostolou +- Dmitry Goncharov +- Douglas Chimento +- eugene +- Fabian Jahr +- fanquake +- Florian Baumgartl +- fyquah +- Gleb Naumenko +- glozow +- Gregory Sanders +- Heebs +- Hennadii Stepanov +- hg333 +- HiLivin +- Igor Cota +- Jadi +- James O'Beirne +- Jameson Lopp +- Jarol Rodriguez +- Jeremy Rand +- Jeremy Rubin +- Joan Karadimov +- John Newbery +- Jon Atack +- João Barbosa +- josibake +- junderw +- Karl-Johan Alm +- katesalazar +- Kennan Mell +- Kiminuo +- Kittywhiskers Van Gogh +- Klement Tan +- Kristaps Kaupe +- Kuro +- Larry Ruane +- lsilva01 +- lucash-dev +- Luke Dashjr +- MarcoFalke +- Martin Leitner-Ankerl +- Martin Zumsande +- Matt Corallo +- Matt Whitlock +- MeshCollider +- Michael Dietz +- Murch +- naiza +- Nathan Garabedian +- Nelson Galdeman +- NikhilBartwal +- Niklas Gögge +- node01 +- nthumann +- Pasta +- Patrick Kamin +- Pavel Safronov +- Pavol Rusnak +- Perlover +- Pieter Wuille +- practicalswift +- pradumnasaraf +- pranabp-bit +- Prateek Sancheti +- Prayank +- Rafael Sadowski +- rajarshimaitra +- randymcmillan +- ritickgoenka +- Rob Fielding +- Rojar Smith +- Russell Yanofsky +- S3RK +- Saibato +- Samuel Dobson +- sanket1729 +- seaona +- Sebastian Falbesoner +- sh15h4nk +- Shashwat +- Shorya +- ShubhamPalriwala +- Shubhankar Gambhir +- Sjors Provoost +- sogoagain +- sstone +- stratospher +- Suriyaa Rocky Sundararuban +- Taeik Lim +- TheCharlatan +- Tim Ruffing +- Tobin Harding +- Troy Giorshev +- Tyler Chambers +- Vasil Dimov +- W. J. van der Laan +- w0xlt +- willcl-ark +- William Casarin +- zealsham +- Zero-1729 + +As well as to everyone that helped with translations on +[Transifex](https://www.transifex.com/bitcoin/bitcoin/). diff --git a/doc/release-notes/release-notes-24408.md b/doc/release-notes/release-notes-24408.md new file mode 100644 index 0000000000..1072ec786a --- /dev/null +++ b/doc/release-notes/release-notes-24408.md @@ -0,0 +1,5 @@ +New RPCs +-------- + +- A new `gettxspendingprevout` RPC has been added, which scans the mempool to find + transactions spending any of the given outpoints. (#24408)
\ No newline at end of file diff --git a/doc/release-process.md b/doc/release-process.md index 3e4748b742..17a03f7dcd 100644 --- a/doc/release-process.md +++ b/doc/release-process.md @@ -6,21 +6,22 @@ Release Process ### Before every release candidate * Update translations see [translation_process.md](https://github.com/bitcoin/bitcoin/blob/master/doc/translation_process.md#synchronising-translations). -* Update manpages, see [gen-manpages.sh](https://github.com/bitcoin/bitcoin/blob/master/contrib/devtools/README.md#gen-manpagessh). * Update release candidate version in `configure.ac` (`CLIENT_VERSION_RC`). +* Update manpages (after rebuilding the binaries), see [gen-manpages.py](https://github.com/bitcoin/bitcoin/blob/master/contrib/devtools/README.md#gen-manpagespy). +* Update bitcoin.conf and commit, see [gen-bitcoin-conf.sh](https://github.com/bitcoin/bitcoin/blob/master/contrib/devtools/README.md#gen-bitcoin-confsh). ### Before every major and minor release * Update [bips.md](bips.md) to account for changes since the last release (don't forget to bump the version number on the first line). * Update version in `configure.ac` (don't forget to set `CLIENT_VERSION_RC` to `0`). +* Update manpages (see previous section) * Write release notes (see "Write the release notes" below). ### Before every major release * On both the master branch and the new release branch: - update `CLIENT_VERSION_MAJOR` in [`configure.ac`](../configure.ac) - - update `CLIENT_VERSION_MAJOR`, `PACKAGE_VERSION`, and `PACKAGE_STRING` in [`build_msvc/bitcoin_config.h`](/build_msvc/bitcoin_config.h) -* On the new release branch in [`configure.ac`](../configure.ac) and [`build_msvc/bitcoin_config.h`](/build_msvc/bitcoin_config.h) (see [this commit](https://github.com/bitcoin/bitcoin/commit/742f7dd)): +* On the new release branch in [`configure.ac`](../configure.ac)(see [this commit](https://github.com/bitcoin/bitcoin/commit/742f7dd)): - set `CLIENT_VERSION_MINOR` to `0` - set `CLIENT_VERSION_BUILD` to `0` - set `CLIENT_VERSION_IS_RELEASE` to `true` @@ -28,16 +29,23 @@ Release Process #### Before branch-off * Update hardcoded [seeds](/contrib/seeds/README.md), see [this pull request](https://github.com/bitcoin/bitcoin/pull/7415) for an example. -* Update [`src/chainparams.cpp`](/src/chainparams.cpp) m_assumed_blockchain_size and m_assumed_chain_state_size with the current size plus some overhead (see [this](#how-to-calculate-assumed-blockchain-and-chain-state-size) for information on how to calculate them). -* Update [`src/chainparams.cpp`](/src/chainparams.cpp) chainTxData with statistics about the transaction count and rate. Use the output of the `getchaintxstats` RPC, see - [this pull request](https://github.com/bitcoin/bitcoin/pull/20263) for an example. Reviewers can verify the results by running `getchaintxstats <window_block_count> <window_final_block_hash>` with the `window_block_count` and `window_final_block_hash` from your output. -* Update `src/chainparams.cpp` nMinimumChainWork and defaultAssumeValid (and the block height comment) with information from the `getblockheader` (and `getblockhash`) RPCs. - - The selected value must not be orphaned so it may be useful to set the value two blocks back from the tip. - - Testnet should be set some tens of thousands back from the tip due to reorgs there. - - This update should be reviewed with a reindex-chainstate with assumevalid=0 to catch any defect - that causes rejection of blocks in the past history. +* Update the following variables in [`src/chainparams.cpp`](/src/chainparams.cpp) for mainnet, testnet, and signet: + - `m_assumed_blockchain_size` and `m_assumed_chain_state_size` with the current size plus some overhead (see + [this](#how-to-calculate-assumed-blockchain-and-chain-state-size) for information on how to calculate them). + - The following updates should be reviewed with `reindex-chainstate` and `assumevalid=0` to catch any defect + that causes rejection of blocks in the past history. + - `chainTxData` with statistics about the transaction count and rate. Use the output of the `getchaintxstats` RPC with an + `nBlocks` of 4096 (28 days) and a `bestblockhash` of RPC `getbestblockhash`; see + [this pull request](https://github.com/bitcoin/bitcoin/pull/20263) for an example. Reviewers can verify the results by running + `getchaintxstats <window_block_count> <window_final_block_hash>` with the `window_block_count` and `window_final_block_hash` from your output. + - `defaultAssumeValid` with the output of RPC `getblockhash` using the `height` of `window_final_block_height` above + (and update the block height comment with that height), taking into account the following: + - On mainnet, the selected value must not be orphaned, so it may be useful to set the height two blocks back from the tip. + - Testnet should be set with a height some tens of thousands back from the tip, due to reorgs there. + - `nMinimumChainWork` with the "chainwork" value of RPC `getblockheader` using the same height as that selected for the previous step. - Clear the release notes and move them to the wiki (see "Write the release notes" below). -- Translations on Transifex +- Translations on Transifex: + - Pull translations from Transifex into the master branch. - Create [a new resource](https://www.transifex.com/bitcoin/bitcoin/content/) named after the major version with the slug `[bitcoin.qt-translation-<RRR>x]`, where `RRR` is the major branch number padded with zeros. Use `src/qt/locale/bitcoin_en.xlf` to create it. - In the project workflow settings, ensure that [Translation Memory Fill-up](https://docs.transifex.com/translation-memory/enabling-autofill) is enabled and that [Translation Memory Context Matching](https://docs.transifex.com/translation-memory/translation-memory-with-context) is disabled. - Update the Transifex slug in [`.tx/config`](/.tx/config) to the slug of the resource created in the first step. This identifies which resource the translations will be synchronized from. @@ -47,20 +55,22 @@ Release Process #### After branch-off (on the major release branch) - Update the versions. +- Create the draft, named "*version* Release Notes Draft", as a [collaborative wiki](https://github.com/bitcoin-core/bitcoin-devwiki/wiki/_new). +- Clear the release notes: `cp doc/release-notes-empty-template.md doc/release-notes.md` - Create a pinned meta-issue for testing the release candidate (see [this issue](https://github.com/bitcoin/bitcoin/issues/17079) for an example) and provide a link to it in the release announcements where useful. - Translations on Transifex - Change the auto-update URL for the new major version's resource away from `master` and to the branch, e.g. `https://raw.githubusercontent.com/bitcoin/bitcoin/<branch>/src/qt/locale/bitcoin_en.xlf`. Do not forget this or it will keep tracking the translations on master instead, drifting away from the specific major release. #### Before final release -- Merge the release notes from the wiki into the branch. +- Merge the release notes from [the wiki](https://github.com/bitcoin-core/bitcoin-devwiki/wiki/) into the branch. - Ensure the "Needs release note" label is removed from all relevant pull requests and issues. #### Tagging a release (candidate) To tag the version (or release candidate) in git, use the `make-tag.py` script from [bitcoin-maintainer-tools](https://github.com/bitcoin-core/bitcoin-maintainer-tools). From the root of the repository run: - ../bitcoin-maintainer-tools/make-tag.py v(new version, e.g. 0.20.0) + ../bitcoin-maintainer-tools/make-tag.py v(new version, e.g. 23.0) This will perform a few last-minute consistency checks in the build system files, and if they pass, create a signed tag. @@ -98,7 +108,7 @@ Checkout the Bitcoin Core version you'd like to build: pushd ./bitcoin SIGNER='(your builder key, ie bluematt, sipa, etc)' VERSION='(new version without v-prefix, e.g. 0.20.0)' -git fetch "v${VERSION}" +git fetch origin "v${VERSION}" git checkout "v${VERSION}" popd ``` @@ -110,28 +120,24 @@ against other `guix-attest` signatures. git -C ./guix.sigs pull ``` -### Create the macOS SDK tarball: (first time, or when SDK version changes) +### Create the macOS SDK tarball (first time, or when SDK version changes) Create the macOS SDK tarball, see the [macdeploy instructions](/contrib/macdeploy/README.md#deterministic-macos-dmg-notes) for details. -### Build and attest to build outputs: +### Build and attest to build outputs Follow the relevant Guix README.md sections: -- [Performing a build](/contrib/guix/README.md#performing-a-build) +- [Building](/contrib/guix/README.md#building) - [Attesting to build outputs](/contrib/guix/README.md#attesting-to-build-outputs) -### Verify other builders' signatures to your own. (Optional) - -Add other builders keys to your gpg keyring, and/or refresh keys: See `../bitcoin/contrib/builder-keys/README.md`. +### Verify other builders' signatures to your own (optional) -Follow the relevant Guix README.md sections: +- [Add other builders keys to your gpg keyring, and/or refresh keys](/contrib/builder-keys/README.md) - [Verifying build output attestations](/contrib/guix/README.md#verifying-build-output-attestations) -### Next steps: - -Commit your signature to guix.sigs: +### Commit your non codesigned signature to guix.sigs ```sh pushd ./guix.sigs @@ -141,29 +147,27 @@ git push # Assuming you can push to the guix.sigs tree popd ``` -Codesigner only: Create Windows/macOS detached signatures: -- Only one person handles codesigning. Everyone else should skip to the next step. -- Only once the Windows/macOS builds each have 3 matching signatures may they be signed with their respective release keys. +## Codesigning -Codesigner only: Sign the macOS binary: +### macOS codesigner only: Create detached macOS signatures (assuming [signapple](https://github.com/achow101/signapple/) is installed and up to date with master branch) - transfer bitcoin-osx-unsigned.tar.gz to macOS for signing tar xf bitcoin-osx-unsigned.tar.gz - ./detached-sig-create.sh -s "Key ID" + ./detached-sig-create.sh /path/to/codesign.p12 Enter the keychain password and authorize the signature - Move signature-osx.tar.gz back to the guix-build host + signature-osx.tar.gz will be created -Codesigner only: Sign the windows binaries: +### Windows codesigner only: Create detached Windows signatures tar xf bitcoin-win-unsigned.tar.gz ./detached-sig-create.sh -key /path/to/codesign.key Enter the passphrase for the key when prompted signature-win.tar.gz will be created -Code-signer only: It is advised to test that the code signature attaches properly prior to tagging by performing the `guix-codesign` step. +### Windows and macOS codesigners only: test code signatures +It is advised to test that the code signature attaches properly prior to tagging by performing the `guix-codesign` step. However if this is done, once the release has been tagged in the bitcoin-detached-sigs repo, the `guix-codesign` step must be performed again in order for the guix attestation to be valid when compared against the attestations of non-codesigner builds. -Codesigner only: Commit the detached codesign payloads: +### Windows and macOS codesigners only: Commit the detached codesign payloads ```sh pushd ./bitcoin-detached-sigs @@ -178,16 +182,21 @@ git push the current branch and new tag popd ``` -Non-codesigners: wait for Windows/macOS detached signatures: +### Non-codesigners: wait for Windows and macOS detached signatures -- Once the Windows/macOS builds each have 3 matching signatures, they will be signed with their respective release keys. +- Once the Windows and macOS builds each have 3 matching signatures, they will be signed with their respective release keys. - Detached signatures will then be committed to the [bitcoin-detached-sigs](https://github.com/bitcoin-core/bitcoin-detached-sigs) repository, which can be combined with the unsigned apps to create signed binaries. -Create (and optionally verify) the codesigned outputs: +### Create the codesigned build outputs -- [Codesigning](/contrib/guix/README.md#codesigning) +- [Codesigning build outputs](/contrib/guix/README.md#codesigning-build-outputs) + +### Verify other builders' signatures to your own (optional) + +- [Add other builders keys to your gpg keyring, and/or refresh keys](/contrib/builder-keys/README.md) +- [Verifying build output attestations](/contrib/guix/README.md#verifying-build-output-attestations) -Commit your signature for the signed macOS/Windows binaries: +### Commit your codesigned signature to guix.sigs (for the signed macOS/Windows binaries) ```sh pushd ./guix.sigs @@ -197,7 +206,7 @@ git push # Assuming you can push to the guix.sigs tree popd ``` -### After 3 or more people have guix-built and their results match: +## After 3 or more people have guix-built and their results match Combine the `all.SHA256SUMS.asc` file from all signers into `SHA256SUMS.asc`: @@ -253,6 +262,10 @@ cat "$VERSION"/*/all.SHA256SUMS.asc > SHA256SUMS.asc - bitcoincore.org maintained versions update: [table](https://github.com/bitcoin-core/bitcoincore.org/commits/master/_includes/posts/maintenance-table.md) + - Delete post-EOL [release branches](https://github.com/bitcoin/bitcoin/branches/all) and create a tag `v${branch_name}-final`. + + - Delete ["Needs backport" labels](https://github.com/bitcoin/bitcoin/labels?q=backport) for non-existing branches. + - bitcoincore.org RPC documentation update - Install [golang](https://golang.org/doc/install) @@ -271,26 +284,7 @@ cat "$VERSION"/*/all.SHA256SUMS.asc > SHA256SUMS.asc - Push the flatpak to flathub, e.g. https://github.com/flathub/org.bitcoincore.bitcoin-qt/pull/2 - - Push the latest version to master (if applicable), e.g. https://github.com/bitcoin-core/packaging/pull/32 - - - Create a new branch for the major release "0.xx" from master (used to build the snap package) and request the - track (if applicable), e.g. https://forum.snapcraft.io/t/track-request-for-bitcoin-core-snap/10112/7 - - - Notify MarcoFalke so that he can start building the snap package - - - https://code.launchpad.net/~bitcoin-core/bitcoin-core-snap/+git/packaging (Click "Import Now" to fetch the branch) - - https://code.launchpad.net/~bitcoin-core/bitcoin-core-snap/+git/packaging/+ref/0.xx (Click "Create snap package") - - Name it "bitcoin-core-snap-0.xx" - - Leave owner and series as-is - - Select architectures that are compiled via guix - - Leave "automatically build when branch changes" unticked - - Tick "automatically upload to store" - - Put "bitcoin-core" in the registered store package name field - - Tick the "edge" box - - Put "0.xx" in the track field - - Click "create snap package" - - Click "Request builds" for every new release on this branch (after updating the snapcraft.yml in the branch to reflect the latest guix results) - - Promote release on https://snapcraft.io/bitcoin-core/releases if it passes sanity checks + - Push the snap, see https://github.com/bitcoin-core/packaging/blob/master/snap/build.md - This repo @@ -315,15 +309,16 @@ cat "$VERSION"/*/all.SHA256SUMS.asc > SHA256SUMS.asc Both variables are used as a guideline for how much space the user needs on their drive in total, not just strictly for the blockchain. Note that all values should be taken from a **fully synced** node and have an overhead of 5-10% added on top of its base value. -To calculate `m_assumed_blockchain_size`: -- For `mainnet` -> Take the size of the data directory, excluding `/regtest` and `/testnet3` directories. -- For `testnet` -> Take the size of the `/testnet3` directory. - +To calculate `m_assumed_blockchain_size`, take the size in GiB of these directories: +- For `mainnet` -> the data directory, excluding the `/testnet3`, `/signet`, and `/regtest` directories and any overly large files, e.g. a huge `debug.log` +- For `testnet` -> `/testnet3` +- For `signet` -> `/signet` -To calculate `m_assumed_chain_state_size`: -- For `mainnet` -> Take the size of the `/chainstate` directory. -- For `testnet` -> Take the size of the `/testnet3/chainstate` directory. +To calculate `m_assumed_chain_state_size`, take the size in GiB of these directories: +- For `mainnet` -> `/chainstate` +- For `testnet` -> `/testnet3/chainstate` +- For `signet` -> `/signet/chainstate` Notes: - When taking the size for `m_assumed_blockchain_size`, there's no need to exclude the `/chainstate` directory since it's a guideline value and an overhead will be added anyway. -- The expected overhead for growth may change over time, so it may not be the same value as last release; pay attention to that when changing the variables. +- The expected overhead for growth may change over time. Consider whether the percentage needs to be changed in response; if so, update it here in this section. diff --git a/doc/tor.md b/doc/tor.md index a3ec1987aa..08d031d084 100644 --- a/doc/tor.md +++ b/doc/tor.md @@ -16,17 +16,19 @@ configure Tor. ## How to see information about your Tor configuration via Bitcoin Core There are several ways to see your local onion address in Bitcoin Core: -- in the debug log (grep for "tor:" or "AddLocal") -- in the output of RPC `getnetworkinfo` in the "localaddresses" section -- in the output of the CLI `-netinfo` peer connections dashboard +- in the "Local addresses" output of CLI `-netinfo` +- in the "localaddresses" output of RPC `getnetworkinfo` +- in the debug log (grep for "AddLocal"; the Tor address ends in `.onion`) You may set the `-debug=tor` config logging option to have additional information in the debug log about your Tor configuration. -CLI `-addrinfo` returns the number of addresses known to your node per network -type, including Tor v2 and v3. This is useful to see how many onion addresses -are known to your node for `-onlynet=onion` and how many Tor v3 addresses it -knows when upgrading to Bitcoin Core v22.0 and up that supports Tor v3 only. +CLI `-addrinfo` returns the number of addresses known to your node per +network. This can be useful to see how many onion peers your node knows, +e.g. for `-onlynet=onion`. + +To fetch a number of onion addresses that your node knows, for example seven +addresses, use the `getnodeaddresses 7 onion` RPC. ## 1. Run Bitcoin Core behind a Tor proxy @@ -41,9 +43,11 @@ outgoing connections, but more is possible. -onion=ip:port Set the proxy server to use for Tor onion services. You do not need to set this if it's the same as -proxy. You can use -onion=0 to explicitly disable access to onion services. + ------------------------------------------------------------------ Note: Only the -proxy option sets the proxy for DNS requests; with -onion they will not route over Tor, so use -proxy if you have privacy concerns. + ------------------------------------------------------------------ -listen When using -proxy, listening is disabled by default. If you want to manually configure an onion service (see section 3), you'll @@ -54,14 +58,10 @@ outgoing connections, but more is possible. -seednode=X SOCKS5. In Tor mode, such addresses can also be exchanged with other P2P nodes. - -onlynet=onion Make outgoing connections only to .onion addresses. Incoming - connections are not affected by this option. This option can be - specified multiple times to allow multiple network types, e.g. - onlynet=ipv4, onlynet=ipv6, onlynet=onion, onlynet=i2p. - Warning: if you use -onlynet with values other than onion, and - the -onion or -proxy option is set, then outgoing onion - connections will still be made; use -noonion or -onion=0 to - disable outbound onion connections in this case. + -onlynet=onion Make automatic outbound connections only to .onion addresses. + Inbound and manual connections are not affected by this option. + It can be specified multiple times to allow multiple networks, + e.g. onlynet=onion, onlynet=i2p, onlynet=cjdns. In a typical situation, this suffices to run behind a Tor proxy: @@ -134,7 +134,7 @@ You can also check the group of the cookie file. On most Linux systems, the Tor auth cookie will usually be `/run/tor/control.authcookie`: ``` -stat -c '%G' /run/tor/control.authcookie +TORGROUP=$(stat -c '%G' /run/tor/control.authcookie) ``` Once you have determined the `${TORGROUP}` and selected the `${USER}` that will diff --git a/doc/tracing.md b/doc/tracing.md index 87fc9603fe..b6e3b9263a 100644 --- a/doc/tracing.md +++ b/doc/tracing.md @@ -101,19 +101,115 @@ Is called *after* a block is connected to the chain. Can, for example, be used to benchmark block connections together with `-reindex`. Arguments passed: -1. Block Header Hash as `pointer to C-style String` (64 characters) +1. Block Header Hash as `pointer to unsigned chars` (i.e. 32 bytes in little-endian) 2. Block Height as `int32` 3. Transactions in the Block as `uint64` 4. Inputs spend in the Block as `int32` 5. SigOps in the Block (excluding coinbase SigOps) `uint64` 6. Time it took to connect the Block in microseconds (µs) as `uint64` -7. Block Header Hash as `pointer to unsigned chars` (i.e. 32 bytes in little-endian) -Note: The 7th argument can't be accessed by bpftrace and is purposefully chosen -to be the block header hash as bytes. See [bpftrace argument limit] for more -details. +### Context `utxocache` -[bpftrace argument limit]: #bpftrace-argument-limit +The following tracepoints cover the in-memory UTXO cache. UTXOs are, for example, +added to and removed (spent) from the cache when we connect a new block. +**Note**: Bitcoin Core uses temporary clones of the _main_ UTXO cache +(`chainstate.CoinsTip()`). For example, the RPCs `generateblock` and +`getblocktemplate` call `TestBlockValidity()`, which applies the UTXO set +changes to a temporary cache. Similarly, mempool consistency checks, which are +frequent on regtest, also apply the the UTXO set changes to a temporary cache. +Changes to the _main_ UTXO cache and to temporary caches trigger the tracepoints. +We can't tell if a temporary cache or the _main_ cache was changed. + +#### Tracepoint `utxocache:flush` + +Is called *after* the in-memory UTXO cache is flushed. + +Arguments passed: +1. Time it took to flush the cache microseconds as `int64` +2. Flush state mode as `uint32`. It's an enumerator class with values `0` + (`NONE`), `1` (`IF_NEEDED`), `2` (`PERIODIC`), `3` (`ALWAYS`) +3. Cache size (number of coins) before the flush as `uint64` +4. Cache memory usage in bytes as `uint64` +5. If pruning caused the flush as `bool` + +#### Tracepoint `utxocache:add` + +Is called when a coin is added to a UTXO cache. This can be a temporary UTXO cache too. + +Arguments passed: +1. Transaction ID (hash) as `pointer to unsigned chars` (i.e. 32 bytes in little-endian) +2. Output index as `uint32` +3. Block height the coin was added to the UTXO-set as `uint32` +4. Value of the coin as `int64` +5. If the coin is a coinbase as `bool` + +#### Tracepoint `utxocache:spent` + +Is called when a coin is spent from a UTXO cache. This can be a temporary UTXO cache too. + +Arguments passed: +1. Transaction ID (hash) as `pointer to unsigned chars` (i.e. 32 bytes in little-endian) +2. Output index as `uint32` +3. Block height the coin was spent, as `uint32` +4. Value of the coin as `int64` +5. If the coin is a coinbase as `bool` + +#### Tracepoint `utxocache:uncache` + +Is called when a coin is purposefully unloaded from a UTXO cache. This +happens, for example, when we load an UTXO into a cache when trying to accept +a transaction that turns out to be invalid. The loaded UTXO is uncached to avoid +filling our UTXO cache up with irrelevant UTXOs. + +Arguments passed: +1. Transaction ID (hash) as `pointer to unsigned chars` (i.e. 32 bytes in little-endian) +2. Output index as `uint32` +3. Block height the coin was uncached, as `uint32` +4. Value of the coin as `int64` +5. If the coin is a coinbase as `bool` + +### Context `coin_selection` + +#### Tracepoint `coin_selection:selected_coins` + +Is called when `SelectCoins` completes. + +Arguments passed: +1. Wallet name as `pointer to C-style string` +2. Coin selection algorithm name as `pointer to C-style string` +3. Selection target value as `int64` +4. Calculated waste metric of the solution as `int64` +5. Total value of the selected inputs as `int64` + +#### Tracepoint `coin_selection:normal_create_tx_internal` + +Is called when the first `CreateTransactionInternal` completes. + +Arguments passed: +1. Wallet name as `pointer to C-style string` +2. Whether `CreateTransactionInternal` succeeded as `bool` +3. The expected transaction fee as an `int64` +4. The position of the change output as an `int32` + +#### Tracepoint `coin_selection:attempting_aps_create_tx` + +Is called when `CreateTransactionInternal` is called the second time for the optimistic +Avoid Partial Spends selection attempt. This is used to determine whether the next +tracepoints called are for the Avoid Partial Spends solution, or a different transaction. + +Arguments passed: +1. Wallet name as `pointer to C-style string` + +#### Tracepoint `coin_selection:aps_create_tx_internal` + +Is called when the second `CreateTransactionInternal` with Avoid Partial Spends enabled completes. + +Arguments passed: +1. Wallet name as `pointer to C-style string` +2. Whether the Avoid Partial Spends solution will be used as `bool` +3. Whether `CreateTransactionInternal` succeeded as` bool` +4. The expected transaction fee as an `int64` +5. The position of the change output as an `int32` ## Adding tracepoints to Bitcoin Core diff --git a/doc/zmq.md b/doc/zmq.md index 85f3370130..4055505d74 100644 --- a/doc/zmq.md +++ b/doc/zmq.md @@ -76,7 +76,7 @@ The option to set the PUB socket's outbound message high water mark -zmqpubhashblockhwm=n -zmqpubrawblockhwm=n -zmqpubrawtxhwm=n - -zmqpubsequencehwm=address + -zmqpubsequencehwm=n The high water mark value must be an integer greater than or equal to 0. @@ -84,15 +84,18 @@ For instance: $ bitcoind -zmqpubhashtx=tcp://127.0.0.1:28332 \ -zmqpubhashtx=tcp://192.168.1.2:28332 \ + -zmqpubhashblock="tcp://[::1]:28333" \ -zmqpubrawtx=ipc:///tmp/bitcoind.tx.raw \ -zmqpubhashtxhwm=10000 Each PUB notification has a topic and body, where the header corresponds to the notification type. For instance, for the notification `-zmqpubhashtx` the topic is `hashtx` (no null -terminator) and the body is the transaction hash (32 -bytes) for all but `sequence` topic. For `sequence`, the body -is structured as the following based on the type of message: +terminator). These options can also be provided in bitcoin.conf. + +The topics are: + +`sequence`: the body is structured as the following based on the type of message: <32-byte hash>C : Blockhash connected <32-byte hash>D : Blockhash disconnected @@ -101,7 +104,24 @@ is structured as the following based on the type of message: Where the 8-byte uints correspond to the mempool sequence number. -These options can also be provided in bitcoin.conf. +`rawtx`: Notifies about all transactions, both when they are added to mempool or when a new block arrives. This means a transaction could be published multiple times. First, when it enters the mempool and then again in each block that includes it. The messages are ZMQ multipart messages with three parts. The first part is the topic (`rawtx`), the second part is the serialized transaction, and the last part is a sequence number (representing the message count to detect lost messages). + + | rawtx | <serialized transaction> | <uint32 sequence number in Little Endian> + +`hashtx`: Notifies about all transactions, both when they are added to mempool or when a new block arrives. This means a transaction could be published multiple times. First, when it enters the mempool and then again in each block that includes it. The messages are ZMQ multipart messages with three parts. The first part is the topic (`hashtx`), the second part is the 32-byte transaction hash, and the last part is a sequence number (representing the message count to detect lost messages). + + | hashtx | <32-byte transaction hash in Little Endian> | <uint32 sequence number in Little Endian> + + +`rawblock`: Notifies when the chain tip is updated. Messages are ZMQ multipart messages with three parts. The first part is the topic (`rawblock`), the second part is the serialized block, and the last part is a sequence number (representing the message count to detect lost messages). + + | rawblock | <serialized block> | <uint32 sequence number in Little Endian> + +`hashblock`: Notifies when the chain tip is updated. Messages are ZMQ multipart messages with three parts. The first part is the topic (`hashblock`), the second part is the 32-byte block hash, and the last part is a sequence number (representing the message count to detect lost messages). + + | hashblock | <32-byte block hash in Little Endian> | <uint32 sequence number in Little Endian> + +**_NOTE:_** Note that the 32-byte hashes are in Little Endian and not in the Big Endian format that the RPC interface and block explorers use to display transaction and block hashes. ZeroMQ endpoint specifiers for TCP (and others) are documented in the [ZeroMQ API](http://api.zeromq.org/4-0:_start). @@ -125,6 +145,9 @@ Setting the keepalive values appropriately for your operating environment may improve connectivity in situations where long-lived connections are silently dropped by network middle boxes. +Also, the socket's ZMQ_IPV6 option is enabled to accept connections from IPv6 +hosts as well. If needed, this option has to be set on the client side too. + ## Remarks From the perspective of bitcoind, the ZeroMQ socket is write-only; PUB |