diff options
Diffstat (limited to 'doc')
| -rw-r--r-- | doc/README.md | 3 | ||||
| -rw-r--r-- | doc/REST-interface.md | 14 | ||||
| -rw-r--r-- | doc/assumeutxo.md | 138 | ||||
| -rw-r--r-- | doc/bips.md | 10 | ||||
| -rw-r--r-- | doc/build-android.md | 17 | ||||
| -rw-r--r-- | doc/build-unix.md | 56 | ||||
| -rw-r--r-- | doc/dependencies.md | 7 | ||||
| -rw-r--r-- | doc/developer-notes.md | 9 | ||||
| -rw-r--r-- | doc/fuzzing.md | 4 | ||||
| -rw-r--r-- | doc/multisig-tutorial.md | 241 | ||||
| -rw-r--r-- | doc/policy/README.md | 15 | ||||
| -rw-r--r-- | doc/policy/mempool-limits.md | 65 | ||||
| -rw-r--r-- | doc/policy/mempool-replacements.md | 69 | ||||
| -rw-r--r-- | doc/policy/packages.md | 59 | ||||
| -rw-r--r-- | doc/release-notes-14707.md | 19 | ||||
| -rw-r--r-- | doc/release-notes-23093.md | 11 | ||||
| -rw-r--r-- | doc/release-notes-23113.md | 9 | ||||
| -rw-r--r-- | doc/release-notes-gui-459.md | 4 | ||||
| -rw-r--r-- | doc/release-notes.md | 28 | ||||
| -rw-r--r-- | doc/release-process.md | 9 | ||||
| -rw-r--r-- | doc/tracing.md | 49 | ||||
| -rw-r--r-- | doc/zmq.md | 27 |
22 files changed, 803 insertions, 60 deletions
diff --git a/doc/README.md b/doc/README.md index aabfe220bc..2800937d30 100644 --- a/doc/README.md +++ b/doc/README.md @@ -71,16 +71,19 @@ The Bitcoin repo's [root README](/README.md) contains relevant information on th ### Miscellaneous - [Assets Attribution](assets-attribution.md) +- [Assumeutxo design](assumeutxo.md) - [bitcoin.conf Configuration File](bitcoin-conf.md) - [Files](files.md) - [Fuzz-testing](fuzzing.md) - [I2P Support](i2p.md) - [Init Scripts (systemd/upstart/openrc)](init.md) - [Managing Wallets](managing-wallets.md) +- [Multisig Tutorial](multisig-tutorial.md) - [PSBT support](psbt.md) - [Reduce Memory](reduce-memory.md) - [Reduce Traffic](reduce-traffic.md) - [Tor Support](tor.md) +- [Transaction Relay Policy](policy/README.md) - [ZMQ](zmq.md) License diff --git a/doc/REST-interface.md b/doc/REST-interface.md index 8b281acca7..51a73b89fc 100644 --- a/doc/REST-interface.md +++ b/doc/REST-interface.md @@ -52,6 +52,20 @@ With the /notxdetails/ option JSON response will only contain the transaction ha Given a block hash: returns <COUNT> amount of blockheaders in upward direction. Returns empty if the block doesn't exist or it isn't in the active chain. +#### Blockfilter Headers +`GET /rest/blockfilterheaders/<FILTERTYPE>/<COUNT>/<BLOCK-HASH>.<bin|hex|json>` + +Given a block hash: returns <COUNT> amount of blockfilter headers in upward +direction for the filter type <FILTERTYPE>. +Returns empty if the block doesn't exist or it isn't in the active chain. + +#### Blockfilters +`GET /rest/blockfilter/<FILTERTYPE>/<BLOCK-HASH>.<bin|hex|json>` + +Given a block hash: returns the block filter of the given block of type +<FILTERTYPE>. +Responds with 404 if the block doesn't exist. + #### Blockhash by height `GET /rest/blockhashbyheight/<HEIGHT>.<bin|hex|json>` diff --git a/doc/assumeutxo.md b/doc/assumeutxo.md new file mode 100644 index 0000000000..2726cf779b --- /dev/null +++ b/doc/assumeutxo.md @@ -0,0 +1,138 @@ +# assumeutxo + +Assumeutxo is a feature that allows fast bootstrapping of a validating bitcoind +instance with a very similar security model to assumevalid. + +The RPC commands `dumptxoutset` and `loadtxoutset` are used to respectively generate +and load UTXO snapshots. The utility script `./contrib/devtools/utxo_snapshot.sh` may +be of use. + +## General background + +- [assumeutxo proposal](https://github.com/jamesob/assumeutxo-docs/tree/2019-04-proposal/proposal) +- [Github issue](https://github.com/bitcoin/bitcoin/issues/15605) +- [draft PR](https://github.com/bitcoin/bitcoin/pull/15606) + +## Design notes + +- A new block index `nStatus` flag is introduced, `BLOCK_ASSUMED_VALID`, to mark block + index entries that are required to be assumed-valid by a chainstate created + from a UTXO snapshot. This flag is mostly used as a way to modify certain + CheckBlockIndex() logic to account for index entries that are pending validation by a + chainstate running asynchronously in the background. We also use this flag to control + which index entries are added to setBlockIndexCandidates during LoadBlockIndex(). + +- Indexing implementations via BaseIndex can no longer assume that indexation happens + sequentially, since background validation chainstates can submit BlockConnected + events out of order with the active chain. + +- The concept of UTXO snapshots is treated as an implementation detail that lives + behind the ChainstateManager interface. The external presentation of the changes + required to facilitate the use of UTXO snapshots is the understanding that there are + now certain regions of the chain that can be temporarily assumed to be valid (using + the nStatus flag mentioned above). In certain cases, e.g. wallet rescanning, this is + very similar to dealing with a pruned chain. + + Logic outside ChainstateManager should try not to know about snapshots, instead + preferring to work in terms of more general states like assumed-valid. + + +## Chainstate phases + +Chainstate within the system goes through a number of phases when UTXO snapshots are +used, as managed by `ChainstateManager`. At various points there can be multiple +`CChainState` objects in existence to facilitate both maintaining the network tip and +performing historical validation of the assumed-valid chain. + +It is worth noting that though there are multiple separate chainstates, those +chainstates share use of a common block index (i.e. they hold the same `BlockManager` +reference). + +The subheadings below outline the phases and the corresponding changes to chainstate +data. + +### "Normal" operation via initial block download + +`ChainstateManager` manages a single CChainState object, for which +`m_snapshot_blockhash` is null. This chainstate is (maybe obviously) +considered active. This is the "traditional" mode of operation for bitcoind. + +| | | +| ---------- | ----------- | +| number of chainstates | 1 | +| active chainstate | ibd | + +### User loads a UTXO snapshot via `loadtxoutset` RPC + +`ChainstateManager` initializes a new chainstate (see `ActivateSnapshot()`) to load the +snapshot contents into. During snapshot load and validation (see +`PopulateAndValidateSnapshot()`), the new chainstate is not considered active and the +original chainstate remains in use as active. + +| | | +| ---------- | ----------- | +| number of chainstates | 2 | +| active chainstate | ibd | + +Once the snapshot chainstate is loaded and validated, it is promoted to active +chainstate and a sync to tip begins. A new chainstate directory is created in the +datadir for the snapshot chainstate called +`chainstate_[SHA256 blockhash of snapshot base block]`. + +| | | +| ---------- | ----------- | +| number of chainstates | 2 | +| active chainstate | snapshot | + +The snapshot begins to sync to tip from its base block, technically in parallel with +the original chainstate, but it is given priority during block download and is +allocated most of the cache (see `MaybeRebalanceCaches()` and usages) as our chief +consideration is getting to network tip. + +**Failure consideration:** if shutdown happens at any point during this phase, both +chainstates will be detected during the next init and the process will resume. + +### Snapshot chainstate hits network tip + +Once the snapshot chainstate leaves IBD, caches are rebalanced +(via `MaybeRebalanceCaches()` in `ActivateBestChain()`) and more cache is given +to the background chainstate, which is responsible for doing full validation of the +assumed-valid parts of the chain. + +**Note:** at this point, ValidationInterface callbacks will be coming in from both +chainstates. Considerations here must be made for indexing, which may no longer be happening +sequentially. + +### Background chainstate hits snapshot base block + +Once the tip of the background chainstate hits the base block of the snapshot +chainstate, we stop use of the background chainstate by setting `m_stop_use` (not yet +committed - see #15606), in `CompleteSnapshotValidation()`, which is checked in +`ActivateBestChain()`). We hash the background chainstate's UTXO set contents and +ensure it matches the compiled value in `CMainParams::m_assumeutxo_data`. + +The background chainstate data lingers on disk until shutdown, when in +`ChainstateManager::Reset()`, the background chainstate is cleaned up with +`ValidatedSnapshotShutdownCleanup()`, which renames the `chainstate_[hash]` datadir as +`chainstate`. + +| | | +| ---------- | ----------- | +| number of chainstates | 2 (ibd has `m_stop_use=true`) | +| active chainstate | snapshot | + +**Failure consideration:** if bitcoind unexpectedly halts after `m_stop_use` is set on +the background chainstate but before `CompleteSnapshotValidation()` can finish, the +need to complete snapshot validation will be detected on subsequent init by +`ChainstateManager::CheckForUncleanShutdown()`. + +### Bitcoind restarts sometime after snapshot validation has completed + +When bitcoind initializes again, what began as the snapshot chainstate is now +indistinguishable from a chainstate that has been built from the traditional IBD +process, and will be initialized as such. + +| | | +| ---------- | ----------- | +| number of chainstates | 1 | +| active chainstate | ibd | diff --git a/doc/bips.md b/doc/bips.md index 45954bcfd8..27adcedd31 100644 --- a/doc/bips.md +++ b/doc/bips.md @@ -32,7 +32,7 @@ BIPs that are implemented by Bitcoin Core (up-to-date up to **v22.0**): * [`BIP 111`](https://github.com/bitcoin/bips/blob/master/bip-0111.mediawiki): `NODE_BLOOM` service bit added, and enforced for all peer versions as of **v0.13.0** ([PR #6579](https://github.com/bitcoin/bitcoin/pull/6579) and [PR #6641](https://github.com/bitcoin/bitcoin/pull/6641)). * [`BIP 112`](https://github.com/bitcoin/bips/blob/master/bip-0112.mediawiki): The CHECKSEQUENCEVERIFY opcode has been implemented since **v0.12.1** ([PR #7524](https://github.com/bitcoin/bitcoin/pull/7524)), and has been *buried* since **v0.19.0** ([PR #16060](https://github.com/bitcoin/bitcoin/pull/16060)). * [`BIP 113`](https://github.com/bitcoin/bips/blob/master/bip-0113.mediawiki): Median time past lock-time calculations have been implemented since **v0.12.1** ([PR #6566](https://github.com/bitcoin/bitcoin/pull/6566)), and has been *buried* since **v0.19.0** ([PR #16060](https://github.com/bitcoin/bitcoin/pull/16060)). -* [`BIP 125`](https://github.com/bitcoin/bips/blob/master/bip-0125.mediawiki): Opt-in full replace-by-fee signaling honoured in mempool and mining as of **v0.12.0** ([PR 6871](https://github.com/bitcoin/bitcoin/pull/6871)). Enabled by default in the wallet GUI as of **v0.18.1** ([PR #11605](https://github.com/bitcoin/bitcoin/pull/11605)) +* [`BIP 125`](https://github.com/bitcoin/bips/blob/master/bip-0125.mediawiki): Opt-in full replace-by-fee signaling partially implemented. See doc/policy/mempool-replacements.md. * [`BIP 130`](https://github.com/bitcoin/bips/blob/master/bip-0130.mediawiki): direct headers announcement is negotiated with peer versions `>=70012` as of **v0.12.0** ([PR 6494](https://github.com/bitcoin/bitcoin/pull/6494)). * [`BIP 133`](https://github.com/bitcoin/bips/blob/master/bip-0133.mediawiki): feefilter messages are respected and sent for peer versions `>=70013` as of **v0.13.0** ([PR 7542](https://github.com/bitcoin/bitcoin/pull/7542)). * [`BIP 141`](https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki): Segregated Witness (Consensus Layer) as of **v0.13.0** ([PR 8149](https://github.com/bitcoin/bitcoin/pull/8149)), defined for mainnet as of **v0.13.1** ([PR 8937](https://github.com/bitcoin/bitcoin/pull/8937)), and *buried* since **v0.19.0** ([PR #16060](https://github.com/bitcoin/bitcoin/pull/16060)). @@ -57,3 +57,11 @@ BIPs that are implemented by Bitcoin Core (up-to-date up to **v22.0**): with mainnet activation as of **v0.21.1** ([PR 21377](https://github.com/bitcoin/bitcoin/pull/21377), [PR 21686](https://github.com/bitcoin/bitcoin/pull/21686)). * [`BIP 350`](https://github.com/bitcoin/bips/blob/master/bip-0350.mediawiki): Addresses for native v1+ segregated Witness outputs use Bech32m instead of Bech32 as of **v22.0** ([PR 20861](https://github.com/bitcoin/bitcoin/pull/20861)). +* [`BIP 380`](https://github.com/bitcoin/bips/blob/master/bip-0380.mediawiki) + [`381`](https://github.com/bitcoin/bips/blob/master/bip-0381.mediawiki) + [`382`](https://github.com/bitcoin/bips/blob/master/bip-0382.mediawiki) + [`383`](https://github.com/bitcoin/bips/blob/master/bip-0383.mediawiki) + [`384`](https://github.com/bitcoin/bips/blob/master/bip-0384.mediawiki) + [`385`](https://github.com/bitcoin/bips/blob/master/bip-0385.mediawiki): + Output Script Descriptors, and most of Script Expressions are implemented as of **v0.17.0** ([PR 13697](https://github.com/bitcoin/bitcoin/pull/13697)). +* [`BIP 386`](https://github.com/bitcoin/bips/blob/master/bip-0386.mediawiki): tr() Output Script Descriptors are implemented as of **v22.0** ([PR 22051](https://github.com/bitcoin/bitcoin/pull/22051)). diff --git a/doc/build-android.md b/doc/build-android.md index 7a8a9e6a65..2f2e01c441 100644 --- a/doc/build-android.md +++ b/doc/build-android.md @@ -3,9 +3,22 @@ ANDROID BUILD NOTES This guide describes how to build and package the `bitcoin-qt` GUI for Android on Linux and macOS. -## Preparation -You will need to get the Android NDK and build dependencies for Android as described in [depends/README.md](../depends/README.md). +## Dependencies + +Before proceeding with an Android build one needs to get the [Android SDK](https://developer.android.com/studio) and use the "SDK Manager" tool to download the NDK and one or more "Platform packages" (these are Android versions and have a corresponding API level). + +The minimum supported Android NDK version is [r23](https://github.com/android/ndk/wiki/Changelog-r23). + +In order to build `ANDROID_API_LEVEL` (API level corresponding to the Android version targeted, e.g. Android 9.0 Pie is 28 and its "Platform package" needs to be available) and `ANDROID_TOOLCHAIN_BIN` (path to toolchain binaries depending on the platform the build is being performed on) need to be set. + +API levels from 24 to 29 have been tested to work. + +If the build includes Qt, environment variables `ANDROID_SDK` and `ANDROID_NDK` need to be set as well but can otherwise be omitted. +This is an example command for a default build with no disabled dependencies: + + ANDROID_SDK=/home/user/Android/Sdk ANDROID_NDK=/home/user/Android/Sdk/ndk-bundle make HOST=aarch64-linux-android ANDROID_API_LEVEL=28 ANDROID_TOOLCHAIN_BIN=/home/user/Android/Sdk/ndk-bundle/toolchains/llvm/prebuilt/linux-x86_64/bin + ## Building and packaging diff --git a/doc/build-unix.md b/doc/build-unix.md index 02c36eea7c..f50a9b23c0 100644 --- a/doc/build-unix.md +++ b/doc/build-unix.md @@ -42,13 +42,12 @@ Optional dependencies: ------------|------------------|---------------------- miniupnpc | UPnP Support | Firewall-jumping support libnatpmp | NAT-PMP Support | Firewall-jumping support - libdb4.8 | Berkeley DB | Optional, wallet storage (only needed when wallet enabled) + libdb4.8 | Berkeley DB | Wallet storage (only needed when legacy wallet enabled) qt | GUI | GUI toolkit (only needed when GUI enabled) - libqrencode | QR codes in GUI | Optional for generating QR codes (only needed when GUI enabled) - univalue | Utility | JSON parsing and encoding (bundled version will be used unless --with-system-univalue passed to configure) - libzmq3 | ZMQ notification | Optional, allows generating ZMQ notifications (requires ZMQ version >= 4.0.0) - sqlite3 | SQLite DB | Optional, wallet storage (only needed when wallet enabled) - systemtap | Tracing (USDT) | Optional, statically defined tracepoints + libqrencode | QR codes in GUI | QR code generation (only needed when GUI enabled) + libzmq3 | ZMQ notification | ZMQ notifications (requires ZMQ version >= 4.0.0) + sqlite3 | SQLite DB | Wallet storage (only needed when descriptor wallet enabled) + systemtap | Tracing (USDT) | Statically defined tracepoints For the versions used, see [dependencies.md](dependencies.md) @@ -85,19 +84,15 @@ Now, you can either build from self-compiled [depends](/depends/README.md) or in sudo apt-get install libevent-dev libboost-dev libboost-system-dev libboost-filesystem-dev libboost-test-dev -Berkeley DB is required for the wallet. - -Ubuntu and Debian have their own `libdb-dev` and `libdb++-dev` packages, but these will install -Berkeley DB 5.1 or later. This will break binary wallet compatibility with the distributed executables, which -are based on BerkeleyDB 4.8. If you do not care about wallet compatibility, -pass `--with-incompatible-bdb` to configure. - -Otherwise, you can build Berkeley DB [yourself](#berkeley-db). - SQLite is required for the descriptor wallet: sudo apt install libsqlite3-dev +Berkeley DB is required for the legacy wallet. Ubuntu and Debian have their own `libdb-dev` and `libdb++-dev` packages, +but these will install Berkeley DB 5.1 or later. This will break binary wallet compatibility with the distributed +executables, which are based on BerkeleyDB 4.8. If you do not care about wallet compatibility, pass +`--with-incompatible-bdb` to configure. Otherwise, you can build Berkeley DB [yourself](#berkeley-db). + To build Bitcoin Core without wallet, see [*Disable-wallet mode*](#disable-wallet-mode) Optional port mapping libraries (see: `--with-miniupnpc`, `--enable-upnp-default`, and `--with-natpmp`, `--enable-natpmp-default`): @@ -146,20 +141,18 @@ Now, you can either build from self-compiled [depends](/depends/README.md) or in sudo dnf install libevent-devel boost-devel -Berkeley DB is required for the wallet: +SQLite is required for the descriptor wallet: + + sudo dnf install sqlite-devel + +Berkeley DB is required for the legacy wallet: sudo dnf install libdb4-devel libdb4-cxx-devel Newer Fedora releases, since Fedora 33, have only `libdb-devel` and `libdb-cxx-devel` packages, but these will install Berkeley DB 5.3 or later. This will break binary wallet compatibility with the distributed executables, which are based on Berkeley DB 4.8. If you do not care about wallet compatibility, -pass `--with-incompatible-bdb` to configure. - -Otherwise, you can build Berkeley DB [yourself](#berkeley-db). - -SQLite is required for the descriptor wallet: - - sudo dnf install sqlite-devel +pass `--with-incompatible-bdb` to configure. Otherwise, you can build Berkeley DB [yourself](#berkeley-db). To build Bitcoin Core without wallet, see [*Disable-wallet mode*](#disable-wallet-mode) @@ -225,8 +218,10 @@ turned off by default. See the configure options for NAT-PMP behavior desired: Berkeley DB ----------- -It is recommended to use Berkeley DB 4.8. If you have to build it yourself, -you can use [the installation script included in contrib/](/contrib/install_db4.sh) + +The legacy wallet uses Berkeley DB. To ensure backwards compatibility it is +recommended to use Berkeley DB 4.8. If you have to build it yourself, you can +use [the installation script included in contrib/](/contrib/install_db4.sh) like so: ```shell @@ -239,15 +234,6 @@ Otherwise, you can build Bitcoin Core from self-compiled [depends](/depends/READ **Note**: You only need Berkeley DB if the wallet is enabled (see [*Disable-wallet mode*](#disable-wallet-mode)). -Boost ------ -If you need to build Boost yourself: - - sudo su - ./bootstrap.sh - ./bjam install - - Security -------- To help make your Bitcoin Core installation more secure by making certain attacks impossible to @@ -326,7 +312,7 @@ This example lists the steps necessary to setup and build a command line only, n Note: Enabling wallet support requires either compiling against a Berkeley DB newer than 4.8 (package `db`) using `--with-incompatible-bdb`, or building and depending on a local version of Berkeley DB 4.8. The readily available Arch Linux packages are currently built using -`--with-incompatible-bdb` according to the [PKGBUILD](https://projects.archlinux.org/svntogit/community.git/tree/bitcoin/trunk/PKGBUILD). +`--with-incompatible-bdb` according to the [PKGBUILD](https://github.com/archlinux/svntogit-community/blob/packages/bitcoin/trunk/PKGBUILD). As mentioned above, when maintaining portability of the wallet between the standard Bitcoin Core distributions and independently built node software is desired, Berkeley DB 4.8 must be used. diff --git a/doc/dependencies.md b/doc/dependencies.md index 0c1fd6ba98..24422f1d7b 100644 --- a/doc/dependencies.md +++ b/doc/dependencies.md @@ -8,11 +8,10 @@ These are the dependencies currently used by Bitcoin Core. You can find instruct | Berkeley DB | [4.8.30](https://www.oracle.com/technetwork/database/database-technologies/berkeleydb/downloads/index.html) | 4.8.x | No | | | | Boost | [1.71.0](https://www.boost.org/users/download/) | [1.64.0](https://github.com/bitcoin/bitcoin/pull/22320) | No | | | | Clang<sup>[ \* ](#note1)</sup> | | [7.0](https://releases.llvm.org/download.html) (C++17 & std::filesystem support) | | | | -| Expat | [2.2.7](https://libexpat.github.io/) | | No | Yes | | -| fontconfig | [2.12.1](https://www.freedesktop.org/software/fontconfig/release/) | | No | Yes | | -| FreeType | [2.7.1](https://download.savannah.gnu.org/releases/freetype) | | No | | [Yes](https://github.com/bitcoin/bitcoin/blob/master/depends/packages/qt.mk) (Android only) | +| Fontconfig | [2.12.6](https://www.freedesktop.org/software/fontconfig/release/) | | No | Yes | | +| FreeType | [2.11.0](https://download.savannah.gnu.org/releases/freetype) | | No | | [Yes](https://github.com/bitcoin/bitcoin/blob/master/depends/packages/qt.mk) (Android only) | | GCC | | [8.1](https://gcc.gnu.org/) (C++17 & std::filesystem support) | | | | -| glibc | | [2.17](https://www.gnu.org/software/libc/) | | | | | +| glibc | | [2.18](https://www.gnu.org/software/libc/) | | | | | | HarfBuzz-NG | | | | | [Yes](https://github.com/bitcoin/bitcoin/blob/master/depends/packages/qt.mk) | | libevent | [2.1.12-stable](https://github.com/libevent/libevent/releases) | [2.0.21](https://github.com/bitcoin/bitcoin/pull/18676) | No | | | | libnatpmp | git commit [4536032...](https://github.com/miniupnp/libnatpmp/tree/4536032ae32268a45c073a4d5e91bbab4534773a) | | No | | | diff --git a/doc/developer-notes.md b/doc/developer-notes.md index a05ea93a46..1888897856 100644 --- a/doc/developer-notes.md +++ b/doc/developer-notes.md @@ -1029,6 +1029,9 @@ Current subtrees include: - Subtree at https://github.com/bitcoin-core/univalue-subtree ; maintained by Core contributors. - Deviates from upstream https://github.com/jgarzik/univalue. +- src/minisketch + - Upstream at https://github.com/sipa/minisketch ; maintained by Core contributors. + Upgrading LevelDB --------------------- @@ -1251,6 +1254,12 @@ A few guidelines for introducing and reviewing new RPC interfaces: - *Rationale*: User-facing consistency. +- Use `fs::path::u8string()` and `fs::u8path()` functions when converting path + to JSON strings, not `fs::PathToString` and `fs::PathFromString` + + - *Rationale*: JSON strings are Unicode strings, not byte strings, and + RFC8259 requires JSON to be encoded as UTF-8. + Internal interface guidelines ----------------------------- diff --git a/doc/fuzzing.md b/doc/fuzzing.md index 0880f9f581..73d04837f1 100644 --- a/doc/fuzzing.md +++ b/doc/fuzzing.md @@ -19,6 +19,10 @@ $ FUZZ=process_message src/test/fuzz/fuzz There is also a runner script to execute all fuzz targets. Refer to `./test/fuzz/test_runner.py --help` for more details. +## Overview of Bitcoin Core fuzzing + +[Google](https://github.com/google/fuzzing/) has a good overview of fuzzing in general, with contributions from key architects of some of the most-used fuzzers. [This paper](https://agroce.github.io/bitcoin_report.pdf) includes an external overview of the status of Bitcoin Core fuzzing, as of summer 2021. [John Regehr](https://blog.regehr.org/archives/1687) provides good advice on writing code that assists fuzzers in finding bugs, which is useful for developers to keep in mind. + ## Fuzzing harnesses and output [`process_message`](https://github.com/bitcoin/bitcoin/blob/master/src/test/fuzz/process_message.cpp) is a fuzzing harness for the [`ProcessMessage(...)` function (`net_processing`)](https://github.com/bitcoin/bitcoin/blob/master/src/net_processing.cpp). The available fuzzing harnesses are found in [`src/test/fuzz/`](https://github.com/bitcoin/bitcoin/tree/master/src/test/fuzz). diff --git a/doc/multisig-tutorial.md b/doc/multisig-tutorial.md new file mode 100644 index 0000000000..0793040418 --- /dev/null +++ b/doc/multisig-tutorial.md @@ -0,0 +1,241 @@ +# 1. Multisig Tutorial + +Currently, it is possible to create a multisig wallet using Bitcoin Core only. + +Although there is already a brief explanation about the multisig in the [Descriptors documentation](https://github.com/bitcoin/bitcoin/blob/master/doc/descriptors.md#multisig), this tutorial proposes to use the signet (instead of regtest), bringing the reader closer to a real environment and explaining some functions in more detail. + +This tutorial uses [jq](https://github.com/stedolan/jq) JSON processor to process the results from RPC and stores the relevant values in bash variables. This makes the tutorial reproducible and easier to follow step by step. + +Before starting this tutorial, start the bitcoin node on the signet network. + +```bash +./src/bitcoind -signet -daemon +``` + +This tutorial also uses the default WPKH derivation path to get the xpubs and does not conform to [BIP 45](https://github.com/bitcoin/bips/blob/master/bip-0045.mediawiki) or [BIP 87](https://github.com/bitcoin/bips/blob/master/bip-0087.mediawiki). + +At the time of writing, there is no way to extract a specific path from wallets in Bitcoin Core. For this, an external signer/xpub can be used. + +[PR #22341](https://github.com/bitcoin/bitcoin/pull/22341), which is still under development, introduces a new wallet RPC `getxpub`. It takes a BIP32 path as an argument and returns the xpub, along with the master key fingerprint. + +## 1.1 Basic Multisig Workflow + +### 1.1 Create the Descriptor Wallets + +For a 2-of-3 multisig, create 3 descriptor wallets. It is important that they are of the descriptor type in order to retrieve the wallet descriptors. These wallets contain HD seed and private keys, which will be used to sign the PSBTs and derive the xpub. + +These three wallets should not be used directly for privacy reasons (public key reuse). They should only be used to sign transactions for the (watch-only) multisig wallet. + +```bash +for ((n=1;n<=3;n++)) +do + ./src/bitcoin-cli -signet -named createwallet wallet_name="participant_${n}" descriptors=true +done +``` + +Extract the xpub of each wallet. To do this, the `listdescriptors` RPC is used. By default, Bitcoin Core single-sig wallets are created using path `m/44'/1'/0'` for PKH, `m/84'/1'/0'` for WPKH, `m/49'/1'/0'` for P2WPKH-nested-in-P2SH and `m/86'/1'/0'` for P2TR based accounts. Each of them uses the chain 0 for external addresses and chain 1 for internal ones, as shown in the example below. + +``` +wpkh([1004658e/84'/1'/0']tpubDCBEcmVKbfC9KfdydyLbJ2gfNL88grZu1XcWSW9ytTM6fitvaRmVyr8Ddf7SjZ2ZfMx9RicjYAXhuh3fmLiVLPodPEqnQQURUfrBKiiVZc8/0/*)#g8l47ngv + +wpkh([1004658e/84'/1'/0']tpubDCBEcmVKbfC9KfdydyLbJ2gfNL88grZu1XcWSW9ytTM6fitvaRmVyr8Ddf7SjZ2ZfMx9RicjYAXhuh3fmLiVLPodPEqnQQURUfrBKiiVZc8/1/*)#en65rxc5 +``` + +The suffix (after #) is the checksum. Descriptors can optionally be suffixed with a checksum to protect against typos or copy-paste errors. +All RPCs in Bitcoin Core will include the checksum in their output. + +```bash +declare -A xpubs + +for ((n=1;n<=3;n++)) +do + xpubs["internal_xpub_${n}"]=$(./src/bitcoin-cli -signet -rpcwallet="participant_${n}" listdescriptors | jq '.descriptors | [.[] | select(.desc | startswith("wpkh") and contains("/1/*"))][0] | .desc' | grep -Po '(?<=\().*(?=\))') + + xpubs["external_xpub_${n}"]=$(./src/bitcoin-cli -signet -rpcwallet="participant_${n}" listdescriptors | jq '.descriptors | [.[] | select(.desc | startswith("wpkh") and contains("/0/*") )][0] | .desc' | grep -Po '(?<=\().*(?=\))') +done +``` + +`jq` is used to extract the xpub from the `wpkh` descriptor. + +The following command can be used to verify if the xpub was generated correctly. + +```bash +for x in "${!xpubs[@]}"; do printf "[%s]=%s\n" "$x" "${xpubs[$x]}" ; done +``` + +As previously mentioned, this step extracts the `m/84'/1'/0'` account instead of the path defined in [BIP 45](https://github.com/bitcoin/bips/blob/master/bip-0045.mediawiki) or [BIP 87](https://github.com/bitcoin/bips/blob/master/bip-0087.mediawiki), since there is no way to extract a specific path in Bitcoin Core at the time of writing. + +### 1.2 Define the Multisig Descriptors + +Define the external and internal multisig descriptors, add the checksum and then, join both in a JSON array. + +```bash +external_desc="wsh(sortedmulti(2,${xpubs["external_xpub_1"]},${xpubs["external_xpub_2"]},${xpubs["external_xpub_3"]}))" +internal_desc="wsh(sortedmulti(2,${xpubs["internal_xpub_1"]},${xpubs["internal_xpub_2"]},${xpubs["internal_xpub_3"]}))" + +external_desc_sum=$(./src/bitcoin-cli -signet getdescriptorinfo $external_desc | jq '.descriptor') +internal_desc_sum=$(./src/bitcoin-cli -signet getdescriptorinfo $internal_desc | jq '.descriptor') + +multisig_ext_desc="{\"desc\": $external_desc_sum, \"active\": true, \"internal\": false, \"timestamp\": \"now\"}" +multisig_int_desc="{\"desc\": $internal_desc_sum, \"active\": true, \"internal\": true, \"timestamp\": \"now\"}" + +multisig_desc="[$multisig_ext_desc, $multisig_int_desc]" +``` + +`external_desc` and `internal_desc` specify the output type (`wsh`, in this case) and the xpubs involved. They also use BIP 67 (`sortedmulti`), so the wallet can be recreated without worrying about the order of xpubs. Conceptually, descriptors describe a list of scriptPubKey (along with information for spending from it) [[source](https://github.com/bitcoin/bitcoin/issues/21199#issuecomment-780772418)]. + +Note that at least two descriptors are usually used, one for internal derivation paths and external ones. There are discussions about eliminating this redundancy, as can been seen in the issue [#17190](https://github.com/bitcoin/bitcoin/issues/17190). + +After creating the descriptors, it is necessary to add the checksum, which is required by the `importdescriptors` RPC. + +The checksum for a descriptor without one can be computed using the `getdescriptorinfo` RPC. The response has the `descriptor` field, which is the descriptor with the checksum added. + +There are other fields that can be added to the descriptors: + +* `active`: Sets the descriptor to be the active one for the corresponding output type (`wsh`, in this case). +* `internal`: Indicates whether matching outputs should be treated as something other than incoming payments (e.g. change). +* `timestamp`: Sets the time from which to start rescanning the blockchain for the descriptor, in UNIX epoch time. + +Documentation for these and other parameters can be found by typing `./src/bitcoin-cli help importdescriptors`. + +`multisig_desc` concatenates external and internal descriptors in a JSON array and then it will be used to create the multisig wallet. + +### 1.3 Create the Multisig Wallet + +To create the multisig wallet, first create an empty one (no keys, HD seed and private keys disabled). + +Then import the descriptors created in the previous step using the `importdescriptors` RPC. + +After that, `getwalletinfo` can be used to check if the wallet was created successfully. + +```bash +./src/bitcoin-cli -signet -named createwallet wallet_name="multisig_wallet_01" disable_private_keys=true blank=true descriptors=true + +./src/bitcoin-cli -signet -rpcwallet="multisig_wallet_01" importdescriptors "$multisig_desc" + +./src/bitcoin-cli -signet -rpcwallet="multisig_wallet_01" getwalletinfo +``` + +Once the wallets have already been created and this tutorial needs to be repeated or resumed, it is not necessary to recreate them, just load them with the command below: + +```bash +for ((n=1;n<=3;n++)); do ./src/bitcoin-cli -signet loadwallet "participant_${n}"; done +``` + +### 1.4 Fund the wallet + +The wallet can receive signet coins by generating a new address and passing it as parameters to `getcoins.py` script. + +This script will print a captcha in dot-matrix to the terminal, using unicode Braille characters. After solving the captcha, the coins will be sent directly to the address or wallet (according to the parameters). + +The url used by the script can also be accessed directly. At time of writing, the url is [`https://signetfaucet.com`](https://signetfaucet.com). + +Coins received by the wallet must have at least 1 confirmation before they can be spent. It is necessary to wait for a new block to be mined before continuing. + +```bash +receiving_address=$(./src/bitcoin-cli -signet -rpcwallet="multisig_wallet_01" getnewaddress) + +./contrib/signet/getcoins.py -c ./src/bitcoin-cli -a $receiving_address +``` + +To copy the receiving address onto the clipboard, use the following command. This can be useful when getting coins via the signet faucet mentioned above. + +```bash +echo -n "$receiving_address" | xclip -sel clip +``` + +The `getbalances` RPC may be used to check the balance. Coins with `trusted` status can be spent. + +```bash +./src/bitcoin-cli -signet -rpcwallet="multisig_wallet_01" getbalances +``` + +### 1.5 Create a PSBT + +Unlike singlesig wallets, multisig wallets cannot create and sign transactions directly because they require the signatures of the co-signers. Instead they create a Partially Signed Bitcoin Transaction (PSBT). + +PSBT is a data format that allows wallets and other tools to exchange information about a Bitcoin transaction and the signatures necessary to complete it. [[source](https://bitcoinops.org/en/topics/psbt/)] + +The current PSBT version (v0) is defined in [BIP 174](https://github.com/bitcoin/bips/blob/master/bip-0174.mediawiki). + +For simplicity, the destination address is taken from the `participant_1` wallet in the code above, but it can be any valid bitcoin address. + +The `walletcreatefundedpsbt` RPC is used to create and fund a transaction in the PSBT format. It is the first step in creating the PSBT. + +```bash +balance=$(./src/bitcoin-cli -signet -rpcwallet="multisig_wallet_01" getbalance) + +amount=$(echo "$balance * 0.8" | bc -l | sed -e 's/^\./0./' -e 's/^-\./-0./') + +destination_addr=$(./src/bitcoin-cli -signet -rpcwallet="participant_1" getnewaddress) + +funded_psbt=$(./src/bitcoin-cli -signet -named -rpcwallet="multisig_wallet_01" walletcreatefundedpsbt outputs="{\"$destination_addr\": $amount}" | jq -r '.psbt') +``` + +There is also the `createpsbt` RPC, which serves the same purpose, but it has no access to the wallet or to the UTXO set. It is functionally the same as `createrawtransaction` and just drops the raw transaction into an otherwise blank PSBT. [[source](https://bitcointalk.org/index.php?topic=5131043.msg50573609#msg50573609)] In most cases, `walletcreatefundedpsbt` solves the problem. + +The `send` RPC can also return a PSBT if more signatures are needed to sign the transaction. + +### 1.6 Decode or Analyze the PSBT + +Optionally, the PSBT can be decoded to a JSON format using `decodepsbt` RPC. + +The `analyzepsbt` RPC analyzes and provides information about the current status of a PSBT and its inputs, e.g. missing signatures. + +```bash +./src/bitcoin-cli -signet decodepsbt $funded_psbt + +./src/bitcoin-cli -signet analyzepsbt $funded_psbt +``` + +### 1.7 Update the PSBT + +In the code above, two PSBTs are created. One signed by `participant_1` wallet and other, by the `participant_2` wallet. + +The `walletprocesspsbt` is used by the wallet to sign a PSBT. + +```bash +psbt_1=$(./src/bitcoin-cli -signet -rpcwallet="participant_1" walletprocesspsbt $funded_psbt | jq '.psbt') + +psbt_2=$(./src/bitcoin-cli -signet -rpcwallet="participant_2" walletprocesspsbt $funded_psbt | jq '.psbt') +``` + +### 1.8 Combine the PSBT + +The PSBT, if signed separately by the co-signers, must be combined into one transaction before being finalized. This is done by `combinepsbt` RPC. + +```bash +combined_psbt=$(./src/bitcoin-cli -signet combinepsbt "[$psbt_1, $psbt_2]") +``` + +There is an RPC called `joinpsbts`, but it has a different purpose than `combinepsbt`. `joinpsbts` joins the inputs from multiple distinct PSBTs into one PSBT. + +In the example above, the PSBTs are the same, but signed by different participants. If the user tries to merge them using `joinpsbts`, the error `Input txid:pos exists in multiple PSBTs` is returned. To be able to merge different PSBTs into one, they must have different inputs and outputs. + +### 1.9 Finalize and Broadcast the PSBT + +The `finalizepsbt` RPC is used to produce a network serialized transaction which can be broadcast with `sendrawtransaction`. + +It checks that all inputs have complete scriptSigs and scriptWitnesses and, if so, encodes them into network serialized transactions. + +```bash +finalized_psbt_hex=$(./src/bitcoin-cli -signet finalizepsbt $combined_psbt | jq -r '.hex') + +./src/bitcoin-cli -signet sendrawtransaction $finalized_psbt_hex +``` + +### 1.10 Alternative Workflow (PSBT sequential signatures) + +Instead of each wallet signing the original PSBT and combining them later, the wallets can also sign the PSBTs sequentially. This is less scalable than the previously presented parallel workflow, but it works. + +After that, the rest of the process is the same: the PSBT is finalized and transmitted to the network. + +```bash +psbt_1=$(./src/bitcoin-cli -signet -rpcwallet="participant_1" walletprocesspsbt $funded_psbt | jq -r '.psbt') + +psbt_2=$(./src/bitcoin-cli -signet -rpcwallet="participant_2" walletprocesspsbt $psbt_1 | jq -r '.psbt') + +finalized_psbt_hex=$(./src/bitcoin-cli -signet finalizepsbt $psbt_2 | jq -r '.hex') + +./src/bitcoin-cli -signet sendrawtransaction $finalized_psbt_hex +```
\ No newline at end of file diff --git a/doc/policy/README.md b/doc/policy/README.md new file mode 100644 index 0000000000..6e8686365d --- /dev/null +++ b/doc/policy/README.md @@ -0,0 +1,15 @@ +# Transaction Relay Policy + +**Policy** (Mempool or Transaction Relay Policy) is the node's set of validation rules, in addition +to consensus, enforced for unconfirmed transactions before submitting them to the mempool. These +rules are local to the node and configurable (e.g. `-minrelaytxfee`, `-limitancestorsize`, +`-incrementalRelayFee`). Policy may include restrictions on the transaction itself, the transaction +in relation to the current chain tip, and the transaction in relation to the node's mempool +contents. Policy is *not* applied to transactions in blocks. + +This documentation is not an exhaustive list of all policy rules. + +- [Mempool Limits](mempool-limits.md) +- [Mempool Replacements](mempool-replacements.md) +- [Packages](packages.md) + diff --git a/doc/policy/mempool-limits.md b/doc/policy/mempool-limits.md new file mode 100644 index 0000000000..73ab017f1b --- /dev/null +++ b/doc/policy/mempool-limits.md @@ -0,0 +1,65 @@ +# Mempool Limits + +## Definitions + +Given any two transactions Tx0 and Tx1 where Tx1 spends an output of Tx0, +Tx0 is a *parent* of Tx1 and Tx1 is a *child* of Tx0. + +A transaction's *ancestors* include, recursively, its parents, the parents of its parents, etc. +A transaction's *descendants* include, recursively, its children, the children of its children, etc. + +A mempool entry's *ancestor count* is the total number of in-mempool (unconfirmed) transactions in +its ancestor set, including itself. +A mempool entry's *descendant count* is the total number of in-mempool (unconfirmed) transactions in +its descendant set, including itself. + +A mempool entry's *ancestor size* is the aggregated virtual size of in-mempool (unconfirmed) +transactions in its ancestor set, including itself. +A mempool entry's *descendant size* is the aggregated virtual size of in-mempool (unconfirmed) +transactions in its descendant set, including itself. + +Transactions submitted to the mempool must not exceed the ancestor and descendant limits (aka +mempool *package limits*) set by the node (see `-limitancestorcount`, `-limitancestorsize`, +`-limitdescendantcount`, `-limitdescendantsize`). + +## Exemptions + +### CPFP Carve Out + +**CPFP Carve Out** if a transaction candidate for submission to the +mempool would cause some mempool entry to exceed its descendant limits, an exemption is made if all +of the following conditions are met: + +1. The candidate transaction is no more than 10,000 virtual bytes. + +2. The candidate transaction has an ancestor count of 2 (itself and exactly 1 ancestor). + +3. The in-mempool transaction's descendant count, including the candidate transaction, would only + exceed the limit by 1. + +*Rationale*: this rule was introduced to prevent pinning by domination of a transaction's descendant +limits in two-party contract protocols such as LN. Also see the [mailing list +post](https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2018-November/016518.html). + +This rule was introduced in [PR #15681](https://github.com/bitcoin/bitcoin/pull/15681). + +### Single-Conflict RBF Carve Out + +When a candidate transaction for submission to the mempool would replace mempool entries, it may +also decrease the descendant count of other mempool entries. Since ancestor/descendant limits are +calculated prior to removing the would-be-replaced transactions, they may be overestimated. + +An exemption is given for a candidate transaction that would replace mempool transactions and meets +all of the following conditions: + +1. The candidate transaction has exactly 1 directly conflicting transaction. + +2. The candidate transaction does not spend any unconfirmed inputs that are not also spent by the + directly conflicting transaction. + +The following discounts are given to account for the would-be-replaced transaction(s): + +1. The descendant count limit is temporarily increased by 1. + +2. The descendant size limit temporarily is increased by the virtual size of the to-be-replaced + directly conflicting transaction. diff --git a/doc/policy/mempool-replacements.md b/doc/policy/mempool-replacements.md new file mode 100644 index 0000000000..3e844f8d7b --- /dev/null +++ b/doc/policy/mempool-replacements.md @@ -0,0 +1,69 @@ +# Mempool Replacements + +## Current Replace-by-Fee Policy + +A transaction conflicts with an in-mempool transaction ("directly conflicting transaction") if they +spend one or more of the same inputs. A transaction may conflict with multiple in-mempool +transactions. + +A transaction ("replacement transaction") may replace its directly conflicting transactions and +their in-mempool descendants (together, "original transactions") if, in addition to passing all +other consensus and policy rules, each of the following conditions are met: + +1. The directly conflicting transactions all signal replaceability explicitly. A transaction is + signaling replaceability if any of its inputs have an nSequence number less than (0xffffffff - 1). + + *Rationale*: See [BIP125 + explanation](https://github.com/bitcoin/bips/blob/master/bip-0125.mediawiki#motivation). + +2. The replacement transaction only include an unconfirmed input if that input was included in + one of the directly conflicting transactions. An unconfirmed input spends an output from a + currently-unconfirmed transaction. + + *Rationale*: When RBF was originally implemented, the mempool did not keep track of + ancestor feerates yet. This rule was suggested as a temporary restriction. + +3. The replacement transaction pays an absolute fee of at least the sum paid by the original + transactions. + + *Rationale*: Only requiring the replacement transaction to have a higher feerate could allow an + attacker to bypass node minimum relay feerate requirements and cause the network to repeatedly + relay slightly smaller replacement transactions without adding any more fees. Additionally, if + any of the original transactions would be included in the next block assembled by an economically + rational miner, a replacement policy allowing the replacement transaction to decrease the absolute + fees in the next block would be incentive-incompatible. + +4. The additional fees (difference between absolute fee paid by the replacement transaction and the + sum paid by the original transactions) pays for the replacement transaction's bandwidth at or + above the rate set by the node's incremental relay feerate. For example, if the incremental relay + feerate is 1 satoshi/vB and the replacement transaction is 500 virtual bytes total, then the + replacement pays a fee at least 500 satoshis higher than the sum of the original transactions. + + *Rationale*: Try to prevent DoS attacks where an attacker causes the network to repeatedly relay + transactions each paying a tiny additional amount in fees, e.g. just 1 satoshi. + +5. The number of original transactions does not exceed 100. More precisely, the sum of all + directly conflicting transactions' descendant counts (number of transactions inclusive of itself + and its descendants) must not exceed 100; it is possible that this overestimates the true number + of original transactions. + + *Rationale*: Try to prevent DoS attacks where an attacker is able to easily occupy and flush out + significant portions of the node's mempool using replacements with multiple directly conflicting + transactions, each with large descendant sets. + +This set of rules is similar but distinct from BIP125. + +## History + +* Opt-in full replace-by-fee (without inherited signaling) honoured in mempool and mining as of + **v0.12.0** ([PR 6871](https://github.com/bitcoin/bitcoin/pull/6871)). + +* [BIP125](https://github.com/bitcoin/bips/blob/master/bip-0125.mediawiki) defined based on + Bitcoin Core implementation. + +* The incremental relay feerate used to calculate the required additional fees is distinct from + `minRelayTxFee` and configurable using `-incrementalrelayfee` + ([PR #9380](https://github.com/bitcoin/bitcoin/pull/9380)). + +* RBF enabled by default in the wallet GUI as of **v0.18.1** ([PR + #11605](https://github.com/bitcoin/bitcoin/pull/11605)). diff --git a/doc/policy/packages.md b/doc/policy/packages.md new file mode 100644 index 0000000000..07698f2af2 --- /dev/null +++ b/doc/policy/packages.md @@ -0,0 +1,59 @@ +# Package Mempool Accept + +## Definitions + +A **package** is an ordered list of transactions, representable by a connected Directed Acyclic +Graph (a directed edge exists between a transaction that spends the output of another transaction). + +For every transaction `t` in a **topologically sorted** package, if any of its parents are present +in the package, they appear somewhere in the list before `t`. + +A **child-with-unconfirmed-parents** package is a topologically sorted package that consists of +exactly one child and all of its unconfirmed parents (no other transactions may be present). +The last transaction in the package is the child, and its package can be canonically defined based +on the current state: each of its inputs must be available in the UTXO set as of the current chain +tip or some preceding transaction in the package. + +## Package Mempool Acceptance Rules + +The following rules are enforced for all packages: + +* Packages cannot exceed `MAX_PACKAGE_COUNT=25` count and `MAX_PACKAGE_SIZE=101KvB` total size + (#20833) + + - *Rationale*: This is already enforced as mempool ancestor/descendant limits. If + transactions in a package are all related, exceeding this limit would mean that the package + can either be split up or it wouldn't pass individual mempool policy. + + - Note that, if these mempool limits change, package limits should be reconsidered. Users may + also configure their mempool limits differently. + +* Packages must be topologically sorted. (#20833) + +* Packages cannot have conflicting transactions, i.e. no two transactions in a package can spend + the same inputs. Packages cannot have duplicate transactions. (#20833) + +* No transaction in a package can conflict with a mempool transaction. BIP125 Replace By Fee is + currently disabled for packages. (#20833) + + - Package RBF may be enabled in the future. + +* When packages are evaluated against ancestor/descendant limits, the union of all transactions' + descendants and ancestors is considered. (#21800) + + - *Rationale*: This is essentially a "worst case" heuristic intended for packages that are + heavily connected, i.e. some transaction in the package is the ancestor or descendant of all + the other transactions. + +The following rules are only enforced for packages to be submitted to the mempool (not enforced for +test accepts): + +* Packages must be child-with-unconfirmed-parents packages. This also means packages must contain at + least 2 transactions. (#22674) + + - *Rationale*: This allows for fee-bumping by CPFP. Allowing multiple parents makes it possible + to fee-bump a batch of transactions. Restricting packages to a defined topology is easier to + reason about and simplifies the validation logic greatly. + + - Warning: Batched fee-bumping may be unsafe for some use cases. Users and application developers + should take caution if utilizing multi-parent packages. diff --git a/doc/release-notes-14707.md b/doc/release-notes-14707.md new file mode 100644 index 0000000000..b53204f788 --- /dev/null +++ b/doc/release-notes-14707.md @@ -0,0 +1,19 @@ +Wallet `receivedby` RPCs now include coinbase transactions +------------- + +Previously, the following wallet RPCs excluded coinbase transactions: + +`getreceivedbyaddress` + +`getreceivedbylabel` + +`listreceivedbyaddress` + +`listreceivedbylabel` + +This release changes this behaviour and returns results accounting for received coins from coinbase outputs. + +A new option, `include_immature_coinbase` (default=`false`), determines whether to account for immature coinbase transactions. +Immature coinbase transactions are coinbase transactions that have 100 or fewer confirmations, and are not spendable. + +The previous behaviour can be restored using the configuration `-deprecatedrpc=exclude_coinbase`, but may be removed in a future release. diff --git a/doc/release-notes-23093.md b/doc/release-notes-23093.md deleted file mode 100644 index 68fbaec53c..0000000000 --- a/doc/release-notes-23093.md +++ /dev/null @@ -1,11 +0,0 @@ -Notable changes -=============== - -Updated RPCs ------------- - -- `upgradewallet` will now automatically flush the keypool if upgrading -from a non-HD wallet to an HD wallet, to immediately start using the -newly-generated HD keys. -- a new RPC `newkeypool` has been added, which will flush (entirely -clear and refill) the keypool. diff --git a/doc/release-notes-23113.md b/doc/release-notes-23113.md new file mode 100644 index 0000000000..b0904c9d7b --- /dev/null +++ b/doc/release-notes-23113.md @@ -0,0 +1,9 @@ +Notable changes +=============== + +Updated RPCs +------------ + +- Both `createmultisig` and `addmultisigaddress` now include a `warnings` +field, which will show a warning if a non-legacy address type is requested +when using uncompressed public keys. diff --git a/doc/release-notes-gui-459.md b/doc/release-notes-gui-459.md new file mode 100644 index 0000000000..b590ac5d45 --- /dev/null +++ b/doc/release-notes-gui-459.md @@ -0,0 +1,4 @@ +GUI changes +----------- + +- The Bech32 checkbox has been replaced with a dropdown for all address types, including the new Bech32m (BIP-350) standard for Taproot enabled wallets. diff --git a/doc/release-notes.md b/doc/release-notes.md index 9541aeadca..17b0ef545d 100644 --- a/doc/release-notes.md +++ b/doc/release-notes.md @@ -77,6 +77,21 @@ Otherwise, please use the `rescanblockchain` RPC to trigger a rescan. (#23123) Updated RPCs ------------ +- `upgradewallet` will now automatically flush the keypool if upgrading + from a non-HD wallet to an HD wallet, to immediately start using the + newly-generated HD keys. (#23093) + +- a new RPC `newkeypool` has been added, which will flush (entirely + clear and refill) the keypool. (#23093) + +- The `validateaddress` RPC now returns an `error_locations` array for invalid + addresses, with the indices of invalid character locations in the address (if + known). For example, this will attempt to locate up to two Bech32 errors, and + return their locations if successful. Success and correctness are only guaranteed + if fewer than two substitution errors have been made. + The error message returned in the `error` field now also returns more specific + errors when decoding fails. (#16807) + - The `-deprecatedrpc=addresses` configuration option has been removed. RPCs `gettxout`, `getrawtransaction`, `decoderawtransaction`, `decodescript`, `gettransaction verbose=true` and REST endpoints `/rest/tx`, `/rest/getutxos`, @@ -99,6 +114,15 @@ Updated RPCs causes the lock to be written persistently to the wallet database. This allows UTXOs to remain locked even after node restarts or crashes. (#23065) +- The top-level fee fields `fee`, `modifiedfee`, `ancestorfees` and `descendantfees` + returned by RPCs `getmempoolentry`,`getrawmempool(verbose=true)`, + `getmempoolancestors(verbose=true)` and `getmempooldescendants(verbose=true)` + are deprecated and will be removed in the next major version (use + `-deprecated=fees` if needed in this version). The same fee fields can be accessed + through the `fees` object in the result. WARNING: deprecated + fields `ancestorfees` and `descendantfees` are denominated in sats, whereas all + fields in the `fees` object are denominated in BTC. (#22689) + New RPCs -------- @@ -127,6 +151,10 @@ Updated settings mean `-persistmempool=1`. Passing `-persistmempool=0`, `-persistmempool=1` and `-nopersistmempool` is unaffected. (#23061) +- `-maxuploadtarget` now allows human readable byte units [k|K|m|M|g|G|t|T]. + E.g. `-maxuploadtarget=500g`. No whitespace, +- or fractions allowed. + Default is `M` if no suffix provided. (#23249) + Tools and Utilities ------------------- diff --git a/doc/release-process.md b/doc/release-process.md index 6a5202d0f9..f786b345b1 100644 --- a/doc/release-process.md +++ b/doc/release-process.md @@ -19,8 +19,7 @@ Release Process * On both the master branch and the new release branch: - update `CLIENT_VERSION_MAJOR` in [`configure.ac`](../configure.ac) - - update `CLIENT_VERSION_MAJOR`, `PACKAGE_VERSION`, and `PACKAGE_STRING` in [`build_msvc/bitcoin_config.h`](/build_msvc/bitcoin_config.h) -* On the new release branch in [`configure.ac`](../configure.ac) and [`build_msvc/bitcoin_config.h`](/build_msvc/bitcoin_config.h) (see [this commit](https://github.com/bitcoin/bitcoin/commit/742f7dd)): +* On the new release branch in [`configure.ac`](../configure.ac)(see [this commit](https://github.com/bitcoin/bitcoin/commit/742f7dd)): - set `CLIENT_VERSION_MINOR` to `0` - set `CLIENT_VERSION_BUILD` to `0` - set `CLIENT_VERSION_IS_RELEASE` to `true` @@ -60,7 +59,7 @@ Release Process To tag the version (or release candidate) in git, use the `make-tag.py` script from [bitcoin-maintainer-tools](https://github.com/bitcoin-core/bitcoin-maintainer-tools). From the root of the repository run: - ../bitcoin-maintainer-tools/make-tag.py v(new version, e.g. 0.20.0) + ../bitcoin-maintainer-tools/make-tag.py v(new version, e.g. 23.0) This will perform a few last-minute consistency checks in the build system files, and if they pass, create a signed tag. @@ -253,6 +252,10 @@ cat "$VERSION"/*/all.SHA256SUMS.asc > SHA256SUMS.asc - bitcoincore.org maintained versions update: [table](https://github.com/bitcoin-core/bitcoincore.org/commits/master/_includes/posts/maintenance-table.md) + - Delete post-EOL [release branches](https://github.com/bitcoin/bitcoin/branches/all) and create a tag `v${branch_name}-final`. + + - Delete ["Needs backport" labels](https://github.com/bitcoin/bitcoin/labels?q=backport) for non-existing branches. + - bitcoincore.org RPC documentation update - Install [golang](https://golang.org/doc/install) diff --git a/doc/tracing.md b/doc/tracing.md index 57104c43a0..5b9ba09c2f 100644 --- a/doc/tracing.md +++ b/doc/tracing.md @@ -108,6 +108,55 @@ Arguments passed: 5. SigOps in the Block (excluding coinbase SigOps) `uint64` 6. Time it took to connect the Block in microseconds (µs) as `uint64` +### Context `utxocache` + +#### Tracepoint `utxocache:flush` + +Is called *after* the caches and indexes are flushed depending on the mode +`CChainState::FlushStateToDisk` is called with. + +Arguments passed: +1. Duration in microseconds as `int64` +2. Flush state mode as `uint32`. It's an enumerator class with values `0` + (`NONE`), `1` (`IF_NEEDED`), `2` (`PERIODIC`), `3` (`ALWAYS`) +3. Number of coins flushed as `uint64` +4. Memory usage in bytes as `uint64` +5. If the flush was pruned as `bool` +6. If it was full flush as `bool` + +#### Tracepoint `utxocache:add` + +It is called when a new coin is added to the UTXO cache. + +Arguments passed: +1. Transaction ID (hash) as `pointer to unsigned chars` (i.e. 32 bytes in little-endian) +2. Output index as `uint32` +3. Block height the coin was added to the UTXO-set as `uint32` +4. Value of the coin as `int64` +5. If the coin is a coinbase as `bool` + +#### Tracepoint `utxocache:spent` + +It is called when a coin is spent from the UTXO cache. + +Arguments passed: +1. Transaction ID (hash) as `pointer to unsigned chars` (i.e. 32 bytes in little-endian) +2. Output index as `uint32` +3. Block height the coin was spent, as `uint32` +4. Value of the coin as `int64` +5. If the coin is a coinbase as `bool` + +#### Tracepoint `utxocache:uncache` + +It is called when the UTXO with the given outpoint is removed from the cache. + +Arguments passed: +1. Transaction ID (hash) as `pointer to unsigned chars` (i.e. 32 bytes in little-endian) +2. Output index as `uint32` +3. Block height the coin was uncached, as `uint32` +4. Value of the coin as `int64` +. If the coin is a coinbase as `bool` + ## Adding tracepoints to Bitcoin Core To add a new tracepoint, `#include <util/trace.h>` in the compilation unit where diff --git a/doc/zmq.md b/doc/zmq.md index 0521fe08d8..b832e71734 100644 --- a/doc/zmq.md +++ b/doc/zmq.md @@ -91,9 +91,11 @@ For instance: Each PUB notification has a topic and body, where the header corresponds to the notification type. For instance, for the notification `-zmqpubhashtx` the topic is `hashtx` (no null -terminator) and the body is the transaction hash (32 -bytes) for all but `sequence` topic. For `sequence`, the body -is structured as the following based on the type of message: +terminator). These options can also be provided in bitcoin.conf. + +The topics are: + +`sequence`: the body is structured as the following based on the type of message: <32-byte hash>C : Blockhash connected <32-byte hash>D : Blockhash disconnected @@ -102,7 +104,24 @@ is structured as the following based on the type of message: Where the 8-byte uints correspond to the mempool sequence number. -These options can also be provided in bitcoin.conf. +`rawtx`: Notifies about all transactions, both when they are added to mempool or when a new block arrives. This means a transaction could be published multiple times. First, when it enters the mempool and then again in each block that includes it. The messages are ZMQ multipart messages with three parts. The first part is the topic (`rawtx`), the second part is the serialized transaction, and the last part is a sequence number (representing the message count to detect lost messages). + + | rawtx | <serialized transaction> | <uint32 sequence number in Little Endian> + +`hashtx`: Notifies about all transactions, both when they are added to mempool or when a new block arrives. This means a transaction could be published multiple times. First, when it enters the mempool and then again in each block that includes it. The messages are ZMQ multipart messages with three parts. The first part is the topic (`hashtx`), the second part is the 32-byte transaction hash, and the last part is a sequence number (representing the message count to detect lost messages). + + | hashtx | <32-byte transaction hash in Little Endian> | <uint32 sequence number in Little Endian> + + +`rawblock`: Notifies when the chain tip is updated. Messages are ZMQ multipart messages with three parts. The first part is the topic (`rawblock`), the second part is the serialized block, and the last part is a sequence number (representing the message count to detect lost messages). + + | rawblock | <serialized block> | <uint32 sequence number in Little Endian> + +`hashblock`: Notifies when the chain tip is updated. Messages are ZMQ multipart messages with three parts. The first part is the topic (`hashblock`), the second part is the 32-byte block hash, and the last part is a sequence number (representing the message count to detect lost messages). + + | hashblock | <32-byte block hash in Little Endian> | <uint32 sequence number in Little Endian> + +**_NOTE:_** Note that the 32-byte hashes are in Little Endian and not in the Big Endian format that the RPC interface and block explorers use to display transaction and block hashes. ZeroMQ endpoint specifiers for TCP (and others) are documented in the [ZeroMQ API](http://api.zeromq.org/4-0:_start). |