diff options
Diffstat (limited to 'doc')
50 files changed, 942 insertions, 156 deletions
diff --git a/doc/REST-interface.md b/doc/REST-interface.md index ac7cd45f70..bf669235e3 100644 --- a/doc/REST-interface.md +++ b/doc/REST-interface.md @@ -77,6 +77,20 @@ $ curl localhost:18332/rest/getutxos/checkmempool/b2cdfd7b89def827ff8af7cd9bff76 } ``` +####Memory pool +`GET /rest/mempool/info.json` + +Returns various information about the TX mempool. +Only supports JSON as output format. +* size : (numeric) the number of transactions in the TX mempool +* bytes : (numeric) size of the TX mempool in bytes +* usage : (numeric) total TX mempool memory usage + +`GET /rest/mempool/contents.json` + +Returns transactions in the TX mempool. +Only supports JSON as output format. + Risks ------------- Running a web browser on the same node with a REST enabled bitcoind can be a risk. Accessing prepared XSS websites could read out tx/block data of your node by placing links like `<script src="http://127.0.0.1:8332/rest/tx/1234567890.json">` which might break the nodes privacy. diff --git a/doc/assets-attribution.md b/doc/assets-attribution.md index c6da1a4586..2dd930d6a4 100644 --- a/doc/assets-attribution.md +++ b/doc/assets-attribution.md @@ -1,46 +1 @@ -The following is a list of assets used in the bitcoin source and their proper attribution. - -[Typicons/Stephen Hutchings](http://typicons.com) ------------------------ - -### Info -* Icon Pack: Typicons (http://typicons.com) -* Designer: Stephen Hutchings (and more) -* License: MIT -* Site: [https://github.com/stephenhutchings/typicons.font](https://github.com/stephenhutchings/typicons.font) - -### Assets Used - src/qt/res/icons/add.png, src/qt/res/icons/address-book.png, - src/qt/res/icons/configure.png, src/qt/res/icons/connect4.png, - src/qt/res/icons/debugwindow.png, src/qt/res/icons/edit.png, - src/qt/res/icons/exitcopy.png, src/qt/res/icons/editpaste.png, - src/qt/res/icons/export.png, src/qt/res/icons/eye.png, - src/qt/res/icons/filesave.png, src/qt/res/icons/history.png, - src/qt/res/icons/info.png, src/qt/res/icons/key.png, - src/qt/res/icons/lock_*.png, src/qt/res/icons/open.png, - src/qt/res/icons/overview.png, src/qt/res/icons/quit.png, - src/qt/res/icons/receive.png, src/qt/res/icons/remove.png, - src/qt/res/icons/send.png, src/qt/res/icons/synced.png, - src/qt/res/icons/transaction*.png, src/qt/res/icons/tx_output.png, - src/qt/res/icons/warning.png - -Jonas Schnelli ------------------------ - -### Info -* Designer: Jonas Schnelli -* Bitcoin Icon: (based on the original bitcoin logo from Bitboy) -* Some icons are based on Stephan Hutchings Typicons -* License: MIT - -### Assets Used - src/qt/res/icons/about.png, src/qt/res/icons/about_qt.png, - src/qt/res/icons/bitcoin.icns, src/qt/res/icons/bitcoin.ico, - src/qt/res/icons/bitcoin.png, src/qt/res/icons/clock*.png, - src/qt/res/icons/connect[0-3].png, src/qt/res/icons/eye_minus.png, - src/qt/res/icons/eye_plus.png, src/qt/res/icons/verify.png, - src/qt/res/icons/tx_inout.png, src/qt/res/icons/tx_input.png, - src/qt/res/src/verify.svg, src/qt/res/src/bitcoin.svg, - src/qt/res/src/clock*.svg, src/qt/res/src/connect*.svg, - src/qt/res/src/mine.svg, src/qt/res/src/qt.svg, src/qt/res/src/tx*.svg, - src/qt/res/src/verify.svg, +The list of assets used in the bitcoin source and their attribution can now be found in [contrib/debian/copyright](../contrib/debian/copyright). diff --git a/doc/bips.md b/doc/bips.md index 90e98ed419..c84bd966f5 100644 --- a/doc/bips.md +++ b/doc/bips.md @@ -1,4 +1,4 @@ -BIPs that are implemented by Bitcoin Core (up-to-date up to **v0.10.0**): +BIPs that are implemented by Bitcoin Core (up-to-date up to **v0.12.0**): * [`BIP 11`](https://github.com/bitcoin/bips/blob/master/bip-0011.mediawiki): Multisig outputs are standard since **v0.6.0** ([PR #669](https://github.com/bitcoin/bitcoin/pull/669)). * [`BIP 13`](https://github.com/bitcoin/bips/blob/master/bip-0013.mediawiki): The address format for P2SH addresses has been implemented since **v0.6.0** ([PR #669](https://github.com/bitcoin/bitcoin/pull/669)). @@ -16,3 +16,4 @@ BIPs that are implemented by Bitcoin Core (up-to-date up to **v0.10.0**): * [`BIP 61`](https://github.com/bitcoin/bips/blob/master/bip-0061.mediawiki): The 'reject' protocol message (and the protocol version bump to 70002) was added in **v0.9.0** ([PR #3185](https://github.com/bitcoin/bitcoin/pull/3185)). * [`BIP 66`](https://github.com/bitcoin/bips/blob/master/bip-0066.mediawiki): The strict DER rules and associated version 3 blocks have been implemented since **v0.10.0** ([PR #5713](https://github.com/bitcoin/bitcoin/pull/5713)). * [`BIP 70`](https://github.com/bitcoin/bips/blob/master/bip-0070.mediawiki) [`71`](https://github.com/bitcoin/bips/blob/master/bip-0071.mediawiki) [`72`](https://github.com/bitcoin/bips/blob/master/bip-0072.mediawiki): Payment Protocol support has been available in Bitcoin Core GUI since **v0.9.0** ([PR #5216](https://github.com/bitcoin/bitcoin/pull/5216)). +* [`BIP 111`](https://github.com/bitcoin/bips/blob/master/bip-0111.mediawiki): `NODE_BLOOM` service bit added, but only enforced for peer versions `>=70011` as of **v0.12.0** ([PR #6579](https://github.com/bitcoin/bitcoin/pull/6579)). diff --git a/doc/build-osx.md b/doc/build-osx.md index dc319dd1c4..201fe9522b 100644 --- a/doc/build-osx.md +++ b/doc/build-osx.md @@ -32,7 +32,7 @@ Instructions: Homebrew #### Install dependencies using Homebrew - brew install autoconf automake berkeley-db4 libtool boost miniupnpc openssl pkg-config protobuf qt5 + brew install autoconf automake berkeley-db4 libtool boost miniupnpc openssl pkg-config protobuf qt5 libevent NOTE: Building with Qt4 is still supported, however, could result in a broken UI. As such, building with Qt5 is recommended. @@ -70,7 +70,7 @@ Download Qt Creator from http://www.qt.io/download/. Download the "community edi 6. Confirm the "summary page" 7. In the "Projects" tab select "Manage Kits..." 8. Select the default "Desktop" kit and select "Clang (x86 64bit in /usr/bin)" as compiler -9. Select LLDB as debugger (you might need to set the path to your installtion) +9. Select LLDB as debugger (you might need to set the path to your installation) 10. Start debugging with Qt Creator Creating a release build diff --git a/doc/build-unix.md b/doc/build-unix.md index 92aed7725e..e02a5e42f7 100644 --- a/doc/build-unix.md +++ b/doc/build-unix.md @@ -31,8 +31,9 @@ These dependencies are required: Library | Purpose | Description ------------|------------------|---------------------- - libssl | SSL Support | Secure communications - libboost | Boost | C++ Library + libssl | Crypto | Random Number Generation, Elliptic Curve Cryptography + libboost | Utility | Library for threading, data structures, etc + libevent | Networking | OS independent asynchronous networking Optional dependencies: @@ -57,7 +58,7 @@ Dependency Build Instructions: Ubuntu & Debian ---------------------------------------------- Build requirements: - sudo apt-get install build-essential libtool autotools-dev autoconf pkg-config libssl-dev + sudo apt-get install build-essential libtool autotools-dev autoconf pkg-config libssl-dev libevent-dev For Ubuntu 12.04 and later or Debian 7 and later libboost-all-dev has to be installed: @@ -154,7 +155,8 @@ make install # Configure Bitcoin Core to use our own-built instance of BDB cd $BITCOIN_ROOT -./configure (other args...) LDFLAGS="-L${BDB_PREFIX}/lib/" CPPFLAGS="-I${BDB_PREFIX}/include/" +./autogen.sh +./configure LDFLAGS="-L${BDB_PREFIX}/lib/" CPPFLAGS="-I${BDB_PREFIX}/include/" # (other args...) ``` **Note**: You only need Berkeley DB if the wallet is enabled (see the section *Disable-Wallet mode* below). diff --git a/doc/files.md b/doc/files.md index 80195535bb..c083bcb038 100644 --- a/doc/files.md +++ b/doc/files.md @@ -1,12 +1,17 @@ -Used in 0.8.0 ---------------------- -* wallet.dat: personal wallet (BDB) with keys and transactions -* peers.dat: peer IP address database (custom format); since 0.7.0 + +* banlist.dat: stores the IPs/Subnets of banned nodes +* bitcoin.conf: contains configuration settings for bitcoind or bitcoin-qt +* bitcoind.pid: stores the process id of bitcoind while running * blocks/blk000??.dat: block data (custom, 128 MiB per file); since 0.8.0 * blocks/rev000??.dat; block undo data (custom); since 0.8.0 (format changed since pre-0.8) * blocks/index/*; block index (LevelDB); since 0.8.0 * chainstate/*; block chain state database (LevelDB); since 0.8.0 * database/*: BDB database environment; only used for wallet since 0.8.0 +* db.log: wallet database log file +* debug.log: contains debug information and general logging generated by bitcoind or bitcoin-qt +* fee_estimates.dat: stores statistics used to estimate minimum transaction fees and priorities required for confirmation; since 0.10.0 +* peers.dat: peer IP address database (custom format); since 0.7.0 +* wallet.dat: personal wallet (BDB) with keys and transactions Only used in pre-0.8.0 --------------------- diff --git a/doc/gitian-building.md b/doc/gitian-building.md index 630b3c04a7..b434ae8a5b 100644 --- a/doc/gitian-building.md +++ b/doc/gitian-building.md @@ -5,7 +5,7 @@ Gitian building Gitian is the deterministic build process that is used to build the Bitcoin Core executables. It provides a way to be reasonably sure that the -executables are really built from source on GitHub. It also makes sure that +executables are really built from the source on GitHub. It also makes sure that the same, tested dependencies are used and statically built into the executable. Multiple developers build the source code by following a specific descriptor @@ -13,8 +13,8 @@ Multiple developers build the source code by following a specific descriptor These results are compared and only if they match, the build is accepted and uploaded to bitcoin.org. -More independent gitian builders are needed, which is why I wrote this -guide. It is preferred to follow these steps yourself instead of using someone else's +More independent gitian builders are needed, which is why this guide exists. +It is preferred you follow these steps yourself instead of using someone else's VM image to avoid 'contaminating' the build. Table of Contents @@ -39,46 +39,46 @@ This guide explains how to set up the environment, and how to start the builds. Debian Linux was chosen as the host distribution because it has a lightweight install (in contrast to Ubuntu) and is readily available. Any kind of virtualization can be used, for example: -- [VirtualBox](https://www.virtualbox.org/), covered by this guide +- [VirtualBox](https://www.virtualbox.org/) (covered by this guide) - [KVM](http://www.linux-kvm.org/page/Main_Page) - [LXC](https://linuxcontainers.org/), see also [Gitian host docker container](https://github.com/gdm85/tenku/tree/master/docker/gitian-bitcoin-host/README.md). -You can also install on actual hardware instead of using virtualization. +You can also install gitian on actual hardware instead of using virtualization. Create a new VirtualBox VM --------------------------- In the VirtualBox GUI click "Create" and choose the following parameters in the wizard: - + -- Type: Linux, Debian (64 bit) +- Type: Linux, Debian (64-bit)  -- Memory Size: at least 1024MB, anything lower will really slow the build down +- Memory Size: at least 1024MB, anything less will really slow down the build. - + -- Hard Drive: Create a virtual hard drive now +- Hard Disk: Create a virtual hard disk now - + -- Hard Drive file type: Use the default, VDI (VirtualBox Disk Image) +- Hard Disk file type: Use the default, VDI (VirtualBox Disk Image) - + -- Storage on Physical hard drive: Dynamically Allocated +- Storage on physical hard disk: Dynamically Allocated  -- Disk size: at least 40GB; as low as 20GB *may* be possible, but better to err on the safe side -- Push the `Create` button +- File location and size: at least 40GB; as low as 20GB *may* be possible, but better to err on the safe side +- Click `Create` -Get the [Debian 7.8 net installer](http://cdimage.debian.org/cdimage/archive/7.8.0/amd64/iso-cd/debian-7.8.0-amd64-netinst.iso) (a more recent minor version should also work, see also [Debian Network installation](https://www.debian.org/CD/netinst/)). +Get the [Debian 8.1 net installer](http://cdimage.debian.org/debian-cd/8.1.0/amd64/iso-cd/debian-8.1.0-amd64-netinst.iso) (a more recent minor version should also work, see also [Debian Network installation](https://www.debian.org/CD/netinst/)). This DVD image can be validated using a SHA256 hashing tool, for example on Unixy OSes by entering the following in a terminal: - echo "e39c36d6adc0fd86c6edb0e03e22919086c883b37ca194d063b8e3e8f6ff6a3a debian-7.8.0-amd64-netinst.iso" | sha256sum -c + echo "5d0a1f804d73aee73eee7efbb38456390558094fd19894a573f1514ca44347e0 debian-8.1.0-amd64-netinst.iso" | sha256sum -c # (must return OK) After creating the VM, we need to configure it. @@ -115,8 +115,9 @@ This section will explain how to install Debian on the newly created VM.  -**Note**: Navigation in the Debian installer: To keep a setting at the default -and proceed, just press `Enter`. To select a different button, press `Tab`. +**Note**: Navigating in the Debian installer: +To keep a setting at the default and proceed, just press `Enter`. +To select a different button, press `Tab`. - Choose locale and keyboard settings (doesn't matter, you can just go with the defaults or select your own information) @@ -126,7 +127,7 @@ and proceed, just press `Enter`. To select a different button, press `Tab`. - The VM will detect network settings using DHCP, this should all proceed automatically - Configure the network: - - System name `debian`. + - Hostname `debian`. - Leave domain name empty.  @@ -136,6 +137,7 @@ and proceed, just press `Enter`. To select a different button, press `Tab`.  - Name the new user `debian` (the full name doesn't matter, you can leave it empty) +- Set the account username as `debian`   @@ -158,13 +160,9 @@ and proceed, just press `Enter`. To select a different button, press `Tab`.  - - Partitioning scheme: All files in one partition - - - - Finish partitioning and write changes to disk -> *Yes* (`Tab`, `Enter` to select the `Yes` button) - +  - The base system will be installed, this will take a minute or so @@ -172,51 +170,79 @@ and proceed, just press `Enter`. To select a different button, press `Tab`.  -- Enter proxy information (unless you are on an intranet, you can leave this empty) +- Enter proxy information (unless you are on an intranet, leave this empty)  - Wait a bit while 'Select and install software' runs - Participate in popularity contest -> *No* -- Choose software to install. We need just the base system. +- Choose software to install. We need just the base system. +- Make sure only 'SSH server' and 'Standard System Utilities' are checked +- Uncheck 'Debian Desktop Environment' and 'Print Server'  -- Make sure only 'SSH server' and 'Standard System Utilities' are checked -- Uncheck 'Debian Desktop Environment' and 'Print Server' +- Install the GRUB boot loader to the master boot record? -> Yes  -- Install the GRUB boot loader to the master boot record? -> Yes +- Device for boot loader installation -> ata-VBOX_HARDDISK - + - Installation Complete -> *Continue* - After installation, the VM will reboot and you will have a working Debian VM. Congratulations! + + + +After Installation +------------------- +The next step in the guide involves logging in as root via SSH. +SSH login for root users is disabled by default, so we'll enable that now. + +Login to the VM using username `root` and the root password you choose earlier. +You'll be presented with a screen similar to this. + + + +Type: + +``` +sed -i 's/^PermitRootLogin.*/PermitRootLogin yes/' /etc/ssh/sshd_config +``` +and press enter. Then, +``` +/etc/init.d/ssh restart +``` +and enter to restart SSH. Logout by typing 'logout' and pressing 'enter'. + Connecting to the VM ---------------------- After the VM has booted you can connect to it using SSH, and files can be copied from and to the VM using a SFTP utility. Connect to `localhost`, port `22222` (or the port configured when installing the VM). -On Windows you can use putty[1] and WinSCP[2]. +On Windows you can use [putty](http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html) and [WinSCP](http://winscp.net/eng/index.php). -For example to connect as `root` from a Linux command prompt use +For example, to connect as `root` from a Linux command prompt use $ ssh root@localhost -p 22222 The authenticity of host '[localhost]:22222 ([127.0.0.1]:22222)' can't be established. - ECDSA key fingerprint is 8e:71:f9:5b:62:46:de:44:01:da:fb:5f:34:b5:f2:18. + RSA key fingerprint is ae:f5:c8:9f:17:c6:c7:1b:c2:1b:12:31:1d:bb:d0:c7. Are you sure you want to continue connecting (yes/no)? yes - Warning: Permanently added '[localhost]:22222' (ECDSA) to the list of known hosts. + Warning: Permanently added '[localhost]:22222' (RSA) to the list of known hosts. root@localhost's password: (enter root password configured during install) - Linux debian 3.2.0-4-amd64 #1 SMP Debian 3.2.54-2 x86_64 + + The programs included with the Debian GNU/Linux system are free software; + the exact distribution terms for each program are described in the + individual files in /usr/share/doc/*/copyright. + + Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent + permitted by applicable law. root@debian:~# Replace `root` with `debian` to log in as user. -[1] http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html -[2] http://winscp.net/eng/index.php - Setting up Debian for gitian building -------------------------------------- @@ -226,13 +252,10 @@ First we need to log in as `root` to set up dependencies and make sure that our user can use the sudo command. Type/paste the following in the terminal: ```bash -apt-get install git ruby sudo apt-cacher-ng qemu-utils debootstrap lxc python-cheetah parted kpartx bridge-utils +apt-get install git ruby sudo apt-cacher-ng qemu-utils debootstrap lxc python-cheetah parted kpartx bridge-utils make ubuntu-archive-keyring adduser debian sudo ``` -When you get a colorful screen with a question about the 'LXC directory', just -go with the default (`/var/lib/lxc`). - Then set up LXC and the rest with the following, which is a complex jumble of settings and workarounds: ```bash @@ -255,7 +278,7 @@ reboot ``` At the end the VM is rebooted to make sure that the changes take effect. The steps in this -section need only to be performed once. +section only need to be performed once. Installing gitian ------------------ @@ -300,26 +323,27 @@ cd gitian-builder bin/make-base-vm --lxc --arch amd64 --suite precise ``` -There will be a lot of warnings printed during build of the image. These can be ignored. +There will be a lot of warnings printed during the build of the image. These can be ignored. **Note**: When sudo asks for a password, enter the password for the user *debian* not for *root*. Getting and building the inputs -------------------------------- -Follow the instructions in [doc/release-process.md](release-process.md) in the bitcoin repository -under 'Fetch and build inputs' to install sources which require manual intervention. Also follow -the next step: 'Seed the Gitian sources cache', which will fetch all necessary source files allowing -for gitian to work offline. +Follow the instructions in [doc/release-process.md](release-process.md#fetch-and-build-inputs-first-time-or-when-dependency-versions-change) +in the bitcoin repository under 'Fetch and build inputs' to install sources which require +manual intervention. Also optionally follow the next step: 'Seed the Gitian sources cache +and offline git repositories' which will fetch the remaining files required for building +offline. Building Bitcoin ---------------- To build Bitcoin (for Linux, OSX and Windows) just follow the steps under 'perform -gitian builds' in [doc/release-process.md](release-process.md) in the bitcoin repository. +gitian builds' in [doc/release-process.md](release-process.md#perform-gitian-builds) in the bitcoin repository. -This may take a long time as it also builds the dependencies needed for each descriptor. -These dependencies will be cached after a successful build to avoid rebuilding them where possible. +This may take some time as it will build all the dependencies needed for each descriptor. +These dependencies will be cached after a successful build to avoid rebuilding them when possible. At any time you can check the package installation and build progress with @@ -331,13 +355,13 @@ tail -f var/build.log Output from `gbuild` will look something like Initialized empty Git repository in /home/debian/gitian-builder/inputs/bitcoin/.git/ - remote: Reusing existing pack: 35606, done. - remote: Total 35606 (delta 0), reused 0 (delta 0) - Receiving objects: 100% (35606/35606), 26.52 MiB | 4.28 MiB/s, done. - Resolving deltas: 100% (25724/25724), done. + remote: Counting objects: 57959, done. + remote: Total 57959 (delta 0), reused 0 (delta 0), pack-reused 57958 + Receiving objects: 100% (57959/57959), 53.76 MiB | 484.00 KiB/s, done. + Resolving deltas: 100% (41590/41590), done. From https://github.com/bitcoin/bitcoin ... (new tags, new branch etc) - --- Building for precise x86_64 --- + --- Building for precise amd64 --- Stopping target if it is up Making a new image copy stdin: is not a tty @@ -368,6 +392,57 @@ COMMIT=2014_03_windows_unicode_path ./bin/gbuild --commit bitcoin=${COMMIT} --url bitcoin=${URL} ../bitcoin/contrib/gitian-descriptors/gitian-osx.yml ``` +Building fully offline +----------------------- + +For building fully offline including attaching signatures to unsigned builds, the detached-sigs repository +and the bitcoin git repository with the desired tag must both be available locally, and then gbuild must be +told where to find them. It also requires an apt-cacher-ng which is fully-populated but set to offline mode, or +manually disabling gitian-builder's use of apt-get to update the VM build environment. + +To configure apt-cacher-ng as an offline cacher, you will need to first populate its cache with the relevant +files. You must additionally patch target-bin/bootstrap-fixup to set its apt sources to something other than +plain archive.ubuntu.com: us.archive.ubuntu.com works. + +So, if you use LXC: + +```bash +export PATH="$PATH":/path/to/gitian-builder/libexec +export USE_LXC=1 +cd /path/to/gitian-builder +./libexec/make-clean-vm --suite precise --arch amd64 + +LXC_ARCH=amd64 LXC_SUITE=precise on-target -u root apt-get update +LXC_ARCH=amd64 LXC_SUITE=precise on-target -u root \ + -e DEBIAN_FRONTEND=noninteractive apt-get --no-install-recommends -y install \ + $( sed -ne '/^packages:/,/[^-] .*/ {/^- .*/{s/"//g;s/- //;p}}' ../bitcoin/contrib/gitian-descriptors/*|sort|uniq ) +LXC_ARCH=amd64 LXC_SUITE=precise on-target -u root apt-get -q -y purge grub +LXC_ARCH=amd64 LXC_SUITE=precise on-target -u root -e DEBIAN_FRONTEND=noninteractive apt-get -y dist-upgrade +``` + +And then set offline mode for apt-cacher-ng: + +``` +/etc/apt-cacher-ng/acng.conf +[...] +Offlinemode: 1 +[...] + +service apt-cacher-ng restart +``` + +Then when building, override the remote URLs that gbuild would otherwise pull from the gitian descriptors:: +```bash + +cd /some/root/path/ +git clone https://github.com/bitcoin/bitcoin-detached-sigs.git + +BTCPATH=/some/root/path/bitcoin.git +SIGPATH=/some/root/path/bitcoin-detached-sigs.git + +./bin/gbuild --url bitcoin=${BTCPATH},signature=${SIGPATH} ../bitcoin/contrib/gitian-descriptors/gitian-win-signer.yml +``` + Signing externally ------------------- diff --git a/doc/gitian-building/create_new_vm.png b/doc/gitian-building/create_new_vm.png Binary files differnew file mode 100644 index 0000000000..dd22428e17 --- /dev/null +++ b/doc/gitian-building/create_new_vm.png diff --git a/doc/gitian-building/create_vm_hard_disk.png b/doc/gitian-building/create_vm_hard_disk.png Binary files differnew file mode 100644 index 0000000000..8e29816fab --- /dev/null +++ b/doc/gitian-building/create_vm_hard_disk.png diff --git a/doc/gitian-building/create_vm_hard_disk_file_type.png b/doc/gitian-building/create_vm_hard_disk_file_type.png Binary files differnew file mode 100644 index 0000000000..a157211cf5 --- /dev/null +++ b/doc/gitian-building/create_vm_hard_disk_file_type.png diff --git a/doc/gitian-building/create_vm_hard_drive.png b/doc/gitian-building/create_vm_hard_drive.png Binary files differdeleted file mode 100644 index a1706e14fd..0000000000 --- a/doc/gitian-building/create_vm_hard_drive.png +++ /dev/null diff --git a/doc/gitian-building/create_vm_hard_drive_file_type.png b/doc/gitian-building/create_vm_hard_drive_file_type.png Binary files differdeleted file mode 100644 index 251b8ee3e2..0000000000 --- a/doc/gitian-building/create_vm_hard_drive_file_type.png +++ /dev/null diff --git a/doc/gitian-building/create_vm_memsize.png b/doc/gitian-building/create_vm_memsize.png Binary files differindex 33717867a5..5abfee5337 100644 --- a/doc/gitian-building/create_vm_memsize.png +++ b/doc/gitian-building/create_vm_memsize.png diff --git a/doc/gitian-building/create_vm_page1.png b/doc/gitian-building/create_vm_page1.png Binary files differdeleted file mode 100644 index edaebc6223..0000000000 --- a/doc/gitian-building/create_vm_page1.png +++ /dev/null diff --git a/doc/gitian-building/create_vm_storage_physical_hard_disk.png b/doc/gitian-building/create_vm_storage_physical_hard_disk.png Binary files differnew file mode 100644 index 0000000000..cee16a6c63 --- /dev/null +++ b/doc/gitian-building/create_vm_storage_physical_hard_disk.png diff --git a/doc/gitian-building/create_vm_storage_physical_hard_drive.png b/doc/gitian-building/create_vm_storage_physical_hard_drive.png Binary files differdeleted file mode 100644 index 987efaa40c..0000000000 --- a/doc/gitian-building/create_vm_storage_physical_hard_drive.png +++ /dev/null diff --git a/doc/gitian-building/debian_install_10_configure_clock.png b/doc/gitian-building/debian_install_10_configure_clock.png Binary files differindex 467c79018e..7cda038ae4 100644 --- a/doc/gitian-building/debian_install_10_configure_clock.png +++ b/doc/gitian-building/debian_install_10_configure_clock.png diff --git a/doc/gitian-building/debian_install_11_partition_disks.png b/doc/gitian-building/debian_install_11_partition_disks.png Binary files differindex 18110734df..2a648c517f 100644 --- a/doc/gitian-building/debian_install_11_partition_disks.png +++ b/doc/gitian-building/debian_install_11_partition_disks.png diff --git a/doc/gitian-building/debian_install_12_choose_disk.png b/doc/gitian-building/debian_install_12_choose_disk.png Binary files differindex a00d4abf17..0f3acc498e 100644 --- a/doc/gitian-building/debian_install_12_choose_disk.png +++ b/doc/gitian-building/debian_install_12_choose_disk.png diff --git a/doc/gitian-building/debian_install_13_partition_scheme.png b/doc/gitian-building/debian_install_13_partition_scheme.png Binary files differdeleted file mode 100644 index 2f80f19b63..0000000000 --- a/doc/gitian-building/debian_install_13_partition_scheme.png +++ /dev/null diff --git a/doc/gitian-building/debian_install_14_finish.png b/doc/gitian-building/debian_install_14_finish.png Binary files differindex 411d457e95..c8ef0b37ad 100644 --- a/doc/gitian-building/debian_install_14_finish.png +++ b/doc/gitian-building/debian_install_14_finish.png diff --git a/doc/gitian-building/debian_install_15_write_changes.png b/doc/gitian-building/debian_install_15_write_changes.png Binary files differindex f26093982c..d8de00dec6 100644 --- a/doc/gitian-building/debian_install_15_write_changes.png +++ b/doc/gitian-building/debian_install_15_write_changes.png diff --git a/doc/gitian-building/debian_install_16_choose_a_mirror.png b/doc/gitian-building/debian_install_16_choose_a_mirror.png Binary files differindex d2c2e9523b..0bd985b38c 100644 --- a/doc/gitian-building/debian_install_16_choose_a_mirror.png +++ b/doc/gitian-building/debian_install_16_choose_a_mirror.png diff --git a/doc/gitian-building/debian_install_17_choose_a_mirror2.png b/doc/gitian-building/debian_install_17_choose_a_mirror2.png Binary files differdeleted file mode 100644 index cef2db0781..0000000000 --- a/doc/gitian-building/debian_install_17_choose_a_mirror2.png +++ /dev/null diff --git a/doc/gitian-building/debian_install_18_proxy_settings.png b/doc/gitian-building/debian_install_18_proxy_settings.png Binary files differindex 24ba25c109..2c19919f64 100644 --- a/doc/gitian-building/debian_install_18_proxy_settings.png +++ b/doc/gitian-building/debian_install_18_proxy_settings.png diff --git a/doc/gitian-building/debian_install_19_software_selection.png b/doc/gitian-building/debian_install_19_software_selection.png Binary files differindex d462757aff..5430456b14 100644 --- a/doc/gitian-building/debian_install_19_software_selection.png +++ b/doc/gitian-building/debian_install_19_software_selection.png diff --git a/doc/gitian-building/debian_install_1_boot_menu.png b/doc/gitian-building/debian_install_1_boot_menu.png Binary files differindex 27fd849b4f..216502e1c6 100644 --- a/doc/gitian-building/debian_install_1_boot_menu.png +++ b/doc/gitian-building/debian_install_1_boot_menu.png diff --git a/doc/gitian-building/debian_install_20_install_grub.png b/doc/gitian-building/debian_install_20_install_grub.png Binary files differindex de4f9be0c9..d853c15871 100644 --- a/doc/gitian-building/debian_install_20_install_grub.png +++ b/doc/gitian-building/debian_install_20_install_grub.png diff --git a/doc/gitian-building/debian_install_21_install_grub_bootloader.png b/doc/gitian-building/debian_install_21_install_grub_bootloader.png Binary files differnew file mode 100644 index 0000000000..493ab806a6 --- /dev/null +++ b/doc/gitian-building/debian_install_21_install_grub_bootloader.png diff --git a/doc/gitian-building/debian_install_21_finish_installation.png b/doc/gitian-building/debian_install_22_finish_installation.png Binary files differindex b967c3550d..7c4445585b 100644 --- a/doc/gitian-building/debian_install_21_finish_installation.png +++ b/doc/gitian-building/debian_install_22_finish_installation.png diff --git a/doc/gitian-building/debian_install_2_select_a_language.png b/doc/gitian-building/debian_install_2_select_a_language.png Binary files differindex 1c9e0bcfc1..0228ae2c01 100644 --- a/doc/gitian-building/debian_install_2_select_a_language.png +++ b/doc/gitian-building/debian_install_2_select_a_language.png diff --git a/doc/gitian-building/debian_install_3_select_location.png b/doc/gitian-building/debian_install_3_select_location.png Binary files differindex 005c395656..7b18fba975 100644 --- a/doc/gitian-building/debian_install_3_select_location.png +++ b/doc/gitian-building/debian_install_3_select_location.png diff --git a/doc/gitian-building/debian_install_4_configure_keyboard.png b/doc/gitian-building/debian_install_4_configure_keyboard.png Binary files differindex 580c8af7c5..8e46117de4 100644 --- a/doc/gitian-building/debian_install_4_configure_keyboard.png +++ b/doc/gitian-building/debian_install_4_configure_keyboard.png diff --git a/doc/gitian-building/debian_install_5_configure_the_network.png b/doc/gitian-building/debian_install_5_configure_the_network.png Binary files differindex a7fdffc66b..8e3720f243 100644 --- a/doc/gitian-building/debian_install_5_configure_the_network.png +++ b/doc/gitian-building/debian_install_5_configure_the_network.png diff --git a/doc/gitian-building/debian_install_6a_set_up_root_password.png b/doc/gitian-building/debian_install_6a_set_up_root_password.png Binary files differindex 31bd210f38..dcade11967 100644 --- a/doc/gitian-building/debian_install_6a_set_up_root_password.png +++ b/doc/gitian-building/debian_install_6a_set_up_root_password.png diff --git a/doc/gitian-building/debian_install_7_set_up_user_fullname.png b/doc/gitian-building/debian_install_7_set_up_user_fullname.png Binary files differindex bffc6ccd7a..6763c6e08a 100644 --- a/doc/gitian-building/debian_install_7_set_up_user_fullname.png +++ b/doc/gitian-building/debian_install_7_set_up_user_fullname.png diff --git a/doc/gitian-building/debian_install_8_set_up_username.png b/doc/gitian-building/debian_install_8_set_up_username.png Binary files differindex 9e2750ad4e..bb04de96d2 100644 --- a/doc/gitian-building/debian_install_8_set_up_username.png +++ b/doc/gitian-building/debian_install_8_set_up_username.png diff --git a/doc/gitian-building/debian_install_9_user_password.png b/doc/gitian-building/debian_install_9_user_password.png Binary files differindex a26d30cba5..981f1181d7 100644 --- a/doc/gitian-building/debian_install_9_user_password.png +++ b/doc/gitian-building/debian_install_9_user_password.png diff --git a/doc/gitian-building/debian_root_login.png b/doc/gitian-building/debian_root_login.png Binary files differnew file mode 100644 index 0000000000..14cdd5ba5b --- /dev/null +++ b/doc/gitian-building/debian_root_login.png diff --git a/doc/gitian-building/network_settings.png b/doc/gitian-building/network_settings.png Binary files differindex 1d9b6428a7..9e714fd154 100644 --- a/doc/gitian-building/network_settings.png +++ b/doc/gitian-building/network_settings.png diff --git a/doc/gitian-building/port_forwarding_rules.png b/doc/gitian-building/port_forwarding_rules.png Binary files differindex e45c9efffc..9e1fa2af20 100644 --- a/doc/gitian-building/port_forwarding_rules.png +++ b/doc/gitian-building/port_forwarding_rules.png diff --git a/doc/gitian-building/select_startup_disk.png b/doc/gitian-building/select_startup_disk.png Binary files differindex 729b368fd1..5acdc3fe10 100644 --- a/doc/gitian-building/select_startup_disk.png +++ b/doc/gitian-building/select_startup_disk.png diff --git a/doc/init.md b/doc/init.md index 1f206a6c02..ed9ce72154 100644 --- a/doc/init.md +++ b/doc/init.md @@ -29,28 +29,32 @@ file, however it is recommended that a strong and secure password be used as this password is security critical to securing the wallet should the wallet be enabled. -If bitcoind is run with "-daemon" flag, and no rpcpassword is set, it will -print a randomly generated suitable password to stderr. You can also -generate one from the shell yourself like this: +If bitcoind is run with the "-server" flag (set by default), and no rpcpassword is set, +it will use a special cookie file for authentication. The cookie is generated with random +content when the daemon starts, and deleted when it exits. Read access to this file +controls who can access it through RPC. -bash -c 'tr -dc a-zA-Z0-9 < /dev/urandom | head -c32 && echo' +By default the cookie is stored in the data directory, but it's location can be overridden +with the option '-rpccookiefile'. -Once you have a password in hand, set rpcpassword= in /etc/bitcoin/bitcoin.conf +This allows for running bitcoind without having to do any manual configuration. + +`conf`, `pid`, and `wallet` accept relative paths which are interpreted as +relative to the data directory. `wallet` *only* supports relative paths. For an example configuration file that describes the configuration settings, -see contrib/debian/examples/bitcoin.conf. +see `contrib/debian/examples/bitcoin.conf`. 3. Paths --------------------------------- All three configurations assume several paths that might need to be adjusted. -Binary: /usr/bin/bitcoind -Configuration file: /etc/bitcoin/bitcoin.conf -Data directory: /var/lib/bitcoind -PID file: /var/run/bitcoind/bitcoind.pid (OpenRC and Upstart) - /var/lib/bitcoind/bitcoind.pid (systemd) -Lock file: /var/lock/subsys/bitcoind (CentOS) +Binary: `/usr/bin/bitcoind` +Configuration file: `/etc/bitcoin/bitcoin.conf` +Data directory: `/var/lib/bitcoind` +PID file: `/var/run/bitcoind/bitcoind.pid` (OpenRC and Upstart) or `/var/lib/bitcoind/bitcoind.pid` (systemd) +Lock file: `/var/lock/subsys/bitcoind` (CentOS) The configuration file, PID directory (if applicable) and data directory should all be owned by the bitcoin user and group. It is advised for security @@ -65,21 +69,21 @@ can then be controlled by group membership. Installing this .service file consists of just copying it to /usr/lib/systemd/system directory, followed by the command -"systemctl daemon-reload" in order to update running systemd configuration. +`systemctl daemon-reload` in order to update running systemd configuration. -To test, run "systemctl start bitcoind" and to enable for system startup run -"systemctl enable bitcoind" +To test, run `systemctl start bitcoind` and to enable for system startup run +`systemctl enable bitcoind` 4b) OpenRC Rename bitcoind.openrc to bitcoind and drop it in /etc/init.d. Double check ownership and permissions and make it executable. Test it with -"/etc/init.d/bitcoind start" and configure it to run on startup with -"rc-update add bitcoind" +`/etc/init.d/bitcoind start` and configure it to run on startup with +`rc-update add bitcoind` 4c) Upstart (for Debian/Ubuntu based distributions) -Drop bitcoind.conf in /etc/init. Test by running "service bitcoind start" +Drop bitcoind.conf in /etc/init. Test by running `service bitcoind start` it will automatically start on reboot. NOTE: This script is incompatible with CentOS 5 and Amazon Linux 2014 as they @@ -87,7 +91,7 @@ use old versions of Upstart and do not supply the start-stop-daemon utility. 4d) CentOS -Copy bitcoind.init to /etc/init.d/bitcoind. Test by running "service bitcoind start". +Copy bitcoind.init to /etc/init.d/bitcoind. Test by running `service bitcoind start`. Using this script, you can adjust the path and flags to the bitcoind program by setting the BITCOIND and FLAGS environment variables in the file @@ -99,4 +103,3 @@ setting the BITCOIND and FLAGS environment variables in the file Auto respawning is currently only configured for Upstart and systemd. Reasonable defaults have been chosen but YMMV. - diff --git a/doc/release-notes.md b/doc/release-notes.md index 85f9bd9a2f..85cdabc7e3 100644 --- a/doc/release-notes.md +++ b/doc/release-notes.md @@ -4,6 +4,59 @@ release-notes at release time) Notable changes =============== +SSL support for RPC dropped +---------------------------- + +SSL support for RPC, previously enabled by the option `rpcssl` has been dropped +from both the client and the server. This was done in preparation for removing +the dependency on OpenSSL for the daemon completely. + +Trying to use `rpcssl` will result in an error: + + Error: SSL mode for RPC (-rpcssl) is no longer supported. + +If you are one of the few people that relies on this feature, a flexible +migration path is to use `stunnel`. This is an utility that can tunnel +arbitrary TCP connections inside SSL. On e.g. Ubuntu it can be installed with: + + sudo apt-get install stunnel4 + +Then, to tunnel a SSL connection on 28332 to a RPC server bound on localhost on port 18332 do: + + stunnel -d 28332 -r 127.0.0.1:18332 -p stunnel.pem -P '' + +It can also be set up system-wide in inetd style. + +Another way to re-attain SSL would be to setup a httpd reverse proxy. This solution +would allow the use of different authentication, loadbalancing, on-the-fly compression and +caching. A sample config for apache2 could look like: + + Listen 443 + + NameVirtualHost *:443 + <VirtualHost *:443> + + SSLEngine On + SSLCertificateFile /etc/apache2/ssl/server.crt + SSLCertificateKeyFile /etc/apache2/ssl/server.key + + <Location /bitcoinrpc> + ProxyPass http://127.0.0.1:8332/ + ProxyPassReverse http://127.0.0.1:8332/ + # optional enable digest auth + # AuthType Digest + # ... + + # optional bypass bitcoind rpc basic auth + # RequestHeader set Authorization "Basic <hash>" + # get the <hash> from the shell with: base64 <<< bitcoinrpc:<password> + </Location> + + # Or, balance the load: + # ProxyPass / balancer://balancer_cluster_name + + </VirtualHost> + Random-cookie RPC authentication --------------------------------- @@ -27,6 +80,40 @@ Low-level RPC API changes advantage if a JSON library insists on using a lossy floating point type for numbers, which would be dangerous for monetary amounts. +Option parsing behavior +----------------------- + +Command line options are now parsed strictly in the order in which they are +specified. It used to be the case that `-X -noX` ends up, unintuitively, with X +set, as `-X` had precedence over `-noX`. This is no longer the case. Like for +other software, the last specified value for an option will hold. + +`NODE_BLOOM` service bit +------------------------ + +Support for the `NODE_BLOOM` service bit, as described in [BIP +111](https://github.com/bitcoin/bips/blob/master/bip-0111.mediawiki), has been +added to the P2P protocol code. + +BIP 111 defines a service bit to allow peers to advertise that they support +bloom filters (such as used by SPV clients) explicitly. It also bumps the protocol +version to allow peers to identify old nodes which allow bloom filtering of the +connection despite lacking the new service bit. + +In this version, it is only enforced for peers that send protocol versions +`>=70011`. For the next major version it is planned that this restriction will be +removed. It is recommended to update SPV clients to check for the `NODE_BLOOM` +service bit for nodes that report versions newer than 70011. + +Merkle branches removed from wallet +----------------------------------- + +Previously, every wallet transaction stored a Merkle branch to prove its +presence in blocks. This wasn't being used for more than an expensive +sanity check. Since 0.12, these are no longer stored. When loading a +0.12 wallet into an older version, it will automatically rescan to avoid +failed checks. + 0.12.0 Change log ================= @@ -82,3 +169,5 @@ configured specifically to process scriptPubKey and not scriptSig scripts. ### Miscellaneous +- Removed bitrpc.py from contrib + diff --git a/doc/release-notes/release-notes-0.10.1.md b/doc/release-notes/release-notes-0.10.1.md index 5e939600a0..8f59f1f68c 100644 --- a/doc/release-notes/release-notes-0.10.1.md +++ b/doc/release-notes/release-notes-0.10.1.md @@ -101,7 +101,7 @@ Tests: Miscellaneous: - `c9e022b` Initialization: set Boost path locale in main thread - `23126a0` Sanitize command strings before logging them. -- `323de27` Initialization: setup environment before starting QT tests +- `323de27` Initialization: setup environment before starting Qt tests - `7494e09` Initialization: setup environment before starting tests - `df45564` Initialization: set fallback locale as environment variable diff --git a/doc/release-notes/release-notes-0.11.0.md b/doc/release-notes/release-notes-0.11.0.md new file mode 100644 index 0000000000..28e49fb7ea --- /dev/null +++ b/doc/release-notes/release-notes-0.11.0.md @@ -0,0 +1,505 @@ +Bitcoin Core version 0.11.0 is now available from: + + <https://bitcoin.org/bin/bitcoin-core-0.11.0/> + +This is a new major version release, bringing both new features and +bug fixes. + +Please report bugs using the issue tracker at github: + + <https://github.com/bitcoin/bitcoin/issues> + +Upgrading and downgrading +========================= + +How to Upgrade +-------------- + +If you are running an older version, shut it down. Wait until it has completely +shut down (which might take a few minutes for older versions), then run the +installer (on Windows) or just copy over /Applications/Bitcoin-Qt (on Mac) or +bitcoind/bitcoin-qt (on Linux). + +Downgrade warning +------------------ + +Because release 0.10.0 and later makes use of headers-first synchronization and +parallel block download (see further), the block files and databases are not +backwards-compatible with pre-0.10 versions of Bitcoin Core or other software: + +* Blocks will be stored on disk out of order (in the order they are +received, really), which makes it incompatible with some tools or +other programs. Reindexing using earlier versions will also not work +anymore as a result of this. + +* The block index database will now hold headers for which no block is +stored on disk, which earlier versions won't support. + +If you want to be able to downgrade smoothly, make a backup of your entire data +directory. Without this your node will need start syncing (or importing from +bootstrap.dat) anew afterwards. It is possible that the data from a completely +synchronised 0.10 node may be usable in older versions as-is, but this is not +supported and may break as soon as the older version attempts to reindex. + +This does not affect wallet forward or backward compatibility. There are no +known problems when downgrading from 0.11.x to 0.10.x. + +Important information +====================== + +Transaction flooding +--------------------- + +At the time of this release, the P2P network is being flooded with low-fee +transactions. This causes a ballooning of the mempool size. + +If this growth of the mempool causes problematic memory use on your node, it is +possible to change a few configuration options to work around this. The growth +of the mempool can be monitored with the RPC command `getmempoolinfo`. + +One is to increase the minimum transaction relay fee `minrelaytxfee`, which +defaults to 0.00001. This will cause transactions with fewer BTC/kB fee to be +rejected, and thus fewer transactions entering the mempool. + +The other is to restrict the relaying of free transactions with +`limitfreerelay`. This option sets the number of kB/minute at which +free transactions (with enough priority) will be accepted. It defaults to 15. +Reducing this number reduces the speed at which the mempool can grow due +to free transactions. + +For example, add the following to `bitcoin.conf`: + + minrelaytxfee=0.00005 + limitfreerelay=5 + +More robust solutions are being worked on for a follow-up release. + +Notable changes +=============== + +Block file pruning +---------------------- + +This release supports running a fully validating node without maintaining a copy +of the raw block and undo data on disk. To recap, there are four types of data +related to the blockchain in the bitcoin system: the raw blocks as received over +the network (blk???.dat), the undo data (rev???.dat), the block index and the +UTXO set (both LevelDB databases). The databases are built from the raw data. + +Block pruning allows Bitcoin Core to delete the raw block and undo data once +it's been validated and used to build the databases. At that point, the raw data +is used only to relay blocks to other nodes, to handle reorganizations, to look +up old transactions (if -txindex is enabled or via the RPC/REST interfaces), or +for rescanning the wallet. The block index continues to hold the metadata about +all blocks in the blockchain. + +The user specifies how much space to allot for block & undo files. The minimum +allowed is 550MB. Note that this is in addition to whatever is required for the +block index and UTXO databases. The minimum was chosen so that Bitcoin Core will +be able to maintain at least 288 blocks on disk (two days worth of blocks at 10 +minutes per block). In rare instances it is possible that the amount of space +used will exceed the pruning target in order to keep the required last 288 +blocks on disk. + +Block pruning works during initial sync in the same way as during steady state, +by deleting block files "as you go" whenever disk space is allocated. Thus, if +the user specifies 550MB, once that level is reached the program will begin +deleting the oldest block and undo files, while continuing to download the +blockchain. + +For now, block pruning disables block relay. In the future, nodes with block +pruning will at a minimum relay "new" blocks, meaning blocks that extend their +active chain. + +Block pruning is currently incompatible with running a wallet due to the fact +that block data is used for rescanning the wallet and importing keys or +addresses (which require a rescan.) However, running the wallet with block +pruning will be supported in the near future, subject to those limitations. + +Block pruning is also incompatible with -txindex and will automatically disable +it. + +Once you have pruned blocks, going back to unpruned state requires +re-downloading the entire blockchain. To do this, re-start the node with +-reindex. Note also that any problem that would cause a user to reindex (e.g., +disk corruption) will cause a pruned node to redownload the entire blockchain. +Finally, note that when a pruned node reindexes, it will delete any blk???.dat +and rev???.dat files in the data directory prior to restarting the download. + +To enable block pruning on the command line: + +- `-prune=N`: where N is the number of MB to allot for raw block & undo data. + +Modified RPC calls: + +- `getblockchaininfo` now includes whether we are in pruned mode or not. +- `getblock` will check if the block's data has been pruned and if so, return an +error. +- `getrawtransaction` will no longer be able to locate a transaction that has a +UTXO but where its block file has been pruned. + +Pruning is disabled by default. + +Big endian support +-------------------- + +Experimental support for big-endian CPU architectures was added in this +release. All little-endian specific code was replaced with endian-neutral +constructs. This has been tested on at least MIPS and PPC hosts. The build +system will automatically detect the endianness of the target. + +Memory usage optimization +-------------------------- + +There have been many changes in this release to reduce the default memory usage +of a node, among which: + +- Accurate UTXO cache size accounting (#6102); this makes the option `-dbcache` + precise where this grossly underestimated memory usage before +- Reduce size of per-peer data structure (#6064 and others); this increases the + number of connections that can be supported with the same amount of memory +- Reduce the number of threads (#5964, #5679); lowers the amount of (esp. + virtual) memory needed + +Fee estimation changes +---------------------- + +This release improves the algorithm used for fee estimation. Previously, -1 +was returned when there was insufficient data to give an estimate. Now, -1 +will also be returned when there is no fee or priority high enough for the +desired confirmation target. In those cases, it can help to ask for an estimate +for a higher target number of blocks. It is not uncommon for there to be no +fee or priority high enough to be reliably (85%) included in the next block and +for this reason, the default for `-txconfirmtarget=n` has changed from 1 to 2. + +Privacy: Disable wallet transaction broadcast +---------------------------------------------- + +This release adds an option `-walletbroadcast=0` to prevent automatic +transaction broadcast and rebroadcast (#5951). This option allows separating +transaction submission from the node functionality. + +Making use of this, third-party scripts can be written to take care of +transaction (re)broadcast: + +- Send the transaction as normal, either through RPC or the GUI +- Retrieve the transaction data through RPC using `gettransaction` (NOT + `getrawtransaction`). The `hex` field of the result will contain the raw + hexadecimal representation of the transaction +- The transaction can then be broadcasted through arbitrary mechanisms + supported by the script + +One such application is selective Tor usage, where the node runs on the normal +internet but transactions are broadcasted over Tor. + +For an example script see [bitcoin-submittx](https://github.com/laanwj/bitcoin-submittx). + +Privacy: Stream isolation for Tor +---------------------------------- + +This release adds functionality to create a new circuit for every peer +connection, when the software is used with Tor. The new option, +`-proxyrandomize`, is on by default. + +When enabled, every outgoing connection will (potentially) go through a +different exit node. That significantly reduces the chance to get unlucky and +pick a single exit node that is either malicious, or widely banned from the P2P +network. This improves connection reliability as well as privacy, especially +for the initial connections. + +**Important note:** If a non-Tor SOCKS5 proxy is configured that supports +authentication, but doesn't require it, this change may cause that proxy to reject +connections. A user and password is sent where they weren't before. This setup +is exceedingly rare, but in this case `-proxyrandomize=0` can be passed to +disable the behavior. + +0.11.0 Change log +================= + +Detailed release notes follow. This overview includes changes that affect +behavior, not code moves, refactors and string updates. For convenience in locating +the code changes and accompanying discussion, both the pull request and +git merge commit are mentioned. + +### RPC and REST +- #5461 `5f7279a` signrawtransaction: validate private key +- #5444 `103f66b` Add /rest/headers/<count>/<hash>.<ext> +- #4964 `95ecc0a` Add scriptPubKey field to validateaddress RPC call +- #5476 `c986972` Add time offset into getpeerinfo output +- #5540 `84eba47` Add unconfirmed and immature balances to getwalletinfo +- #5599 `40e96a3` Get rid of the internal miner's hashmeter +- #5711 `87ecfb0` Push down RPC locks +- #5754 `1c4e3f9` fix getblocktemplate lock issue +- #5756 `5d901d8` Fix getblocktemplate_proposals test by mining one block +- #5548 `d48ce48` Add /rest/chaininfos +- #5992 `4c4f1b4` Push down RPC reqWallet flag +- #6036 `585b5db` Show zero value txouts in listunspent +- #5199 `6364408` Add RPC call `gettxoutproof` to generate and verify merkle blocks +- #5418 `16341cc` Report missing inputs in sendrawtransaction +- #5937 `40f5e8d` show script verification errors in signrawtransaction result +- #5420 `1fd2d39` getutxos REST command (based on Bip64) +- #6193 `42746b0` [REST] remove json input for getutxos, limit to query max. 15 outpoints +- #6226 `5901596` json: fail read_string if string contains trailing garbage + +### Configuration and command-line options +- #5636 `a353ad4` Add option `-allowselfsignedrootcertificate` to allow self signed root certs (for testing payment requests) +- #5900 `3e8a1f2` Add a consistency check `-checkblockindex` for the block chain data structures +- #5951 `7efc9cf` Make it possible to disable wallet transaction broadcast (using `-walletbroadcast=0`) +- #5911 `b6ea3bc` privacy: Stream isolation for Tor (on by default, use `-proxyrandomize=0` to disable) +- #5863 `c271304` Add autoprune functionality (`-prune=<size>`) +- #6153 `0bcf04f` Parameter interaction: disable upnp if -proxy set +- #6274 `4d9c7fe` Add option `-alerts` to opt out of alert system + +### Block and transaction handling +- #5367 `dcc1304` Do all block index writes in a batch +- #5253 `203632d` Check against MANDATORY flags prior to accepting to mempool +- #5459 `4406c3e` Reject headers that build on an invalid parent +- #5481 `055f3ae` Apply AreSane() checks to the fees from the network +- #5580 `40d65eb` Preemptively catch a few potential bugs +- #5349 `f55c5e9` Implement test for merkle tree malleability in CPartialMerkleTree +- #5564 `a89b837` clarify obscure uses of EvalScript() +- #5521 `8e4578a` Reject non-final txs even in testnet/regtest +- #5707 `6af674e` Change hardcoded character constants to descriptive named constants for db keys +- #5286 `fcf646c` Change the default maximum OP_RETURN size to 80 bytes +- #5710 `175d86e` Add more information to errors in ReadBlockFromDisk +- #5948 `b36f1ce` Use GetAncestor to compute new target +- #5959 `a0bfc69` Add additional block index consistency checks +- #6058 `7e0e7f8` autoprune minor post-merge improvements +- #5159 `2cc1372` New fee estimation code +- #6102 `6fb90d8` Implement accurate UTXO cache size accounting +- #6129 `2a82298` Bug fix for clearing fCheckForPruning +- #5947 `e9af4e6` Alert if it is very likely we are getting a bad chain +- #6203 `c00ae64` Remove P2SH coinbase flag, no longer interesting +- #5985 `37b4e42` Fix removing of orphan transactions +- #6221 `6cb70ca` Prune: Support noncontiguous block files +- #6256 `fce474c` Use best header chain timestamps to detect partitioning +- #6233 `a587606` Advance pindexLastCommonBlock for blocks in chainActive + +### P2P protocol and network code +- #5507 `844ace9` Prevent DOS attacks on in-flight data structures +- #5770 `32a8b6a` Sanitize command strings before logging them +- #5859 `dd4ffce` Add correct bool combiner for net signals +- #5876 `8e4fd0c` Add a NODE_GETUTXO service bit and document NODE_NETWORK +- #6028 `b9311fb` Move nLastTry from CAddress to CAddrInfo +- #5662 `5048465` Change download logic to allow calling getdata on inbound peers +- #5971 `18d2832` replace absolute sleep with conditional wait +- #5918 `7bf5d5e` Use equivalent PoW for non-main-chain requests +- #6059 `f026ab6` chainparams: use SeedSpec6's rather than CAddress's for fixed seeds +- #6080 `31c0bf1` Add jonasschnellis dns seeder +- #5976 `9f7809f` Reduce download timeouts as blocks arrive +- #6172 `b4bbad1` Ignore getheaders requests when not synced +- #5875 `304892f` Be stricter in processing unrequested blocks +- #6333 `41bbc85` Hardcoded seeds update June 2015 + +### Validation +- #5143 `48e1765` Implement BIP62 rule 6 +- #5713 `41e6e4c` Implement BIP66 + +### Build system +- #5501 `c76c9d2` Add mips, mipsel and aarch64 to depends platforms +- #5334 `cf87536` libbitcoinconsensus: Add pkg-config support +- #5514 `ed11d53` Fix 'make distcheck' +- #5505 `a99ef7d` Build winshutdownmonitor.cpp on Windows only +- #5582 `e8a6639` Osx toolchain update +- #5684 `ab64022` osx: bump build sdk to 10.9 +- #5695 `23ef5b7` depends: latest config.guess and config.sub +- #5509 `31dedb4` Fixes when compiling in c++11 mode +- #5819 `f8e68f7` release: use static libstdc++ and disable reduced exports by default +- #5510 `7c3fbc3` Big endian support +- #5149 `c7abfa5` Add script to verify all merge commits are signed +- #6082 `7abbb7e` qt: disable qt tests when one of the checks for the gui fails +- #6244 `0401aa2` configure: Detect (and reject) LibreSSL +- #6269 `95aca44` gitian: Use the new bitcoin-detached-sigs git repo for OSX signatures +- #6285 `ef1d506` Fix scheduler build with some boost versions. +- #6280 `25c2216` depends: fix Boost 1.55 build on GCC 5 +- #6303 `b711599` gitian: add a gitian-win-signer descriptor +- #6246 `8ea6d37` Fix build on FreeBSD +- #6282 `daf956b` fix crash on shutdown when e.g. changing -txindex and abort action +- #6354 `bdf0d94` Gitian windows signing normalization + +### Wallet +- #2340 `811c71d` Discourage fee sniping with nLockTime +- #5485 `d01bcc4` Enforce minRelayTxFee on wallet created tx and add a maxtxfee option +- #5508 `9a5cabf` Add RandAddSeedPerfmon to MakeNewKey +- #4805 `8204e19` Do not flush the wallet in AddToWalletIfInvolvingMe(..) +- #5319 `93b7544` Clean up wallet encryption code +- #5831 `df5c246` Subtract fee from amount +- #6076 `6c97fd1` wallet: fix boost::get usage with boost 1.58 +- #5511 `23c998d` Sort pending wallet transactions before reaccepting +- #6126 `26e08a1` Change default nTxConfirmTarget to 2 +- #6183 `75a4d51` Fix off-by-one error w/ nLockTime in the wallet +- #6276 `c9fd907` Fix getbalance * 0 + +### GUI +- #5219 `f3af0c8` New icons +- #5228 `bb3c75b` HiDPI (retina) support for splash screen +- #5258 `73cbf0a` The RPC Console should be a QWidget to make window more independent +- #5488 `851dfc7` Light blue icon color for regtest +- #5547 `a39aa74` New icon for the debug window +- #5493 `e515309` Adopt style colour for button icons +- #5557 `70477a0` On close of splashscreen interrupt verifyDB +- #5559 `83be8fd` Make the command-line-args dialog better +- #5144 `c5380a9` Elaborate on signverify message dialog warning +- #5489 `d1aa3c6` Optimize PNG files +- #5649 `e0cd2f5` Use text-color icons for system tray Send/Receive menu entries +- #5651 `848f55d` Coin Control: Use U+2248 "ALMOST EQUAL TO" rather than a simple tilde +- #5626 `ab0d798` Fix icon sizes and column width +- #5683 `c7b22aa` add new osx dmg background picture +- #5620 `7823598` Payment request expiration bug fix +- #5729 `9c4a5a5` Allow unit changes for read-only BitcoinAmountField +- #5753 `0f44672` Add bitcoin logo to about screen +- #5629 `a956586` Prevent amount overflow problem with payment requests +- #5830 `215475a` Don't save geometry for options and about/help window +- #5793 `d26f0b2` Honor current network when creating autostart link +- #5847 `f238add` Startup script for centos, with documentation +- #5915 `5bd3a92` Fix a static qt5 crash when using certain versions of libxcb +- #5898 `bb56781` Fix rpc console font size to flexible metrics +- #5467 `bc8535b` Payment request / server work - part 2 +- #6161 `180c164` Remove movable option for toolbar +- #6160 `0d862c2` Overviewpage: make sure warning icons gets colored + +### Tests +- #5453 `2f2d337` Add ability to run single test manually to RPC tests +- #5421 `886eb57` Test unexecuted OP_CODESEPARATOR +- #5530 `565b300` Additional rpc tests +- #5611 `37b185c` Fix spurious windows test failures after 012598880c +- #5613 `2eda47b` Fix smartfees test for change to relay policy +- #5612 `e3f5727` Fix zapwallettxes test +- #5642 `30a5b5f` Prepare paymentservertests for new unit tests +- #5784 `e3a3cd7` Fix usage of NegateSignatureS in script_tests +- #5813 `ee9f2bf` Add unit tests for next difficulty calculations +- #5855 `d7989c0` Travis: run unit tests in different orders +- #5852 `cdae53e` Reinitialize state in between individual unit tests. +- #5883 `164d7b6` tests: add a BasicTestingSetup and apply to all tests +- #5940 `446bb70` Regression test for ResendWalletTransactions +- #6052 `cf7adad` fix and enable bip32 unit test +- #6039 `734f80a` tests: Error when setgenerate is used on regtest +- #6074 `948beaf` Correct the PUSHDATA4 minimal encoding test in script_invalid.json +- #6032 `e08886d` Stop nodes after RPC tests, even with --nocleanup +- #6075 `df1609f` Add additional script edge condition tests +- #5981 `da38dc6` Python P2P testing +- #5958 `9ef00c3` Add multisig rpc tests +- #6112 `fec5c0e` Add more script edge condition tests + +### Miscellaneous +- #5457, #5506, #5952, #6047 Update libsecp256k1 +- #5437 `84857e8` Add missing CAutoFile::IsNull() check in main +- #5490 `ec20fd7` Replace uint256/uint160 with opaque blobs where possible +- #5654, #5764 Adding jonasschnelli's GPG key +- #5477 `5f04d1d` OS X 10.10: LSSharedFileListItemResolve() is deprecated +- #5679 `beff11a` Get rid of DetectShutdownThread +- #5787 `9bd8c9b` Add fanquake PGP key +- #5366 `47a79bb` No longer check osx compatibility in RenameThread +- #5689 `07f4386` openssl: abstract out OPENSSL_cleanse +- #5708 `8b298ca` Add list of implemented BIPs +- #5809 `46bfbe7` Add bitcoin-cli man page +- #5839 `86eb461` keys: remove libsecp256k1 verification until it's actually supported +- #5749 `d734d87` Help messages correctly formatted (79 chars) +- #5884 `7077fe6` BUGFIX: Stack around the variable 'rv' was corrupted +- #5849 `41259ca` contrib/init/bitcoind.openrc: Compatibility with previous OpenRC init script variables +- #5950 `41113e3` Fix locale fallback and guard tests against invalid locale settings +- #5965 `7c6bfb1` Add git-subtree-check.sh script +- #6033 `1623f6e` FreeBSD, OpenBSD thread renaming +- #6064 `b46e7c2` Several changes to mruset +- #6104 `3e2559c` Show an init message while activating best chain +- #6125 `351f73e` Clean up parsing of bool command line args +- #5964 `b4c219b` Lightweight task scheduler +- #6116 `30dc3c1` [OSX] rename Bitcoin-Qt.app to Bitcoin-Core.app +- #6168 `b3024f0` contrib/linearize: Support linearization of testnet blocks +- #6098 `7708fcd` Update Windows resource files (and add one for bitcoin-tx) +- #6159 `e1412d3` Catch errors on datadir lock and pidfile delete +- #6186 `182686c` Fix two problems in CSubnet parsing +- #6174 `df992b9` doc: add translation strings policy +- #6210 `dfdb6dd` build: disable optional use of gmp in internal secp256k1 build +- #6264 `94cd705` Remove translation for -help-debug options +- #6286 `3902c15` Remove berkeley-db4 workaround in MacOSX build docs +- #6319 `3f8fcc9` doc: update mailing list address + +Credits +======= + +Thanks to everyone who directly contributed to this release: + +- 21E14 +- Adam Weiss +- Alex Morcos +- ayeowch +- azeteki +- Ben Holden-Crowther +- bikinibabe +- BitcoinPRReadingGroup +- Blake Jakopovic +- BtcDrak +- charlescharles +- Chris Arnesen +- Ciemon +- CohibAA +- Corinne Dashjr +- Cory Fields +- Cozz Lovan +- Daira Hopwood +- Daniel Kraft +- Dave Collins +- David A. Harding +- dexX7 +- Earlz +- Eric Lombrozo +- Eric R. Schulz +- Everett Forth +- Flavien Charlon +- fsb4000 +- Gavin Andresen +- Gregory Maxwell +- Heath +- Ivan Pustogarov +- Jacob Welsh +- Jameson Lopp +- Jason Lewicki +- Jeff Garzik +- Jonas Schnelli +- Jonathan Brown +- Jorge Timón +- joshr +- jtimon +- Julian Yap +- Luca Venturini +- Luke Dashjr +- Manuel Araoz +- MarcoFalke +- Matt Bogosian +- Matt Corallo +- Micha +- Michael Ford +- Mike Hearn +- mrbandrews +- Nicolas Benoit +- paveljanik +- Pavel Janík +- Pavel Vasin +- Peter Todd +- Philip Kaufmann +- Pieter Wuille +- pstratem +- randy-waterhouse +- rion +- Rob Van Mieghem +- Ross Nicoll +- Ruben de Vries +- sandakersmann +- Shaul Kfir +- Shawn Wilkinson +- sinetek +- Suhas Daftuar +- svost +- Thomas Zander +- Tom Harding +- UdjinM6 +- Vitalii Demianets +- Wladimir J. van der Laan + +And all those who contributed additional code review and/or security research: + +- Sergio Demian Lerner + +As well as everyone that helped translating on [Transifex](https://www.transifex.com/projects/p/bitcoin/). + diff --git a/doc/release-process.md b/doc/release-process.md index 5ecb9334f5..1bfdb8fabd 100644 --- a/doc/release-process.md +++ b/doc/release-process.md @@ -6,39 +6,54 @@ Release Process * * * -###update (commit) version in sources +###first time only or for new builders, check out the source in the following directory hierarchy + cd /path/to/your/toplevel/build + git clone https://github.com/bitcoin/gitian.sigs.git + git clone https://github.com/devrandom/gitian-builder.git + git clone https://github.com/bitcoin/bitcoin.git + +###for bitcoin maintainers/release engineers, update (commit) version in sources + + pushd ./bitcoin contrib/verifysfbinaries/verify.sh doc/README* share/setup.nsi src/clientversion.h (change CLIENT_VERSION_IS_RELEASE to true) -###tag version in git +###for bitcoin maintainers/release engineers, tag version in git git tag -s v(new version, e.g. 0.8.0) -###write release notes. git shortlog helps a lot, for example: +###for bitcoin maintainers/release engineers, write release notes. git shortlog helps a lot, for example: git shortlog --no-merges v(current version, e.g. 0.7.2)..v(new version, e.g. 0.8.0) + popd * * * -###update gitian - - In order to take advantage of the new caching features in gitian, be sure to update to a recent version (`e9741525c` or later is recommended) +###update gitian, gitian.sigs, checkout bitcoin version, and perform gitian builds -###perform gitian builds - - From a directory containing the bitcoin source, gitian-builder and gitian.sigs + To ensure your gitian descriptors are accurate for direct reference for gbuild, below, run the following from a directory containing the bitcoin source: + pushd ./bitcoin export SIGNER=(your gitian key, ie bluematt, sipa, etc) export VERSION=(new version, e.g. 0.8.0) - pushd ./bitcoin git checkout v${VERSION} popd + + Ensure your gitian.sigs are up-to-date if you wish to gverify your builds against other gitian signatures: + + pushd ./gitian.sigs + git pull + popd + + Ensure your gitian-builder sources are up-to-date to take advantage of the new caching features of gitian (`e9741525c` or later is recommended) + pushd ./gitian-builder + git pull -###fetch and build inputs: (first time, or when dependency versions change) +###fetch and create inputs: (first time, or when dependency versions change) mkdir -p inputs wget -P inputs https://bitcoincore.org/cfields/osslsigncode-Backports-to-1.7.1.patch @@ -52,28 +67,44 @@ Release Process tar -C /Volumes/Xcode/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/ -czf MacOSX10.9.sdk.tar.gz MacOSX10.9.sdk -###Optional: Seed the Gitian sources cache +###Optional: Seed the Gitian sources cache and offline git repositories - By default, gitian will fetch source files as needed. For offline builds, they can be fetched ahead of time: +By default, gitian will fetch source files as needed. To cache them ahead of time: make -C ../bitcoin/depends download SOURCES_PATH=`pwd`/cache/common - Only missing files will be fetched, so this is safe to re-run for each build. +Only missing files will be fetched, so this is safe to re-run for each build. + +Clone the detached-sigs repository: + + popd + git clone https://github.com/bitcoin/bitcoin-detached-sigs.git + pushd ./bitcoin-builder + +NOTE: Offline builds must use the --url flag to ensure gitian fetches only from local URLs. +For example: ./bin/bguild --url bitcoin=/path/to/bitcoin,signature=/path/to/sigs {rest of arguments} +The following gbuild invocations DO NOT DO THIS by default. -###Build Bitcoin Core for Linux, Windows, and OS X: +###Build (and optionally verify) Bitcoin Core for Linux, Windows, and OS X: ./bin/gbuild --commit bitcoin=v${VERSION} ../bitcoin/contrib/gitian-descriptors/gitian-linux.yml ./bin/gsign --signer $SIGNER --release ${VERSION}-linux --destination ../gitian.sigs/ ../bitcoin/contrib/gitian-descriptors/gitian-linux.yml + ./bin/gverify -v -d ../gitian.sigs/ -r ${VERSION}-linux ../bitcoin/contrib/gitian-descriptors/gitian-linux.yml mv build/out/bitcoin-*.tar.gz build/out/src/bitcoin-*.tar.gz ../ + ./bin/gbuild --commit bitcoin=v${VERSION} ../bitcoin/contrib/gitian-descriptors/gitian-win.yml ./bin/gsign --signer $SIGNER --release ${VERSION}-win-unsigned --destination ../gitian.sigs/ ../bitcoin/contrib/gitian-descriptors/gitian-win.yml + ./bin/gverify -v -d ../gitian.sigs/ -r ${VERSION}-win-unsigned ../bitcoin/contrib/gitian-descriptors/gitian-win.yml mv build/out/bitcoin-*-win-unsigned.tar.gz inputs/bitcoin-win-unsigned.tar.gz mv build/out/bitcoin-*.zip build/out/bitcoin-*.exe ../ + ./bin/gbuild --commit bitcoin=v${VERSION} ../bitcoin/contrib/gitian-descriptors/gitian-osx.yml ./bin/gsign --signer $SIGNER --release ${VERSION}-osx-unsigned --destination ../gitian.sigs/ ../bitcoin/contrib/gitian-descriptors/gitian-osx.yml + ./bin/gverify -v -d ../gitian.sigs/ -r ${VERSION}-osx-unsigned ../bitcoin/contrib/gitian-descriptors/gitian-osx.yml mv build/out/bitcoin-*-osx-unsigned.tar.gz inputs/bitcoin-osx-unsigned.tar.gz mv build/out/bitcoin-*.tar.gz build/out/bitcoin-*.dmg ../ popd + Build output expected: 1. source tarball (bitcoin-${VERSION}.tar.gz) @@ -98,19 +129,21 @@ Commit your signature to gitian.sigs: Once the Windows/OSX builds each have 3 matching signatures, they will be signed with their respective release keys. Detached signatures will then be committed to the bitcoin-detached-sigs repository, which can be combined with the unsigned apps to create signed binaries. - Create the signed OSX binary: + Create (and optionally verify) the signed OSX binary: pushd ./gitian-builder ./bin/gbuild -i --commit signature=v${VERSION} ../bitcoin/contrib/gitian-descriptors/gitian-osx-signer.yml ./bin/gsign --signer $SIGNER --release ${VERSION}-osx-signed --destination ../gitian.sigs/ ../bitcoin/contrib/gitian-descriptors/gitian-osx-signer.yml + ./bin/gverify -v -d ../gitian.sigs/ -r ${VERSION}-osx-signed ../bitcoin/contrib/gitian-descriptors/gitian-osx-signer.yml mv build/out/bitcoin-osx-signed.dmg ../bitcoin-${VERSION}-osx.dmg popd - Create the signed Windows binaries: + Create (and optionally verify) the signed Windows binaries: pushd ./gitian-builder ./bin/gbuild -i --commit signature=v${VERSION} ../bitcoin/contrib/gitian-descriptors/gitian-win-signer.yml ./bin/gsign --signer $SIGNER --release ${VERSION}-win-signed --destination ../gitian.sigs/ ../bitcoin/contrib/gitian-descriptors/gitian-win-signer.yml + ./bin/gverify -v -d ../gitian.sigs/ -r ${VERSION}-win-signed ../bitcoin/contrib/gitian-descriptors/gitian-win-signer.yml mv build/out/bitcoin-*win64-setup.exe ../bitcoin-${VERSION}-win64-setup.exe mv build/out/bitcoin-*win32-setup.exe ../bitcoin-${VERSION}-win32-setup.exe popd diff --git a/doc/shared-libraries.md b/doc/shared-libraries.md index 1fc32112ce..f1448f7258 100644 --- a/doc/shared-libraries.md +++ b/doc/shared-libraries.md @@ -40,3 +40,4 @@ The interface is defined in the C header `bitcoinconsensus.h` located in `src/s ### Example Implementations - [NBitcoin](https://github.com/NicolasDorier/NBitcoin/blob/master/NBitcoin/Script.cs#L814) (.NET Bindings) - [node-libbitcoinconsensus](https://github.com/bitpay/node-libbitcoinconsensus) (Node.js Bindings) +- [java-libbitcoinconsensus](https://github.com/dexX7/java-libbitcoinconsensus) (Java Bindings) diff --git a/doc/tor.md b/doc/tor.md index 560f71fa27..f8b94d19d1 100644 --- a/doc/tor.md +++ b/doc/tor.md @@ -70,9 +70,14 @@ In a typical situation, where you're only reachable via Tor, this should suffice ./bitcoind -proxy=127.0.0.1:9050 -externalip=57qr3yd1nyntf5k.onion -listen -(obviously, replace the Onion address with your own). If you don't care too much -about hiding your node, and want to be reachable on IPv4 as well, additionally -specify: +(obviously, replace the Onion address with your own). It should be noted that you still +listen on all devices and another node could establish a clearnet connection, when knowing +your address. To mitigate this, additionally bind the address of your Tor proxy: + + ./bitcoind ... -bind=127.0.0.1 + +If you don't care too much about hiding your node, and want to be reachable on IPv4 +as well, use `discover` instead: ./bitcoind ... -discover diff --git a/doc/zmq.md b/doc/zmq.md new file mode 100644 index 0000000000..fd04f6d9f0 --- /dev/null +++ b/doc/zmq.md @@ -0,0 +1,98 @@ +# Block and Transaction Broadcasting With ZeroMQ + +[ZeroMQ](http://zeromq.org/) is a lightweight wrapper around TCP +connections, inter-process communications, and shared-memory, +providing various message-oriented semantics such as publish/subcribe, +request/reply, and push/pull. + +The Bitcoin Core daemon can be configured to act as a trusted "border +router", implementing the bitcoin wire protocol and relay, making +consensus decisions, maintaining the local blockchain database, +broadcasting locally generated transactions into the network, and +providing a queryable RPC interface to interact on a polled basis for +requesting blockchain related data. However, there exists only a +limited service to notify external software of events like the arrival +of new blocks or transactions. + +The ZeroMQ facility implements a notification interface through a +set of specific notifiers. Currently there are notifiers that publish +blocks and transactions. This read-only facility requires only the +connection of a corresponding ZeroMQ subscriber port in receiving +software; it is not authenticated nor is there any two-way protocol +involvement. Therefore, subscribers should validate the received data +since it may be out of date, incomplete or even invalid. + +ZeroMQ sockets are self-connecting and self-healing; that is, connects +made between two endpoints will be automatically restored after an +outage, and either end may be freely started or stopped in any order. + +Because ZeroMQ is message oriented, subscribers receive transactions +and blocks all-at-once and do not need to implement any sort of +buffering or reassembly. + +## Prerequisites + +The ZeroMQ feature in Bitcoin Core uses only a very small part of the +ZeroMQ C API, and is thus compatible with any version of ZeroMQ +from 2.1 onward, including all versions in the 3.x and 4.x release +series. Typically, it is packaged by distributions as something like +*libzmq-dev*. + +The C++ wrapper for ZeroMQ is *not* needed. + +## Enabling + +By default, the ZeroMQ port functionality is enabled. Two steps are +required to enable--compiling in the ZeroMQ code, and configuring +runtime operation on the command-line or configuration file. + + $ ./configure --enable-zmq (other options) + +This will produce a binary that is capable of providing the ZeroMQ +facility, but will not do so until also configured properly. + +## Usage + +Currently, the following notifications are supported: + + -zmqpubhashtx=address + -zmqpubhashblock=address + -zmqpubrawblock=address + -zmqpubrawtx=address + +The socket type is PUB and the address must be a valid ZeroMQ +socket address. The same address can be used in more than one notification. + +For instance: + + $ bitcoind -zmqpubhashtx=tcp://127.0.0.1:28332 -zmqpubrawtx=ipc:///tmp/bitcoind.tx.raw + +Each PUB notification has a topic and body, where the header +corresponds to the notification type. For instance, for the notification +`-zmqpubhashtx` the topic is `hashtx` (no null terminator) and the body is the +hexadecimal transaction hash (32 bytes). + +These options can also be provided in bitcoin.conf. + +ZeroMQ endpoint specifiers for TCP (and others) are documented in the +[ZeroMQ API](http://api.zeromq.org). + +Client side, then, the ZeroMQ subscriber socket must have the +ZMQ_SUBSCRIBE option set to one or either of these prefixes (for instance, just `hash`); without +doing so will result in no messages arriving. Please see `contrib/zmq/zmq_sub.py` +for a working example. + +## Remarks + +From the perspective of bitcoind, the ZeroMQ socket is write-only; PUB +sockets don't even have a read function. Thus, there is no state +introduced into bitcoind directly. Furthermore, no information is +broadcast that wasn't already received from the public P2P network. + +No authentication or authorization is done on connecting clients; it +is assumed that the ZeroMQ port is exposed only to trusted entities, +using other means such as firewalling. + +Note that when the block chain tip changes, a reorganisation may occur and just +the tip will be notified. It is up to the subscriber to retrieve the chain +from the last known block to the new tip. |
