diff options
Diffstat (limited to 'doc')
56 files changed, 1047 insertions, 246 deletions
diff --git a/doc/README.md b/doc/README.md index 7b0c39d383..08fd724355 100644 --- a/doc/README.md +++ b/doc/README.md @@ -7,7 +7,7 @@ Setup Running --------------------- -The following are some helpful notes on how to run Bitcoin on your native platform. +The following are some helpful notes on how to run Bitcoin on your native platform. ### Unix @@ -26,7 +26,7 @@ Unpack the files into a directory and run: Unpack the files into a directory, and then run bitcoin-qt.exe. -### OSX +### OS X Drag Bitcoin-Qt to your applications folder, and then run Bitcoin-Qt. @@ -41,7 +41,7 @@ Building --------------------- The following are developer notes on how to build Bitcoin on your native platform. They are not complete guides, but include notes on the necessary libraries, compile flags, etc. -- [OSX Build Notes](build-osx.md) +- [OS X Build Notes](build-osx.md) - [Unix Build Notes](build-unix.md) - [Gitian Building Guide](gitian-building.md) @@ -64,7 +64,8 @@ The Bitcoin repo's [root README](https://github.com/bitcoin/bitcoin/blob/master/ ### Resources * Discuss on the [BitcoinTalk](https://bitcointalk.org/) forums, in the [Development & Technical Discussion board](https://bitcointalk.org/index.php?board=6.0). -* Discuss on [#bitcoin-dev](http://webchat.freenode.net/?channels=bitcoin) on Freenode. If you don't have an IRC client use [webchat here](http://webchat.freenode.net/?channels=bitcoin-dev). +* Discuss project-specific development on #bitcoin-core-dev on Freenode. If you don't have an IRC client use [webchat here](http://webchat.freenode.net/?channels=bitcoin-core-dev). +* Discuss general Bitcoin development on #bitcoin-dev on Freenode. If you don't have an IRC client use [webchat here](http://webchat.freenode.net/?channels=bitcoin-dev). ### Miscellaneous - [Assets Attribution](assets-attribution.md) diff --git a/doc/README_osx.txt b/doc/README_osx.txt index a572c7a241..f589bfc676 100644 --- a/doc/README_osx.txt +++ b/doc/README_osx.txt @@ -1,12 +1,12 @@ -Deterministic OSX Dmg Notes. +Deterministic OS X Dmg Notes. -Working OSX DMGs are created in Linux by combining a recent clang, +Working OS X DMGs are created in Linux by combining a recent clang, the Apple's binutils (ld, ar, etc), and DMG authoring tools. Apple uses clang extensively for development and has upstreamed the necessary functionality so that a vanilla clang can take advantage. It supports the use of -F, -target, -mmacosx-version-min, and --sysroot, which are all necessary -when building for OSX. A pre-compiled version of 3.2 is used because it was not +when building for OS X. A pre-compiled version of 3.2 is used because it was not available in the Precise repositories at the time this work was started. In the future, it can be switched to use system packages instead. @@ -29,18 +29,18 @@ originally done in toolchain4. To complicate things further, all builds must target an Apple SDK. These SDKs are free to download, but not redistributable. -To obtain it, register for a developer account, then download the XCode 6.1.1 dmg: +To obtain it, register for a developer account, then download the Xcode 6.1.1 dmg: https://developer.apple.com/devcenter/download.action?path=/Developer_Tools/xcode_6.1.1/xcode_6.1.1.dmg This file is several gigabytes in size, but only a single directory inside is needed: Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk Unfortunately, the usual linux tools (7zip, hpmount, loopback mount) are incapable of opening this file. -To create a tarball suitable for gitian input, mount the dmg in OSX, then create it with: +To create a tarball suitable for Gitian input, mount the dmg in OS X, then create it with: $ tar -C /Volumes/Xcode/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/ -czf MacOSX10.9.sdk.tar.gz MacOSX10.9.sdk -The gitian descriptors build 2 sets of files: Linux tools, then Apple binaries +The Gitian descriptors build 2 sets of files: Linux tools, then Apple binaries which are created using these tools. The build process has been designed to avoid including the SDK's files in Gitian's outputs. All interim tarballs are fully deterministic and may be freely redistributed. @@ -64,20 +64,20 @@ Ideally, the creation could be fixed and genisoimage would no longer be necessar Background images and other features can be added to DMG files by inserting a .DS_Store before creation. The easiest way to create this file is to build a -DMG without one, move it to a device running OSX, customize the layout, then +DMG without one, move it to a device running OS X, customize the layout, then grab the .DS_Store file for later use. That is the approach taken here. -As of OSX Mavericks (10.9), using an Apple-blessed key to sign binaries is a +As of OS X Mavericks (10.9), using an Apple-blessed key to sign binaries is a requirement in order to satisfy the new Gatekeeper requirements. Because this private key cannot be shared, we'll have to be a bit creative in order for the build process to remain somewhat deterministic. Here's how it works: -- Builders use gitian to create an unsigned release. This outputs an unsigned +- Builders use Gitian to create an unsigned release. This outputs an unsigned dmg which users may choose to bless and run. It also outputs an unsigned app structure in the form of a tarball, which also contains all of the tools that have been previously (deterministically) built in order to create a final dmg. - The Apple keyholder uses this unsigned app to create a detached signature, using the script that is also included there. -- Builders feed the unsigned app + detached signature back into gitian. It +- Builders feed the unsigned app + detached signature back into Gitian. It uses the pre-built tools to recombine the pieces into a deterministic dmg. diff --git a/doc/assets-attribution.md b/doc/assets-attribution.md index b5a033dd79..2dd930d6a4 100644 --- a/doc/assets-attribution.md +++ b/doc/assets-attribution.md @@ -1,67 +1 @@ -The following is a list of assets used in the bitcoin source and their proper attribution. - -[Typicons/Stephen Hutchings](http://typicons.com) ------------------------ - -### Info -* Icon Pack: Typicons (http://typicons.com) -* Designer: Stephen Hutchings (and more) -* License: MIT -* Site: [https://github.com/stephenhutchings/typicons.font](https://github.com/stephenhutchings/typicons.font) - -### Assets Used - src/qt/res/icons/add.png - src/qt/res/icons/address-book.png, - src/qt/res/icons/configure.png - src/qt/res/icons/debugwindow.png - src/qt/res/icons/edit.png, - src/qt/res/icons/exitcopy.png - src/qt/res/icons/editpaste.png, - src/qt/res/icons/export.png - src/qt/res/icons/eye.png, - src/qt/res/icons/filesave.png - src/qt/res/icons/history.png, - src/qt/res/icons/info.png - src/qt/res/icons/key.png, - src/qt/res/icons/lock_*.png - src/qt/res/icons/open.png, - src/qt/res/icons/overview.png - src/qt/res/icons/quit.png, - src/qt/res/icons/receive.png - src/qt/res/icons/remove.png, - src/qt/res/icons/send.png - src/qt/res/icons/synced.png, - src/qt/res/icons/transaction*.png - src/qt/res/icons/tx_output.png, - src/qt/res/icons/warning.png - -Other ------------------------ - -### Info -* Designer: Jonas Schnelli, Bitboy, Stephen Hutchings, Marco Falke -* Bitcoin Icon: Based on the original bitcoin logo from Bitboy -* Network connection icon: Inspired by flow-merge.svg from Stephen Hutchings -* Some icons are based on Stephan Hutchings Typicons -* License: MIT - -### Assets Used - src/qt/res/icons/about.png - src/qt/res/icons/about_qt.png, - src/qt/res/icons/bitcoin.icns - src/qt/res/icons/bitcoin.ico, - src/qt/res/icons/bitcoin.png - src/qt/res/icons/clock*.png, - src/qt/res/icons/connect*.png - src/qt/res/icons/eye_minus.png, - src/qt/res/icons/eye_plus.png - src/qt/res/icons/verify.png, - src/qt/res/icons/tx_inout.png - src/qt/res/icons/tx_input.png, - src/qt/res/src/bitcoin.svg, - src/qt/res/src/clock*.svg - src/qt/res/src/connect*.svg, - src/qt/res/src/mine.svg - src/qt/res/src/qt.svg - src/qt/res/src/tx*.svg, - src/qt/res/src/verify.svg +The list of assets used in the bitcoin source and their attribution can now be found in [contrib/debian/copyright](../contrib/debian/copyright). diff --git a/doc/bips.md b/doc/bips.md index 90e98ed419..c780e2dde0 100644 --- a/doc/bips.md +++ b/doc/bips.md @@ -1,4 +1,4 @@ -BIPs that are implemented by Bitcoin Core (up-to-date up to **v0.10.0**): +BIPs that are implemented by Bitcoin Core (up-to-date up to **v0.12.0**): * [`BIP 11`](https://github.com/bitcoin/bips/blob/master/bip-0011.mediawiki): Multisig outputs are standard since **v0.6.0** ([PR #669](https://github.com/bitcoin/bitcoin/pull/669)). * [`BIP 13`](https://github.com/bitcoin/bips/blob/master/bip-0013.mediawiki): The address format for P2SH addresses has been implemented since **v0.6.0** ([PR #669](https://github.com/bitcoin/bitcoin/pull/669)). @@ -14,5 +14,7 @@ BIPs that are implemented by Bitcoin Core (up-to-date up to **v0.10.0**): * [`BIP 37`](https://github.com/bitcoin/bips/blob/master/bip-0037.mediawiki): The bloom filtering for transaction relaying, partial merkle trees for blocks, and the protocol version bump to 70001 (enabling low-bandwidth SPV clients) has been implemented since **v0.8.0** ([PR #1795](https://github.com/bitcoin/bitcoin/pull/1795)). * [`BIP 42`](https://github.com/bitcoin/bips/blob/master/bip-0042.mediawiki): The bug that would have caused the subsidy schedule to resume after block 13440000 was fixed in **v0.9.2** ([PR #3842](https://github.com/bitcoin/bitcoin/pull/3842)). * [`BIP 61`](https://github.com/bitcoin/bips/blob/master/bip-0061.mediawiki): The 'reject' protocol message (and the protocol version bump to 70002) was added in **v0.9.0** ([PR #3185](https://github.com/bitcoin/bitcoin/pull/3185)). +* [`BIP 65`](https://github.com/bitcoin/bips/blob/master/bip-0065.mediawiki): The CHECKLOCKTIMEVERIFY softfork was merged in **v0.12.0** ([PR #6351](https://github.com/bitcoin/bitcoin/pull/6351)), and backported to **v0.11.2** and **v0.10.4**. Mempool-only CLTV was added in [PR #6124](https://github.com/bitcoin/bitcoin/pull/6124). * [`BIP 66`](https://github.com/bitcoin/bips/blob/master/bip-0066.mediawiki): The strict DER rules and associated version 3 blocks have been implemented since **v0.10.0** ([PR #5713](https://github.com/bitcoin/bitcoin/pull/5713)). * [`BIP 70`](https://github.com/bitcoin/bips/blob/master/bip-0070.mediawiki) [`71`](https://github.com/bitcoin/bips/blob/master/bip-0071.mediawiki) [`72`](https://github.com/bitcoin/bips/blob/master/bip-0072.mediawiki): Payment Protocol support has been available in Bitcoin Core GUI since **v0.9.0** ([PR #5216](https://github.com/bitcoin/bitcoin/pull/5216)). +* [`BIP 111`](https://github.com/bitcoin/bips/blob/master/bip-0111.mediawiki): `NODE_BLOOM` service bit added, but only enforced for peer versions `>=70011` as of **v0.12.0** ([PR #6579](https://github.com/bitcoin/bitcoin/pull/6579)). diff --git a/doc/build-openbsd.md b/doc/build-openbsd.md new file mode 100644 index 0000000000..d923301467 --- /dev/null +++ b/doc/build-openbsd.md @@ -0,0 +1,162 @@ +OpenBSD build guide +====================== +(updated for OpenBSD 5.7) + +This guide describes how to build bitcoind and command-line utilities on OpenBSD. + +As OpenBSD is most common as a server OS, we will not bother with the GUI. + +Preparation +------------- + +Run the following as root to install the base dependencies for building: + +```bash +pkg_add gmake libtool libevent +pkg_add autoconf # (select highest version, e.g. 2.69) +pkg_add automake # (select highest version, e.g. 1.15) +pkg_add python # (select version 2.7.x, not 3.x) +ln -sf /usr/local/bin/python2.7 /usr/local/bin/python2 +``` + +The default C++ compiler that comes with OpenBSD 5.7 is g++ 4.2. This version is old (from 2007), and is not able to compile the current version of Bitcoin Core. It is possible to patch it up to compile, but with the planned transition to C++11 this is a losing battle. So here we will be installing a newer compiler. + +GCC +------- + +You can install a newer version of gcc with: + +```bash +pkg_add g++ # (select newest 4.x version, e.g. 4.9.2) +``` + +This compiler will not overwrite the system compiler, it will be installed as `egcc` and `eg++` in `/usr/local/bin`. + +### Building boost + +Do not use `pkg_add boost`! The boost version installed thus is compiled using the `g++` compiler not `eg++`, which will result in a conflict between `/usr/local/lib/libestdc++.so.XX.0` and `/usr/lib/libstdc++.so.XX.0`, resulting in a test crash: + + test_bitcoin:/usr/lib/libstdc++.so.57.0: /usr/local/lib/libestdc++.so.17.0 : WARNING: symbol(_ZN11__gnu_debug17_S_debug_me ssagesE) size mismatch, relink your program + ... + Segmentation fault (core dumped) + +This makes it necessary to build boost, or at least the parts used by Bitcoin Core, manually: + +``` +# Pick some path to install boost to, here we create a directory within the bitcoin directory +BITCOIN_ROOT=$(pwd) +BOOST_PREFIX="${BITCOIN_ROOT}/boost" +mkdir -p $BOOST_PREFIX + +# Fetch the source and verify that it is not tampered with +wget http://heanet.dl.sourceforge.net/project/boost/boost/1.59.0/boost_1_59_0.tar.bz2 +echo '727a932322d94287b62abb1bd2d41723eec4356a7728909e38adb65ca25241ca boost_1_59_0.tar.bz2' | sha256 -c +# MUST output: (SHA256) boost_1_59_0.tar.bz2: OK +tar -xjf boost_1_59_0.tar.bz2 + +# Boost 1.59 needs two small patches for OpenBSD +cd boost_1_59_0 +# Also here: https://gist.githubusercontent.com/laanwj/bf359281dc319b8ff2e1/raw/92250de8404b97bb99d72ab898f4a8cb35ae1ea3/patch-boost_test_impl_execution_monitor_ipp.patch +patch -p0 < /usr/ports/devel/boost/patches/patch-boost_test_impl_execution_monitor_ipp +# https://github.com/boostorg/filesystem/commit/90517e459681790a091566dce27ca3acabf9a70c +sed 's/__OPEN_BSD__/__OpenBSD__/g' < libs/filesystem/src/path.cpp > libs/filesystem/src/path.cpp.tmp +mv libs/filesystem/src/path.cpp.tmp libs/filesystem/src/path.cpp + +# Build w/ minimum configuration necessary for bitcoin +echo 'using gcc : : eg++ : <cxxflags>"-fvisibility=hidden -fPIC" <linkflags>"" <archiver>"ar" <striper>"strip" <ranlib>"ranlib" <rc>"" : ;' > user-config.jam +config_opts="runtime-link=shared threadapi=pthread threading=multi link=static variant=release --layout=tagged --build-type=complete --user-config=user-config.jam -sNO_BZIP2=1" +./bootstrap.sh --without-icu --with-libraries=chrono,filesystem,program_options,system,thread,test +./b2 -d2 -j2 -d1 ${config_opts} --prefix=${BOOST_PREFIX} stage +./b2 -d0 -j4 ${config_opts} --prefix=${BOOST_PREFIX} install +``` + +### Building BerkeleyDB + +BerkeleyDB is only necessary for the wallet functionality. To skip this, pass `--disable-wallet` to `./configure`. + +See "Berkeley DB" in [build_unix.md](build_unix.md) for instructions on how to build BerkeleyDB 4.8. +You cannot use the BerkeleyDB library from ports, for the same reason as boost above (g++/libstd++ incompatibility). + +```bash +# Pick some path to install BDB to, here we create a directory within the bitcoin directory +BITCOIN_ROOT=$(pwd) +BDB_PREFIX="${BITCOIN_ROOT}/db4" +mkdir -p $BDB_PREFIX + +# Fetch the source and verify that it is not tampered with +wget 'http://download.oracle.com/berkeley-db/db-4.8.30.NC.tar.gz' +echo '12edc0df75bf9abd7f82f821795bcee50f42cb2e5f76a6a281b85732798364ef db-4.8.30.NC.tar.gz' | sha256 -c +# MUST output: (SHA256) db-4.8.30.NC.tar.gz: OK +tar -xzf db-4.8.30.NC.tar.gz + +# Build the library and install to specified prefix +cd db-4.8.30.NC/build_unix/ +# Note: Do a static build so that it can be embedded into the executable, instead of having to find a .so at runtime +../dist/configure --enable-cxx --disable-shared --with-pic --prefix=$BDB_PREFIX CC=egcc CXX=eg++ CPP=ecpp +make install +``` + +### Building Bitcoin Core + +**Important**: use `gmake`, not `make`. The non-GNU `make` will exit with a horrible error. + +Preparation: +```bash +export AUTOCONF_VERSION=2.69 # replace this with the autoconf version that you installed +export AUTOMAKE_VERSION=1.15 # replace this with the automake version that you installed +./autogen.sh +``` +Make sure `BDB_PREFIX` and `BOOST_PREFIX` are set to the appropriate paths from the above steps. + +To configure with wallet: +```bash +./configure --with-gui=no --with-boost=$BOOST_PREFIX \ + CC=egcc CXX=eg++ CPP=ecpp \ + LDFLAGS="-L${BDB_PREFIX}/lib/" CPPFLAGS="-I${BDB_PREFIX}/include/" +``` + +To configure without wallet: +```bash +./configure --disable-wallet --with-gui=no --with-boost=$BOOST_PREFIX \ + CC=egcc CXX=eg++ CPP=ecpp +``` + +Build and run the tests: +```bash +gmake +gmake check +``` + +Clang (not currently working) +------------------------------ + +Using a newer g++ results in linking the new code to a new libstdc++. +Libraries built with the old g++, will still import the old library. +This gives conflicts, necessitating rebuild of all C++ dependencies of the application. + +With clang this can - at least theoretically - be avoided because it uses the +base system's libstdc++. + +```bash +pkg_add llvm boost +``` + +```bash +./configure --disable-wallet --with-gui=no CC=clang CXX=clang++ +gmake +``` + +However, this does not appear to work. Compilation succeeds, but link fails +with many 'local symbol discarded' errors: + + local symbol 150: discarded in section `.text._ZN10tinyformat6detail14FormatIterator6finishEv' from libbitcoin_util.a(libbitcoin_util_a-random.o) + local symbol 151: discarded in section `.text._ZN10tinyformat6detail14FormatIterator21streamStateFromFormatERSoRjPKcii' from libbitcoin_util.a(libbitcoin_util_a-random.o) + local symbol 152: discarded in section `.text._ZN10tinyformat6detail12convertToIntIA13_cLb0EE6invokeERA13_Kc' from libbitcoin_util.a(libbitcoin_util_a-random.o) + +According to similar reported errors this is a binutils (ld) issue in 2.15, the +version installed by OpenBSD 5.7: + +- http://openbsd-archive.7691.n7.nabble.com/UPDATE-cppcheck-1-65-td248900.html +- https://llvm.org/bugs/show_bug.cgi?id=9758 + +There is no known workaround for this. diff --git a/doc/build-osx.md b/doc/build-osx.md index dc319dd1c4..69c401b751 100644 --- a/doc/build-osx.md +++ b/doc/build-osx.md @@ -1,6 +1,6 @@ Mac OS X Build Instructions and Notes ==================================== -This guide will show you how to build bitcoind (headless client) for OSX. +This guide will show you how to build bitcoind (headless client) for OS X. Notes ----- @@ -13,8 +13,8 @@ built-in one is located in `/Applications/Utilities`. Preparation ----------- -You need to install XCode with all the options checked so that the compiler -and everything is available in /usr not just /Developer. XCode should be +You need to install Xcode with all the options checked so that the compiler +and everything is available in /usr not just /Developer. Xcode should be available on your OS X installation media, but if not, you can get the current version from https://developer.apple.com/xcode/. If you install Xcode 4.3 or later, you'll need to install its command line tools. This can @@ -32,13 +32,13 @@ Instructions: Homebrew #### Install dependencies using Homebrew - brew install autoconf automake berkeley-db4 libtool boost miniupnpc openssl pkg-config protobuf qt5 + brew install autoconf automake berkeley-db4 libtool boost miniupnpc openssl pkg-config protobuf qt5 libevent NOTE: Building with Qt4 is still supported, however, could result in a broken UI. As such, building with Qt5 is recommended. ### Building `bitcoind` -1. Clone the github tree to get the source code and go into the directory. +1. Clone the GitHub tree to get the source code and go into the directory. git clone https://github.com/bitcoin/bitcoin.git cd bitcoin @@ -62,7 +62,7 @@ Use Qt Creator as IDE You can use Qt Creator as IDE, for debugging and for manipulating forms, etc. Download Qt Creator from http://www.qt.io/download/. Download the "community edition" and only install Qt Creator (uncheck the rest during the installation process). -1. Make sure you installed everything through homebrew mentioned above +1. Make sure you installed everything through Homebrew mentioned above 2. Do a proper ./configure --with-gui=qt5 --enable-debug 3. In Qt Creator do "New Project" -> Import Project -> Import Existing Project 4. Enter "bitcoin-qt" as project name, enter src/qt as location @@ -70,7 +70,7 @@ Download Qt Creator from http://www.qt.io/download/. Download the "community edi 6. Confirm the "summary page" 7. In the "Projects" tab select "Manage Kits..." 8. Select the default "Desktop" kit and select "Clang (x86 64bit in /usr/bin)" as compiler -9. Select LLDB as debugger (you might need to set the path to your installtion) +9. Select LLDB as debugger (you might need to set the path to your installation) 10. Start debugging with Qt Creator Creating a release build diff --git a/doc/build-unix.md b/doc/build-unix.md index 92aed7725e..a9a0028c4a 100644 --- a/doc/build-unix.md +++ b/doc/build-unix.md @@ -1,6 +1,8 @@ UNIX BUILD NOTES ==================== -Some notes on how to build Bitcoin in Unix. +Some notes on how to build Bitcoin in Unix. + +(for OpenBSD specific instructions, see [build-openbsd.md](build-openbsd.md)) Note --------------------- @@ -31,8 +33,9 @@ These dependencies are required: Library | Purpose | Description ------------|------------------|---------------------- - libssl | SSL Support | Secure communications - libboost | Boost | C++ Library + libssl | Crypto | Random Number Generation, Elliptic Curve Cryptography + libboost | Utility | Library for threading, data structures, etc + libevent | Networking | OS independent asynchronous networking Optional dependencies: @@ -43,6 +46,7 @@ Optional dependencies: qt | GUI | GUI toolkit (only needed when GUI enabled) protobuf | Payments in GUI | Data interchange format used for payment protocol (only needed when GUI enabled) libqrencode | QR codes in GUI | Optional for generating QR codes (only needed when GUI enabled) + libzmq3 | ZMQ notification | Optional, allows generating ZMQ notifications (requires ZMQ version >= 4.x) For the versions used in the release, see [release-process.md](release-process.md) under *Fetch and build inputs*. @@ -57,8 +61,8 @@ Dependency Build Instructions: Ubuntu & Debian ---------------------------------------------- Build requirements: - sudo apt-get install build-essential libtool autotools-dev autoconf pkg-config libssl-dev - + sudo apt-get install build-essential libtool autotools-dev autoconf pkg-config libssl-dev libevent-dev + For Ubuntu 12.04 and later or Debian 7 and later libboost-all-dev has to be installed: sudo apt-get install libboost-all-dev @@ -80,6 +84,11 @@ Optional: sudo apt-get install libminiupnpc-dev (see --with-miniupnpc and --enable-upnp-default) +ZMQ dependencies: + + sudo apt-get install libzmq3-dev (provides ZMQ API 4.x) + + Dependencies for the GUI: Ubuntu & Debian ----------------------------------------- @@ -154,7 +163,8 @@ make install # Configure Bitcoin Core to use our own-built instance of BDB cd $BITCOIN_ROOT -./configure (other args...) LDFLAGS="-L${BDB_PREFIX}/lib/" CPPFLAGS="-I${BDB_PREFIX}/include/" +./autogen.sh +./configure LDFLAGS="-L${BDB_PREFIX}/lib/" CPPFLAGS="-I${BDB_PREFIX}/include/" # (other args...) ``` **Note**: You only need Berkeley DB if the wallet is enabled (see the section *Disable-Wallet mode* below). @@ -227,4 +237,3 @@ In this case there is no dependency on Berkeley DB 4.8. Mining is also possible in disable-wallet mode, but only using the `getblocktemplate` RPC call not `getwork`. - diff --git a/doc/developer-notes.md b/doc/developer-notes.md index 7d3d78adfc..7fe292f1f8 100644 --- a/doc/developer-notes.md +++ b/doc/developer-notes.md @@ -1,5 +1,5 @@ -Coding -==================== +Developer Notes +=============== Various coding styles have been used during the history of the codebase, and the result is not very consistent. However, we're now trying to converge to @@ -57,7 +57,7 @@ As Doxygen recognizes the comments by the delimiters (`/**` and `*/` in this cas To describe a class use the same construct above the class definition: ```c++ -/** +/** * Alerts are for notifying old versions if they become too obsolete and * need to upgrade. The message is displayed in the status bar. * @see GetWarnings() @@ -172,15 +172,35 @@ Threads - Shutdown : Does an orderly shutdown of everything. -Pull Request Terminology ------------------------- +Ignoring IDE/editor files +-------------------------- + +In closed-source environments in which everyone uses the same IDE it is common +to add temporary files it produces to the project-wide `.gitignore` file. + +However, in open source software such as Bitcoin Core, where everyone uses +their own editors/IDE/tools, it is less common. Only you know what files your +editor produces and this may change from version to version. The canonical way +to do this is thus to create your local gitignore. Add this to `~/.gitconfig`: + +``` +[core] + excludesfile = /home/.../.gitignore_global +``` -Concept ACK - Agree with the idea and overall direction, but have neither reviewed nor tested the code changes. +(alternatively, type the command `git config --global core.excludesfile ~/.gitignore_global` +on a terminal) -utACK (untested ACK) - Reviewed and agree with the code changes but haven't actually tested them. +Then put your favourite tool's temporary filenames in that file, e.g. +``` +# NetBeans +nbproject/ +``` -Tested ACK - Reviewed the code changes and have verified the functionality or bug fix. +Another option is to create a per-repository excludes file `.git/info/exclude`. +These are not committed but apply only to one repository. -ACK - A loose ACK can be confusing. It's best to avoid them unless it's a documentation/comment only change in which case there is nothing to test/verify; therefore the tested/untested distinction is not there. +If a set of tools is used by the build system or scripts the repository (for +example, lcov) it is perfectly acceptable to add its files to `.gitignore` +and commit them. -NACK - Disagree with the code changes/concept. Should be accompanied by an explanation. diff --git a/doc/dnsseed-policy.md b/doc/dnsseed-policy.md index 814ae3876a..55a5c28258 100644 --- a/doc/dnsseed-policy.md +++ b/doc/dnsseed-policy.md @@ -7,7 +7,7 @@ As such, DNS seeds must be run by entities which have some minimum level of trust within the Bitcoin community. Other implementations of Bitcoin software may also use the same -seeds and may be more exposed. In light of this exposure, this +seeds and may be more exposed. In light of this exposure, this document establishes some basic expectations for operating dnsseeds. 0. A DNS seed operating organization or person is expected to follow good diff --git a/doc/gitian-building.md b/doc/gitian-building.md index 630b3c04a7..00fdce82e8 100644 --- a/doc/gitian-building.md +++ b/doc/gitian-building.md @@ -1,11 +1,11 @@ Gitian building ================ -*Setup instructions for a gitian build of Bitcoin using a Debian VM or physical system.* +*Setup instructions for a Gitian build of Bitcoin using a Debian VM or physical system.* Gitian is the deterministic build process that is used to build the Bitcoin Core executables. It provides a way to be reasonably sure that the -executables are really built from source on GitHub. It also makes sure that +executables are really built from the source on GitHub. It also makes sure that the same, tested dependencies are used and statically built into the executable. Multiple developers build the source code by following a specific descriptor @@ -13,8 +13,8 @@ Multiple developers build the source code by following a specific descriptor These results are compared and only if they match, the build is accepted and uploaded to bitcoin.org. -More independent gitian builders are needed, which is why I wrote this -guide. It is preferred to follow these steps yourself instead of using someone else's +More independent Gitian builders are needed, which is why this guide exists. +It is preferred you follow these steps yourself instead of using someone else's VM image to avoid 'contaminating' the build. Table of Contents @@ -22,9 +22,9 @@ Table of Contents - [Create a new VirtualBox VM](#create-a-new-virtualbox-vm) - [Connecting to the VM](#connecting-to-the-vm) -- [Setting up Debian for gitian building](#setting-up-debian-for-gitian-building) -- [Installing gitian](#installing-gitian) -- [Setting up the gitian image](#setting-up-the-gitian-image) +- [Setting up Debian for Gitian building](#setting-up-debian-for-gitian-building) +- [Installing Gitian](#installing-gitian) +- [Setting up the Gitian image](#setting-up-the-gitian-image) - [Getting and building the inputs](#getting-and-building-the-inputs) - [Building Bitcoin](#building-bitcoin) - [Building an alternative repository](#building-an-alternative-repository) @@ -39,49 +39,49 @@ This guide explains how to set up the environment, and how to start the builds. Debian Linux was chosen as the host distribution because it has a lightweight install (in contrast to Ubuntu) and is readily available. Any kind of virtualization can be used, for example: -- [VirtualBox](https://www.virtualbox.org/), covered by this guide +- [VirtualBox](https://www.virtualbox.org/) (covered by this guide) - [KVM](http://www.linux-kvm.org/page/Main_Page) - [LXC](https://linuxcontainers.org/), see also [Gitian host docker container](https://github.com/gdm85/tenku/tree/master/docker/gitian-bitcoin-host/README.md). -You can also install on actual hardware instead of using virtualization. +You can also install Gitian on actual hardware instead of using virtualization. Create a new VirtualBox VM --------------------------- In the VirtualBox GUI click "Create" and choose the following parameters in the wizard: - + -- Type: Linux, Debian (64 bit) +- Type: Linux, Debian (64-bit)  -- Memory Size: at least 1024MB, anything lower will really slow the build down +- Memory Size: at least 1024MB, anything less will really slow down the build. - + -- Hard Drive: Create a virtual hard drive now - - +- Hard Disk: Create a virtual hard disk now -- Hard Drive file type: Use the default, VDI (VirtualBox Disk Image) + + +- Hard Disk file type: Use the default, VDI (VirtualBox Disk Image) + + + +- Storage on physical hard disk: Dynamically Allocated - - -- Storage on Physical hard drive: Dynamically Allocated -  -- Disk size: at least 40GB; as low as 20GB *may* be possible, but better to err on the safe side -- Push the `Create` button +- File location and size: at least 40GB; as low as 20GB *may* be possible, but better to err on the safe side +- Click `Create` -Get the [Debian 7.8 net installer](http://cdimage.debian.org/cdimage/archive/7.8.0/amd64/iso-cd/debian-7.8.0-amd64-netinst.iso) (a more recent minor version should also work, see also [Debian Network installation](https://www.debian.org/CD/netinst/)). +Get the [Debian 8.x net installer](http://cdimage.debian.org/debian-cd/8.2.0/amd64/iso-cd/debian-8.2.0-amd64-netinst.iso) (a more recent minor version should also work, see also [Debian Network installation](https://www.debian.org/CD/netinst/)). This DVD image can be validated using a SHA256 hashing tool, for example on Unixy OSes by entering the following in a terminal: - echo "e39c36d6adc0fd86c6edb0e03e22919086c883b37ca194d063b8e3e8f6ff6a3a debian-7.8.0-amd64-netinst.iso" | sha256sum -c + echo "d393d17ac6b3113c81186e545c416a00f28ed6e05774284bb5e8f0df39fcbcb9 debian-8.2.0-amd64-netinst.iso" | sha256sum -c # (must return OK) -After creating the VM, we need to configure it. +After creating the VM, we need to configure it. - Click the `Settings` button, then go to the `Network` tab. Adapter 1 should be attached to `NAT`. @@ -115,8 +115,9 @@ This section will explain how to install Debian on the newly created VM.  -**Note**: Navigation in the Debian installer: To keep a setting at the default -and proceed, just press `Enter`. To select a different button, press `Tab`. +**Note**: Navigating in the Debian installer: +To keep a setting at the default and proceed, just press `Enter`. +To select a different button, press `Tab`. - Choose locale and keyboard settings (doesn't matter, you can just go with the defaults or select your own information) @@ -125,22 +126,23 @@ and proceed, just press `Enter`. To select a different button, press `Tab`.  - The VM will detect network settings using DHCP, this should all proceed automatically -- Configure the network: - - System name `debian`. +- Configure the network: + - Hostname `debian`. - Leave domain name empty.  -- Choose a root password and enter it twice (remember it for later) +- Choose a root password and enter it twice (remember it for later)  -- Name the new user `debian` (the full name doesn't matter, you can leave it empty) +- Name the new user `debian` (the full name doesn't matter, you can leave it empty) +- Set the account username as `debian`   -- Choose a user password and enter it twice (remember it for later) +- Choose a user password and enter it twice (remember it for later)  @@ -150,74 +152,98 @@ and proceed, just press `Enter`. To select a different button, press `Tab`.  - Disk setup - - Partitioning method: Guided - Use the entire disk - + - Partitioning method: Guided - Use the entire disk +  - - Select disk to partition: SCSI1 (0,0,0) + - Select disk to partition: SCSI1 (0,0,0)  - - Partitioning scheme: All files in one partition - - - - Finish partitioning and write changes to disk -> *Yes* (`Tab`, `Enter` to select the `Yes` button) - +  - The base system will be installed, this will take a minute or so -- Choose a mirror (any will do) +- Choose a mirror (any will do)  -- Enter proxy information (unless you are on an intranet, you can leave this empty) +- Enter proxy information (unless you are on an intranet, leave this empty)  - Wait a bit while 'Select and install software' runs - Participate in popularity contest -> *No* -- Choose software to install. We need just the base system. +- Choose software to install. We need just the base system. +- Make sure only 'SSH server' and 'Standard System Utilities' are checked +- Uncheck 'Debian Desktop Environment' and 'Print Server'  -- Make sure only 'SSH server' and 'Standard System Utilities' are checked -- Uncheck 'Debian Desktop Environment' and 'Print Server' +- Install the GRUB boot loader to the master boot record? -> Yes  -- Install the GRUB boot loader to the master boot record? -> Yes +- Device for boot loader installation -> ata-VBOX_HARDDISK - + - Installation Complete -> *Continue* - After installation, the VM will reboot and you will have a working Debian VM. Congratulations! + + + +After Installation +------------------- +The next step in the guide involves logging in as root via SSH. +SSH login for root users is disabled by default, so we'll enable that now. + +Login to the VM using username `root` and the root password you chose earlier. +You'll be presented with a screen similar to this. + + + +Type: + +``` +sed -i 's/^PermitRootLogin.*/PermitRootLogin yes/' /etc/ssh/sshd_config +``` +and press enter. Then, +``` +/etc/init.d/ssh restart +``` +and enter to restart SSH. Logout by typing 'logout' and pressing 'enter'. + Connecting to the VM ---------------------- After the VM has booted you can connect to it using SSH, and files can be copied from and to the VM using a SFTP utility. Connect to `localhost`, port `22222` (or the port configured when installing the VM). -On Windows you can use putty[1] and WinSCP[2]. +On Windows you can use [putty](http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html) and [WinSCP](http://winscp.net/eng/index.php). -For example to connect as `root` from a Linux command prompt use +For example, to connect as `root` from a Linux command prompt use $ ssh root@localhost -p 22222 The authenticity of host '[localhost]:22222 ([127.0.0.1]:22222)' can't be established. - ECDSA key fingerprint is 8e:71:f9:5b:62:46:de:44:01:da:fb:5f:34:b5:f2:18. + RSA key fingerprint is ae:f5:c8:9f:17:c6:c7:1b:c2:1b:12:31:1d:bb:d0:c7. Are you sure you want to continue connecting (yes/no)? yes - Warning: Permanently added '[localhost]:22222' (ECDSA) to the list of known hosts. + Warning: Permanently added '[localhost]:22222' (RSA) to the list of known hosts. root@localhost's password: (enter root password configured during install) - Linux debian 3.2.0-4-amd64 #1 SMP Debian 3.2.54-2 x86_64 + + The programs included with the Debian GNU/Linux system are free software; + the exact distribution terms for each program are described in the + individual files in /usr/share/doc/*/copyright. + + Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent + permitted by applicable law. root@debian:~# Replace `root` with `debian` to log in as user. -[1] http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html -[2] http://winscp.net/eng/index.php - -Setting up Debian for gitian building +Setting up Debian for Gitian building -------------------------------------- In this section we will be setting up the Debian installation for Gitian building. @@ -226,18 +252,15 @@ First we need to log in as `root` to set up dependencies and make sure that our user can use the sudo command. Type/paste the following in the terminal: ```bash -apt-get install git ruby sudo apt-cacher-ng qemu-utils debootstrap lxc python-cheetah parted kpartx bridge-utils +apt-get install git ruby sudo apt-cacher-ng qemu-utils debootstrap lxc python-cheetah parted kpartx bridge-utils make ubuntu-archive-keyring adduser debian sudo ``` -When you get a colorful screen with a question about the 'LXC directory', just -go with the default (`/var/lib/lxc`). - Then set up LXC and the rest with the following, which is a complex jumble of settings and workarounds: ```bash # the version of lxc-start in Debian 7.4 needs to run as root, so make sure -# that the build script can exectute it without providing a password +# that the build script can execute it without providing a password echo "%sudo ALL=NOPASSWD: /usr/bin/lxc-start" > /etc/sudoers.d/gitian-lxc # add cgroup for LXC echo "cgroup /sys/fs/cgroup cgroup defaults 0 0" >> /etc/fstab @@ -255,9 +278,9 @@ reboot ``` At the end the VM is rebooted to make sure that the changes take effect. The steps in this -section need only to be performed once. +section only need to be performed once. -Installing gitian +Installing Gitian ------------------ Re-login as the user `debian` that was created during installation. @@ -277,14 +300,14 @@ cd .. **Note**: When sudo asks for a password, enter the password for the user *debian* not for *root*. -Clone the git repositories for bitcoin and gitian. +Clone the git repositories for bitcoin and Gitian. ```bash git clone https://github.com/devrandom/gitian-builder.git git clone https://github.com/bitcoin/bitcoin ``` -Setting up the gitian image +Setting up the Gitian image ------------------------- Gitian needs a virtual image of the operating system to build in. @@ -300,26 +323,27 @@ cd gitian-builder bin/make-base-vm --lxc --arch amd64 --suite precise ``` -There will be a lot of warnings printed during build of the image. These can be ignored. +There will be a lot of warnings printed during the build of the image. These can be ignored. **Note**: When sudo asks for a password, enter the password for the user *debian* not for *root*. Getting and building the inputs -------------------------------- -Follow the instructions in [doc/release-process.md](release-process.md) in the bitcoin repository -under 'Fetch and build inputs' to install sources which require manual intervention. Also follow -the next step: 'Seed the Gitian sources cache', which will fetch all necessary source files allowing -for gitian to work offline. +Follow the instructions in [doc/release-process.md](release-process.md#fetch-and-build-inputs-first-time-or-when-dependency-versions-change) +in the bitcoin repository under 'Fetch and build inputs' to install sources which require +manual intervention. Also optionally follow the next step: 'Seed the Gitian sources cache +and offline git repositories' which will fetch the remaining files required for building +offline. Building Bitcoin ---------------- -To build Bitcoin (for Linux, OSX and Windows) just follow the steps under 'perform -gitian builds' in [doc/release-process.md](release-process.md) in the bitcoin repository. +To build Bitcoin (for Linux, OS X and Windows) just follow the steps under 'perform +Gitian builds' in [doc/release-process.md](release-process.md#perform-gitian-builds) in the bitcoin repository. -This may take a long time as it also builds the dependencies needed for each descriptor. -These dependencies will be cached after a successful build to avoid rebuilding them where possible. +This may take some time as it will build all the dependencies needed for each descriptor. +These dependencies will be cached after a successful build to avoid rebuilding them when possible. At any time you can check the package installation and build progress with @@ -331,13 +355,13 @@ tail -f var/build.log Output from `gbuild` will look something like Initialized empty Git repository in /home/debian/gitian-builder/inputs/bitcoin/.git/ - remote: Reusing existing pack: 35606, done. - remote: Total 35606 (delta 0), reused 0 (delta 0) - Receiving objects: 100% (35606/35606), 26.52 MiB | 4.28 MiB/s, done. - Resolving deltas: 100% (25724/25724), done. + remote: Counting objects: 57959, done. + remote: Total 57959 (delta 0), reused 0 (delta 0), pack-reused 57958 + Receiving objects: 100% (57959/57959), 53.76 MiB | 484.00 KiB/s, done. + Resolving deltas: 100% (41590/41590), done. From https://github.com/bitcoin/bitcoin ... (new tags, new branch etc) - --- Building for precise x86_64 --- + --- Building for precise amd64 --- Stopping target if it is up Making a new image copy stdin: is not a tty @@ -356,7 +380,7 @@ Building an alternative repository ----------------------------------- If you want to do a test build of a pull on GitHub it can be useful to point -the gitian builder at an alternative repository, using the same descriptors +the Gitian builder at an alternative repository, using the same descriptors and inputs. For example: @@ -368,6 +392,57 @@ COMMIT=2014_03_windows_unicode_path ./bin/gbuild --commit bitcoin=${COMMIT} --url bitcoin=${URL} ../bitcoin/contrib/gitian-descriptors/gitian-osx.yml ``` +Building fully offline +----------------------- + +For building fully offline including attaching signatures to unsigned builds, the detached-sigs repository +and the bitcoin git repository with the desired tag must both be available locally, and then gbuild must be +told where to find them. It also requires an apt-cacher-ng which is fully-populated but set to offline mode, or +manually disabling gitian-builder's use of apt-get to update the VM build environment. + +To configure apt-cacher-ng as an offline cacher, you will need to first populate its cache with the relevant +files. You must additionally patch target-bin/bootstrap-fixup to set its apt sources to something other than +plain archive.ubuntu.com: us.archive.ubuntu.com works. + +So, if you use LXC: + +```bash +export PATH="$PATH":/path/to/gitian-builder/libexec +export USE_LXC=1 +cd /path/to/gitian-builder +./libexec/make-clean-vm --suite precise --arch amd64 + +LXC_ARCH=amd64 LXC_SUITE=precise on-target -u root apt-get update +LXC_ARCH=amd64 LXC_SUITE=precise on-target -u root \ + -e DEBIAN_FRONTEND=noninteractive apt-get --no-install-recommends -y install \ + $( sed -ne '/^packages:/,/[^-] .*/ {/^- .*/{s/"//g;s/- //;p}}' ../bitcoin/contrib/gitian-descriptors/*|sort|uniq ) +LXC_ARCH=amd64 LXC_SUITE=precise on-target -u root apt-get -q -y purge grub +LXC_ARCH=amd64 LXC_SUITE=precise on-target -u root -e DEBIAN_FRONTEND=noninteractive apt-get -y dist-upgrade +``` + +And then set offline mode for apt-cacher-ng: + +``` +/etc/apt-cacher-ng/acng.conf +[...] +Offlinemode: 1 +[...] + +service apt-cacher-ng restart +``` + +Then when building, override the remote URLs that gbuild would otherwise pull from the Gitian descriptors:: +```bash + +cd /some/root/path/ +git clone https://github.com/bitcoin/bitcoin-detached-sigs.git + +BTCPATH=/some/root/path/bitcoin.git +SIGPATH=/some/root/path/bitcoin-detached-sigs.git + +./bin/gbuild --url bitcoin=${BTCPATH},signature=${SIGPATH} ../bitcoin/contrib/gitian-descriptors/gitian-win-signer.yml +``` + Signing externally ------------------- @@ -386,7 +461,7 @@ in `gitian.sigs` to your signing machine and do ``` This will create the `.sig` files that can be committed together with the `.assert` files to assert your -gitian build. +Gitian build. Uploading signatures --------------------- diff --git a/doc/gitian-building/create_new_vm.png b/doc/gitian-building/create_new_vm.png Binary files differnew file mode 100644 index 0000000000..dd22428e17 --- /dev/null +++ b/doc/gitian-building/create_new_vm.png diff --git a/doc/gitian-building/create_vm_hard_disk.png b/doc/gitian-building/create_vm_hard_disk.png Binary files differnew file mode 100644 index 0000000000..8e29816fab --- /dev/null +++ b/doc/gitian-building/create_vm_hard_disk.png diff --git a/doc/gitian-building/create_vm_hard_disk_file_type.png b/doc/gitian-building/create_vm_hard_disk_file_type.png Binary files differnew file mode 100644 index 0000000000..a157211cf5 --- /dev/null +++ b/doc/gitian-building/create_vm_hard_disk_file_type.png diff --git a/doc/gitian-building/create_vm_hard_drive.png b/doc/gitian-building/create_vm_hard_drive.png Binary files differdeleted file mode 100644 index a1706e14fd..0000000000 --- a/doc/gitian-building/create_vm_hard_drive.png +++ /dev/null diff --git a/doc/gitian-building/create_vm_hard_drive_file_type.png b/doc/gitian-building/create_vm_hard_drive_file_type.png Binary files differdeleted file mode 100644 index 251b8ee3e2..0000000000 --- a/doc/gitian-building/create_vm_hard_drive_file_type.png +++ /dev/null diff --git a/doc/gitian-building/create_vm_memsize.png b/doc/gitian-building/create_vm_memsize.png Binary files differindex 33717867a5..5abfee5337 100644 --- a/doc/gitian-building/create_vm_memsize.png +++ b/doc/gitian-building/create_vm_memsize.png diff --git a/doc/gitian-building/create_vm_page1.png b/doc/gitian-building/create_vm_page1.png Binary files differdeleted file mode 100644 index edaebc6223..0000000000 --- a/doc/gitian-building/create_vm_page1.png +++ /dev/null diff --git a/doc/gitian-building/create_vm_storage_physical_hard_disk.png b/doc/gitian-building/create_vm_storage_physical_hard_disk.png Binary files differnew file mode 100644 index 0000000000..cee16a6c63 --- /dev/null +++ b/doc/gitian-building/create_vm_storage_physical_hard_disk.png diff --git a/doc/gitian-building/create_vm_storage_physical_hard_drive.png b/doc/gitian-building/create_vm_storage_physical_hard_drive.png Binary files differdeleted file mode 100644 index 987efaa40c..0000000000 --- a/doc/gitian-building/create_vm_storage_physical_hard_drive.png +++ /dev/null diff --git a/doc/gitian-building/debian_install_10_configure_clock.png b/doc/gitian-building/debian_install_10_configure_clock.png Binary files differindex 467c79018e..7cda038ae4 100644 --- a/doc/gitian-building/debian_install_10_configure_clock.png +++ b/doc/gitian-building/debian_install_10_configure_clock.png diff --git a/doc/gitian-building/debian_install_11_partition_disks.png b/doc/gitian-building/debian_install_11_partition_disks.png Binary files differindex 18110734df..2a648c517f 100644 --- a/doc/gitian-building/debian_install_11_partition_disks.png +++ b/doc/gitian-building/debian_install_11_partition_disks.png diff --git a/doc/gitian-building/debian_install_12_choose_disk.png b/doc/gitian-building/debian_install_12_choose_disk.png Binary files differindex a00d4abf17..0f3acc498e 100644 --- a/doc/gitian-building/debian_install_12_choose_disk.png +++ b/doc/gitian-building/debian_install_12_choose_disk.png diff --git a/doc/gitian-building/debian_install_13_partition_scheme.png b/doc/gitian-building/debian_install_13_partition_scheme.png Binary files differdeleted file mode 100644 index 2f80f19b63..0000000000 --- a/doc/gitian-building/debian_install_13_partition_scheme.png +++ /dev/null diff --git a/doc/gitian-building/debian_install_14_finish.png b/doc/gitian-building/debian_install_14_finish.png Binary files differindex 411d457e95..c8ef0b37ad 100644 --- a/doc/gitian-building/debian_install_14_finish.png +++ b/doc/gitian-building/debian_install_14_finish.png diff --git a/doc/gitian-building/debian_install_15_write_changes.png b/doc/gitian-building/debian_install_15_write_changes.png Binary files differindex f26093982c..d8de00dec6 100644 --- a/doc/gitian-building/debian_install_15_write_changes.png +++ b/doc/gitian-building/debian_install_15_write_changes.png diff --git a/doc/gitian-building/debian_install_16_choose_a_mirror.png b/doc/gitian-building/debian_install_16_choose_a_mirror.png Binary files differindex d2c2e9523b..0bd985b38c 100644 --- a/doc/gitian-building/debian_install_16_choose_a_mirror.png +++ b/doc/gitian-building/debian_install_16_choose_a_mirror.png diff --git a/doc/gitian-building/debian_install_17_choose_a_mirror2.png b/doc/gitian-building/debian_install_17_choose_a_mirror2.png Binary files differdeleted file mode 100644 index cef2db0781..0000000000 --- a/doc/gitian-building/debian_install_17_choose_a_mirror2.png +++ /dev/null diff --git a/doc/gitian-building/debian_install_18_proxy_settings.png b/doc/gitian-building/debian_install_18_proxy_settings.png Binary files differindex 24ba25c109..2c19919f64 100644 --- a/doc/gitian-building/debian_install_18_proxy_settings.png +++ b/doc/gitian-building/debian_install_18_proxy_settings.png diff --git a/doc/gitian-building/debian_install_19_software_selection.png b/doc/gitian-building/debian_install_19_software_selection.png Binary files differindex d462757aff..5430456b14 100644 --- a/doc/gitian-building/debian_install_19_software_selection.png +++ b/doc/gitian-building/debian_install_19_software_selection.png diff --git a/doc/gitian-building/debian_install_1_boot_menu.png b/doc/gitian-building/debian_install_1_boot_menu.png Binary files differindex 27fd849b4f..216502e1c6 100644 --- a/doc/gitian-building/debian_install_1_boot_menu.png +++ b/doc/gitian-building/debian_install_1_boot_menu.png diff --git a/doc/gitian-building/debian_install_20_install_grub.png b/doc/gitian-building/debian_install_20_install_grub.png Binary files differindex de4f9be0c9..d853c15871 100644 --- a/doc/gitian-building/debian_install_20_install_grub.png +++ b/doc/gitian-building/debian_install_20_install_grub.png diff --git a/doc/gitian-building/debian_install_21_install_grub_bootloader.png b/doc/gitian-building/debian_install_21_install_grub_bootloader.png Binary files differnew file mode 100644 index 0000000000..493ab806a6 --- /dev/null +++ b/doc/gitian-building/debian_install_21_install_grub_bootloader.png diff --git a/doc/gitian-building/debian_install_21_finish_installation.png b/doc/gitian-building/debian_install_22_finish_installation.png Binary files differindex b967c3550d..7c4445585b 100644 --- a/doc/gitian-building/debian_install_21_finish_installation.png +++ b/doc/gitian-building/debian_install_22_finish_installation.png diff --git a/doc/gitian-building/debian_install_2_select_a_language.png b/doc/gitian-building/debian_install_2_select_a_language.png Binary files differindex 1c9e0bcfc1..0228ae2c01 100644 --- a/doc/gitian-building/debian_install_2_select_a_language.png +++ b/doc/gitian-building/debian_install_2_select_a_language.png diff --git a/doc/gitian-building/debian_install_3_select_location.png b/doc/gitian-building/debian_install_3_select_location.png Binary files differindex 005c395656..7b18fba975 100644 --- a/doc/gitian-building/debian_install_3_select_location.png +++ b/doc/gitian-building/debian_install_3_select_location.png diff --git a/doc/gitian-building/debian_install_4_configure_keyboard.png b/doc/gitian-building/debian_install_4_configure_keyboard.png Binary files differindex 580c8af7c5..8e46117de4 100644 --- a/doc/gitian-building/debian_install_4_configure_keyboard.png +++ b/doc/gitian-building/debian_install_4_configure_keyboard.png diff --git a/doc/gitian-building/debian_install_5_configure_the_network.png b/doc/gitian-building/debian_install_5_configure_the_network.png Binary files differindex a7fdffc66b..8e3720f243 100644 --- a/doc/gitian-building/debian_install_5_configure_the_network.png +++ b/doc/gitian-building/debian_install_5_configure_the_network.png diff --git a/doc/gitian-building/debian_install_6a_set_up_root_password.png b/doc/gitian-building/debian_install_6a_set_up_root_password.png Binary files differindex 31bd210f38..dcade11967 100644 --- a/doc/gitian-building/debian_install_6a_set_up_root_password.png +++ b/doc/gitian-building/debian_install_6a_set_up_root_password.png diff --git a/doc/gitian-building/debian_install_7_set_up_user_fullname.png b/doc/gitian-building/debian_install_7_set_up_user_fullname.png Binary files differindex bffc6ccd7a..6763c6e08a 100644 --- a/doc/gitian-building/debian_install_7_set_up_user_fullname.png +++ b/doc/gitian-building/debian_install_7_set_up_user_fullname.png diff --git a/doc/gitian-building/debian_install_8_set_up_username.png b/doc/gitian-building/debian_install_8_set_up_username.png Binary files differindex 9e2750ad4e..bb04de96d2 100644 --- a/doc/gitian-building/debian_install_8_set_up_username.png +++ b/doc/gitian-building/debian_install_8_set_up_username.png diff --git a/doc/gitian-building/debian_install_9_user_password.png b/doc/gitian-building/debian_install_9_user_password.png Binary files differindex a26d30cba5..981f1181d7 100644 --- a/doc/gitian-building/debian_install_9_user_password.png +++ b/doc/gitian-building/debian_install_9_user_password.png diff --git a/doc/gitian-building/debian_root_login.png b/doc/gitian-building/debian_root_login.png Binary files differnew file mode 100644 index 0000000000..14cdd5ba5b --- /dev/null +++ b/doc/gitian-building/debian_root_login.png diff --git a/doc/gitian-building/network_settings.png b/doc/gitian-building/network_settings.png Binary files differindex 1d9b6428a7..9e714fd154 100644 --- a/doc/gitian-building/network_settings.png +++ b/doc/gitian-building/network_settings.png diff --git a/doc/gitian-building/port_forwarding_rules.png b/doc/gitian-building/port_forwarding_rules.png Binary files differindex e45c9efffc..9e1fa2af20 100644 --- a/doc/gitian-building/port_forwarding_rules.png +++ b/doc/gitian-building/port_forwarding_rules.png diff --git a/doc/gitian-building/select_startup_disk.png b/doc/gitian-building/select_startup_disk.png Binary files differindex 729b368fd1..5acdc3fe10 100644 --- a/doc/gitian-building/select_startup_disk.png +++ b/doc/gitian-building/select_startup_disk.png diff --git a/doc/init.md b/doc/init.md index 1bbfed4380..e3db5b05ef 100644 --- a/doc/init.md +++ b/doc/init.md @@ -30,20 +30,20 @@ file, however it is recommended that a strong and secure password be used as this password is security critical to securing the wallet should the wallet be enabled. -If bitcoind is run with the "-server" flag (set by default), and no rpcpassword is set, -it will use a special cookie file for authentication. The cookie is generated with random +If bitcoind is run with the "-server" flag (set by default), and no rpcpassword is set, +it will use a special cookie file for authentication. The cookie is generated with random content when the daemon starts, and deleted when it exits. Read access to this file -controls who can access it through RPC. +controls who can access it through RPC. -By default the cookie is stored in the data directory, but it's location can be overridden +By default the cookie is stored in the data directory, but it's location can be overridden with the option '-rpccookiefile'. This allows for running bitcoind without having to do any manual configuration. -`conf`, `pid`, and `wallet` accept relative paths which are interpreted as +`conf`, `pid`, and `wallet` accept relative paths which are interpreted as relative to the data directory. `wallet` *only* supports relative paths. -For an example configuration file that describes the configuration settings, +For an example configuration file that describes the configuration settings, see `contrib/debian/examples/bitcoin.conf`. 3. Paths @@ -103,8 +103,8 @@ use old versions of Upstart and do not supply the start-stop-daemon utility. Copy bitcoind.init to /etc/init.d/bitcoind. Test by running `service bitcoind start`. -Using this script, you can adjust the path and flags to the bitcoind program by -setting the BITCOIND and FLAGS environment variables in the file +Using this script, you can adjust the path and flags to the bitcoind program by +setting the BITCOIND and FLAGS environment variables in the file /etc/sysconfig/bitcoind. You can also use the DAEMONOPTS environment variable here. 4e) Mac OS X @@ -123,4 +123,3 @@ Launch Daemon with a dedicated bitcoin user. Auto respawning is currently only configured for Upstart and systemd. Reasonable defaults have been chosen but YMMV. - diff --git a/doc/release-notes.md b/doc/release-notes.md index 9caac4f7ae..e9b2d75a76 100644 --- a/doc/release-notes.md +++ b/doc/release-notes.md @@ -4,6 +4,59 @@ release-notes at release time) Notable changes =============== +SSL support for RPC dropped +---------------------------- + +SSL support for RPC, previously enabled by the option `rpcssl` has been dropped +from both the client and the server. This was done in preparation for removing +the dependency on OpenSSL for the daemon completely. + +Trying to use `rpcssl` will result in an error: + + Error: SSL mode for RPC (-rpcssl) is no longer supported. + +If you are one of the few people that relies on this feature, a flexible +migration path is to use `stunnel`. This is an utility that can tunnel +arbitrary TCP connections inside SSL. On e.g. Ubuntu it can be installed with: + + sudo apt-get install stunnel4 + +Then, to tunnel a SSL connection on 28332 to a RPC server bound on localhost on port 18332 do: + + stunnel -d 28332 -r 127.0.0.1:18332 -p stunnel.pem -P '' + +It can also be set up system-wide in inetd style. + +Another way to re-attain SSL would be to setup a httpd reverse proxy. This solution +would allow the use of different authentication, loadbalancing, on-the-fly compression and +caching. A sample config for apache2 could look like: + + Listen 443 + + NameVirtualHost *:443 + <VirtualHost *:443> + + SSLEngine On + SSLCertificateFile /etc/apache2/ssl/server.crt + SSLCertificateKeyFile /etc/apache2/ssl/server.key + + <Location /bitcoinrpc> + ProxyPass http://127.0.0.1:8332/ + ProxyPassReverse http://127.0.0.1:8332/ + # optional enable digest auth + # AuthType Digest + # ... + + # optional bypass bitcoind rpc basic auth + # RequestHeader set Authorization "Basic <hash>" + # get the <hash> from the shell with: base64 <<< bitcoinrpc:<password> + </Location> + + # Or, balance the load: + # ProxyPass / balancer://balancer_cluster_name + + </VirtualHost> + Random-cookie RPC authentication --------------------------------- @@ -35,6 +88,42 @@ specified. It used to be the case that `-X -noX` ends up, unintuitively, with X set, as `-X` had precedence over `-noX`. This is no longer the case. Like for other software, the last specified value for an option will hold. +`NODE_BLOOM` service bit +------------------------ + +Support for the `NODE_BLOOM` service bit, as described in [BIP +111](https://github.com/bitcoin/bips/blob/master/bip-0111.mediawiki), has been +added to the P2P protocol code. + +BIP 111 defines a service bit to allow peers to advertise that they support +bloom filters (such as used by SPV clients) explicitly. It also bumps the protocol +version to allow peers to identify old nodes which allow bloom filtering of the +connection despite lacking the new service bit. + +In this version, it is only enforced for peers that send protocol versions +`>=70011`. For the next major version it is planned that this restriction will be +removed. It is recommended to update SPV clients to check for the `NODE_BLOOM` +service bit for nodes that report versions newer than 70011. + +Any sequence of pushdatas in OP_RETURN outputs now allowed +---------------------------------------------------------- + +Previously OP_RETURN outputs with a payload were only relayed and mined if they +had a single pushdata. This restriction has been lifted to allow any +combination of data pushes and numeric constant opcodes (OP_1 to OP_16). The +limit on OP_RETURN output size is now applied to the entire serialized +scriptPubKey, 83 bytes by default. (the previous 80 byte default plus three +bytes overhead) + +Merkle branches removed from wallet +----------------------------------- + +Previously, every wallet transaction stored a Merkle branch to prove its +presence in blocks. This wasn't being used for more than an expensive +sanity check. Since 0.12, these are no longer stored. When loading a +0.12 wallet into an older version, it will automatically rescan to avoid +failed checks. + 0.12.0 Change log ================= @@ -45,6 +134,33 @@ git merge commit are mentioned. ### RPC and REST +Asm representations of scriptSig signatures now contain SIGHASH type decodes +---------------------------------------------------------------------------- + +The `asm` property of each scriptSig now contains the decoded signature hash +type for each signature that provides a valid defined hash type. + +The following items contain assembly representations of scriptSig signatures +and are affected by this change: + +- RPC `getrawtransaction` +- RPC `decoderawtransaction` +- REST `/rest/tx/` (JSON format) +- REST `/rest/block/` (JSON format when including extended tx details) +- `bitcoin-tx -json` + +For example, the `scriptSig.asm` property of a transaction input that +previously showed an assembly representation of: + + 304502207fa7a6d1e0ee81132a269ad84e68d695483745cde8b541e3bf630749894e342a022100c1f7ab20e13e22fb95281a870f3dcf38d782e53023ee313d741ad0cfbc0c509001 + +now shows as: + + 304502207fa7a6d1e0ee81132a269ad84e68d695483745cde8b541e3bf630749894e342a022100c1f7ab20e13e22fb95281a870f3dcf38d782e53023ee313d741ad0cfbc0c5090[ALL] + +Note that the output of the RPC `decodescript` did not change because it is +configured specifically to process scriptPubKey and not scriptSig scripts. + ### Configuration and command-line options ### Block and transaction handling @@ -65,3 +181,11 @@ git merge commit are mentioned. - Removed bitrpc.py from contrib +Addition of ZMQ-based Notifications +================================== + +Bitcoind can now (optionally) asynchronously notify clients through a +ZMQ-based PUB socket of the arrival of new transactions and blocks. +This feature requires installation of the ZMQ C API library 4.x and +configuring its use through the command line or configuration file. +Please see docs/zmq.md for details of operation. diff --git a/doc/release-notes/release-notes-0.10.1.md b/doc/release-notes/release-notes-0.10.1.md index 5e939600a0..8f59f1f68c 100644 --- a/doc/release-notes/release-notes-0.10.1.md +++ b/doc/release-notes/release-notes-0.10.1.md @@ -101,7 +101,7 @@ Tests: Miscellaneous: - `c9e022b` Initialization: set Boost path locale in main thread - `23126a0` Sanitize command strings before logging them. -- `323de27` Initialization: setup environment before starting QT tests +- `323de27` Initialization: setup environment before starting Qt tests - `7494e09` Initialization: setup environment before starting tests - `df45564` Initialization: set fallback locale as environment variable diff --git a/doc/release-notes/release-notes-0.10.3.md b/doc/release-notes/release-notes-0.10.3.md new file mode 100644 index 0000000000..8a110e562c --- /dev/null +++ b/doc/release-notes/release-notes-0.10.3.md @@ -0,0 +1,165 @@ +Bitcoin Core version 0.10.3 is now available from: + + <https://bitcoin.org/bin/bitcoin-core-0.10.3/> + +This is a new minor version release, bringing security fixes and translation +updates. It is recommended to upgrade to this version as soon as possible. + +Please report bugs using the issue tracker at github: + + <https://github.com/bitcoin/bitcoin/issues> + +Upgrading and downgrading +========================= + +How to Upgrade +-------------- + +If you are running an older version, shut it down. Wait until it has completely +shut down (which might take a few minutes for older versions), then run the +installer (on Windows) or just copy over /Applications/Bitcoin-Qt (on Mac) or +bitcoind/bitcoin-qt (on Linux). + +Downgrade warning +------------------ + +Because release 0.10.0 and later makes use of headers-first synchronization and +parallel block download (see further), the block files and databases are not +backwards-compatible with pre-0.10 versions of Bitcoin Core or other software: + +* Blocks will be stored on disk out of order (in the order they are +received, really), which makes it incompatible with some tools or +other programs. Reindexing using earlier versions will also not work +anymore as a result of this. + +* The block index database will now hold headers for which no block is +stored on disk, which earlier versions won't support. + +If you want to be able to downgrade smoothly, make a backup of your entire data +directory. Without this your node will need start syncing (or importing from +bootstrap.dat) anew afterwards. It is possible that the data from a completely +synchronised 0.10 node may be usable in older versions as-is, but this is not +supported and may break as soon as the older version attempts to reindex. + +This does not affect wallet forward or backward compatibility. + +Notable changes +=============== + +Fix buffer overflow in bundled upnp +------------------------------------ + +Bundled miniupnpc was updated to 1.9.20151008. This fixes a buffer overflow in +the XML parser during initial network discovery. + +Details can be found here: http://talosintel.com/reports/TALOS-2015-0035/ + +This applies to the distributed executables only, not when building from source or +using distribution provided packages. + +Additionally, upnp has been disabled by default. This may result in a lower +number of reachable nodes on IPv4, however this prevents future libupnpc +vulnerabilities from being a structural risk to the network +(see https://github.com/bitcoin/bitcoin/pull/6795). + +Test for LowS signatures before relaying +----------------------------------------- + +Make the node require the canonical 'low-s' encoding for ECDSA signatures when +relaying or mining. This removes a nuisance malleability vector. + +Consensus behavior is unchanged. + +If widely deployed this change would eliminate the last remaining known vector +for nuisance malleability on SIGHASH_ALL P2PKH transactions. On the down-side +it will block most transactions made by sufficiently out of date software. + +Unlike the other avenues to change txids on transactions this +one was randomly violated by all deployed bitcoin software prior to +its discovery. So, while other malleability vectors where made +non-standard as soon as they were discovered, this one has remained +permitted. Even BIP62 did not propose applying this rule to +old version transactions, but conforming implementations have become +much more common since BIP62 was initially written. + +Bitcoin Core has produced compatible signatures since a28fb70e in +September 2013, but this didn't make it into a release until 0.9 +in March 2014; Bitcoinj has done so for a similar span of time. +Bitcoinjs and electrum have been more recently updated. + +This does not replace the need for BIP62 or similar, as miners can +still cooperate to break transactions. Nor does it replace the +need for wallet software to handle malleability sanely[1]. This +only eliminates the cheap and irritating DOS attack. + +[1] On the Malleability of Bitcoin Transactions +Marcin Andrychowicz, Stefan Dziembowski, Daniel Malinowski, Łukasz Mazurek +http://fc15.ifca.ai/preproceedings/bitcoin/paper_9.pdf + +Minimum relay fee default increase +----------------------------------- + +The default for the `-minrelaytxfee` setting has been increased from `0.00001` +to `0.00005`. + +This is necessitated by the current transaction flooding, causing +outrageous memory usage on nodes due to the mempool ballooning. This is a +temporary measure, bridging the time until a dynamic method for determining +this fee is merged (which will be in 0.12). + +(see https://github.com/bitcoin/bitcoin/pull/6793, as well as the 0.11.0 +release notes, in which this value was suggested) + +0.10.3 Change log +================= + +Detailed release notes follow. This overview includes changes that affect external +behavior, not code moves, refactors or string updates. + +- #6186 `e4a7d51` Fix two problems in CSubnet parsing +- #6153 `ebd7d8d` Parameter interaction: disable upnp if -proxy set +- #6203 `ecc96f5` Remove P2SH coinbase flag, no longer interesting +- #6226 `181771b` json: fail read_string if string contains trailing garbage +- #6244 `09334e0` configure: Detect (and reject) LibreSSL +- #6276 `0fd8464` Fix getbalance * 0 +- #6274 `be64204` Add option `-alerts` to opt out of alert system +- #6319 `3f55638` doc: update mailing list address +- #6438 `7e66e9c` openssl: avoid config file load/race +- #6439 `255eced` Updated URL location of netinstall for Debian +- #6412 `0739e6e` Test whether created sockets are select()able +- #6694 `f696ea1` [QT] fix thin space word wrap line brake issue +- #6704 `743cc9e` Backport bugfixes to 0.10 +- #6769 `1cea6b0` Test LowS in standardness, removes nuisance malleability vector. +- #6789 `093d7b5` Update miniupnpc to 1.9.20151008 +- #6795 `f2778e0` net: Disable upnp by default +- #6797 `91ef4d9` Do not store more than 200 timedata samples +- #6793 `842c48d` Bump minrelaytxfee default + +Credits +======= + +Thanks to everyone who directly contributed to this release: + +- Adam Weiss +- Alex Morcos +- Casey Rodarmor +- Cory Fields +- fanquake +- Gregory Maxwell +- Jonas Schnelli +- J Ross Nicoll +- Luke Dashjr +- Pavel Vasin +- Pieter Wuille +- randy-waterhouse +- ฿tcDrak +- Tom Harding +- Veres Lajos +- Wladimir J. van der Laan + +And all those who contributed additional code review and/or security research: + +- timothy on IRC for reporting the issue +- Vulnerability in miniupnp discovered by Aleksandar Nikolic of Cisco Talos + +As well as everyone that helped translating on [Transifex](https://www.transifex.com/projects/p/bitcoin/). diff --git a/doc/release-notes/release-notes-0.11.1.md b/doc/release-notes/release-notes-0.11.1.md new file mode 100644 index 0000000000..799205691e --- /dev/null +++ b/doc/release-notes/release-notes-0.11.1.md @@ -0,0 +1,172 @@ +Bitcoin Core version 0.11.1 is now available from: + + <https://bitcoin.org/bin/bitcoin-core-0.11.1/> + +This is a new minor version release, bringing security fixes. It is recommended +to upgrade to this version as soon as possible. + +Please report bugs using the issue tracker at github: + + <https://github.com/bitcoin/bitcoin/issues> + +Upgrading and downgrading +========================= + +How to Upgrade +-------------- + +If you are running an older version, shut it down. Wait until it has completely +shut down (which might take a few minutes for older versions), then run the +installer (on Windows) or just copy over /Applications/Bitcoin-Qt (on Mac) or +bitcoind/bitcoin-qt (on Linux). + +Downgrade warning +------------------ + +Because release 0.10.0 and later makes use of headers-first synchronization and +parallel block download (see further), the block files and databases are not +backwards-compatible with pre-0.10 versions of Bitcoin Core or other software: + +* Blocks will be stored on disk out of order (in the order they are +received, really), which makes it incompatible with some tools or +other programs. Reindexing using earlier versions will also not work +anymore as a result of this. + +* The block index database will now hold headers for which no block is +stored on disk, which earlier versions won't support. + +If you want to be able to downgrade smoothly, make a backup of your entire data +directory. Without this your node will need start syncing (or importing from +bootstrap.dat) anew afterwards. It is possible that the data from a completely +synchronised 0.10 node may be usable in older versions as-is, but this is not +supported and may break as soon as the older version attempts to reindex. + +This does not affect wallet forward or backward compatibility. There are no +known problems when downgrading from 0.11.x to 0.10.x. + +Notable changes +=============== + +Fix buffer overflow in bundled upnp +------------------------------------ + +Bundled miniupnpc was updated to 1.9.20151008. This fixes a buffer overflow in +the XML parser during initial network discovery. + +Details can be found here: http://talosintel.com/reports/TALOS-2015-0035/ + +This applies to the distributed executables only, not when building from source or +using distribution provided packages. + +Additionally, upnp has been disabled by default. This may result in a lower +number of reachable nodes on IPv4, however this prevents future libupnpc +vulnerabilities from being a structural risk to the network +(see https://github.com/bitcoin/bitcoin/pull/6795). + +Test for LowS signatures before relaying +----------------------------------------- + +Make the node require the canonical 'low-s' encoding for ECDSA signatures when +relaying or mining. This removes a nuisance malleability vector. + +Consensus behavior is unchanged. + +If widely deployed this change would eliminate the last remaining known vector +for nuisance malleability on SIGHASH_ALL P2PKH transactions. On the down-side +it will block most transactions made by sufficiently out of date software. + +Unlike the other avenues to change txids on transactions this +one was randomly violated by all deployed bitcoin software prior to +its discovery. So, while other malleability vectors where made +non-standard as soon as they were discovered, this one has remained +permitted. Even BIP62 did not propose applying this rule to +old version transactions, but conforming implementations have become +much more common since BIP62 was initially written. + +Bitcoin Core has produced compatible signatures since a28fb70e in +September 2013, but this didn't make it into a release until 0.9 +in March 2014; Bitcoinj has done so for a similar span of time. +Bitcoinjs and electrum have been more recently updated. + +This does not replace the need for BIP62 or similar, as miners can +still cooperate to break transactions. Nor does it replace the +need for wallet software to handle malleability sanely[1]. This +only eliminates the cheap and irritating DOS attack. + +[1] On the Malleability of Bitcoin Transactions +Marcin Andrychowicz, Stefan Dziembowski, Daniel Malinowski, Łukasz Mazurek +http://fc15.ifca.ai/preproceedings/bitcoin/paper_9.pdf + +Minimum relay fee default increase +----------------------------------- + +The default for the `-minrelaytxfee` setting has been increased from `0.00001` +to `0.00005`. + +This is necessitated by the current transaction flooding, causing +outrageous memory usage on nodes due to the mempool ballooning. This is a +temporary measure, bridging the time until a dynamic method for determining +this fee is merged (which will be in 0.12). + +(see https://github.com/bitcoin/bitcoin/pull/6793, as well as the 0.11 +release notes, in which this value was suggested) + +0.11.1 Change log +================= + +Detailed release notes follow. This overview includes changes that affect +behavior, not code moves, refactors and string updates. For convenience in locating +the code changes and accompanying discussion, both the pull request and +git merge commit are mentioned. + +- #6438 `2531438` openssl: avoid config file load/race +- #6439 `980f820` Updated URL location of netinstall for Debian +- #6384 `8e5a969` qt: Force TLS1.0+ for SSL connections +- #6471 `92401c2` Depends: bump to qt 5.5 +- #6224 `93b606a` Be even stricter in processing unrequested blocks +- #6571 `100ac4e` libbitcoinconsensus: avoid a crash in multi-threaded environments +- #6545 `649f5d9` Do not store more than 200 timedata samples. +- #6694 `834e299` [QT] fix thin space word wrap line break issue +- #6703 `1cd7952` Backport bugfixes to 0.11 +- #6750 `5ed8d0b` Recent rejects backport to v0.11 +- #6769 `71cc9d9` Test LowS in standardness, removes nuisance malleability vector. +- #6789 `b4ad73f` Update miniupnpc to 1.9.20151008 +- #6785 `b4dc33e` Backport to v0.11: In (strCommand == "tx"), return if AlreadyHave() +- #6412 `0095b9a` Test whether created sockets are select()able +- #6795 `4dbcec0` net: Disable upnp by default +- #6793 `e7bcc4a` Bump minrelaytxfee default + +Credits +======= + +Thanks to everyone who directly contributed to this release: + +- Adam Weiss +- Alex Morcos +- Casey Rodarmor +- Cory Fields +- fanquake +- Gregory Maxwell +- Jonas Schnelli +- J Ross Nicoll +- Luke Dashjr +- Pavel Janík +- Pavel Vasin +- Peter Todd +- Pieter Wuille +- randy-waterhouse +- Ross Nicoll +- Suhas Daftuar +- tailsjoin +- ฿tcDrak +- Tom Harding +- Veres Lajos +- Wladimir J. van der Laan + +And those who contributed additional code review and/or security research: + +- timothy on IRC for reporting the issue +- Vulnerability in miniupnp discovered by Aleksandar Nikolic of Cisco Talos + +As well as everyone that helped translating on [Transifex](https://www.transifex.com/projects/p/bitcoin/). + diff --git a/doc/release-process.md b/doc/release-process.md index 5ecb9334f5..9a2362cb85 100644 --- a/doc/release-process.md +++ b/doc/release-process.md @@ -1,86 +1,115 @@ Release Process ==================== -* update translations (ping wumpus, Diapolo or tcatm on IRC) -* see https://github.com/bitcoin/bitcoin/blob/master/doc/translation_process.md#syncing-with-transifex +* Update translations (ping wumpus, Diapolo or tcatm on IRC) see [translation_process.md](https://github.com/bitcoin/bitcoin/blob/master/doc/translation_process.md#syncing-with-transifex) +* Update [bips.md](bips.md) to account for changes since the last release. * * * -###update (commit) version in sources +###First time / New builders +Check out the source code in the following directory hierarchy. + cd /path/to/your/toplevel/build + git clone https://github.com/bitcoin/gitian.sigs.git + git clone https://github.com/bitcoin/bitcoin-detached-sigs.git + git clone https://github.com/devrandom/gitian-builder.git + git clone https://github.com/bitcoin/bitcoin.git + +###Bitcoin maintainers/release engineers, update (commit) version in sources + + pushd ./bitcoin contrib/verifysfbinaries/verify.sh doc/README* share/setup.nsi src/clientversion.h (change CLIENT_VERSION_IS_RELEASE to true) -###tag version in git + # tag version in git git tag -s v(new version, e.g. 0.8.0) -###write release notes. git shortlog helps a lot, for example: + # write release notes. git shortlog helps a lot, for example: git shortlog --no-merges v(current version, e.g. 0.7.2)..v(new version, e.g. 0.8.0) + popd * * * -###update gitian - - In order to take advantage of the new caching features in gitian, be sure to update to a recent version (`e9741525c` or later is recommended) +###Setup and perform Gitian builds -###perform gitian builds + Setup Gitian descriptors: - From a directory containing the bitcoin source, gitian-builder and gitian.sigs - - export SIGNER=(your gitian key, ie bluematt, sipa, etc) - export VERSION=(new version, e.g. 0.8.0) pushd ./bitcoin + export SIGNER=(your Gitian key, ie bluematt, sipa, etc) + export VERSION=(new version, e.g. 0.8.0) git checkout v${VERSION} popd + + Ensure your gitian.sigs are up-to-date if you wish to gverify your builds against other Gitian signatures. + + pushd ./gitian.sigs + git pull + popd + + Ensure gitian-builder is up-to-date to take advantage of new caching features (`e9741525c` or later is recommended). + pushd ./gitian-builder + git pull + +###Fetch and create inputs: (first time, or when dependency versions change) -###fetch and build inputs: (first time, or when dependency versions change) - mkdir -p inputs wget -P inputs https://bitcoincore.org/cfields/osslsigncode-Backports-to-1.7.1.patch wget -P inputs http://downloads.sourceforge.net/project/osslsigncode/osslsigncode/osslsigncode-1.7.1.tar.gz - Register and download the Apple SDK: (see OSX Readme for details) - + Register and download the Apple SDK: see [OS X readme](README_osx.txt) for details. + https://developer.apple.com/devcenter/download.action?path=/Developer_Tools/xcode_6.1.1/xcode_6.1.1.dmg - + Using a Mac, create a tarball for the 10.9 SDK and copy it to the inputs directory: - + tar -C /Volumes/Xcode/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/ -czf MacOSX10.9.sdk.tar.gz MacOSX10.9.sdk -###Optional: Seed the Gitian sources cache +###Optional: Seed the Gitian sources cache and offline git repositories - By default, gitian will fetch source files as needed. For offline builds, they can be fetched ahead of time: +By default, Gitian will fetch source files as needed. To cache them ahead of time: make -C ../bitcoin/depends download SOURCES_PATH=`pwd`/cache/common - Only missing files will be fetched, so this is safe to re-run for each build. +Only missing files will be fetched, so this is safe to re-run for each build. + +NOTE: Offline builds must use the --url flag to ensure Gitian fetches only from local URLs. For example: +``` +./bin/gbuild --url bitcoin=/path/to/bitcoin,signature=/path/to/sigs {rest of arguments} +``` +The gbuild invocations below <b>DO NOT DO THIS</b> by default. + +###Build (and optionally verify) Bitcoin Core for Linux, Windows, and OS X: -###Build Bitcoin Core for Linux, Windows, and OS X: - ./bin/gbuild --commit bitcoin=v${VERSION} ../bitcoin/contrib/gitian-descriptors/gitian-linux.yml ./bin/gsign --signer $SIGNER --release ${VERSION}-linux --destination ../gitian.sigs/ ../bitcoin/contrib/gitian-descriptors/gitian-linux.yml + ./bin/gverify -v -d ../gitian.sigs/ -r ${VERSION}-linux ../bitcoin/contrib/gitian-descriptors/gitian-linux.yml mv build/out/bitcoin-*.tar.gz build/out/src/bitcoin-*.tar.gz ../ + ./bin/gbuild --commit bitcoin=v${VERSION} ../bitcoin/contrib/gitian-descriptors/gitian-win.yml ./bin/gsign --signer $SIGNER --release ${VERSION}-win-unsigned --destination ../gitian.sigs/ ../bitcoin/contrib/gitian-descriptors/gitian-win.yml + ./bin/gverify -v -d ../gitian.sigs/ -r ${VERSION}-win-unsigned ../bitcoin/contrib/gitian-descriptors/gitian-win.yml mv build/out/bitcoin-*-win-unsigned.tar.gz inputs/bitcoin-win-unsigned.tar.gz mv build/out/bitcoin-*.zip build/out/bitcoin-*.exe ../ + ./bin/gbuild --commit bitcoin=v${VERSION} ../bitcoin/contrib/gitian-descriptors/gitian-osx.yml ./bin/gsign --signer $SIGNER --release ${VERSION}-osx-unsigned --destination ../gitian.sigs/ ../bitcoin/contrib/gitian-descriptors/gitian-osx.yml + ./bin/gverify -v -d ../gitian.sigs/ -r ${VERSION}-osx-unsigned ../bitcoin/contrib/gitian-descriptors/gitian-osx.yml mv build/out/bitcoin-*-osx-unsigned.tar.gz inputs/bitcoin-osx-unsigned.tar.gz mv build/out/bitcoin-*.tar.gz build/out/bitcoin-*.dmg ../ popd + Build output expected: 1. source tarball (bitcoin-${VERSION}.tar.gz) 2. linux 32-bit and 64-bit dist tarballs (bitcoin-${VERSION}-linux[32|64].tar.gz) 3. windows 32-bit and 64-bit unsigned installers and dist zips (bitcoin-${VERSION}-win[32|64]-setup-unsigned.exe, bitcoin-${VERSION}-win[32|64].zip) - 4. OSX unsigned installer and dist tarball (bitcoin-${VERSION}-osx-unsigned.dmg, bitcoin-${VERSION}-osx64.tar.gz) - 5. Gitian signatures (in gitian.sigs/${VERSION}-<linux|{win,osx}-unsigned>/(your gitian key)/ + 4. OS X unsigned installer and dist tarball (bitcoin-${VERSION}-osx-unsigned.dmg, bitcoin-${VERSION}-osx64.tar.gz) + 5. Gitian signatures (in gitian.sigs/${VERSION}-<linux|{win,osx}-unsigned>/(your Gitian key)/ ###Next steps: @@ -94,28 +123,31 @@ Commit your signature to gitian.sigs: git push # Assuming you can push to the gitian.sigs tree popd - Wait for Windows/OSX detached signatures: - Once the Windows/OSX builds each have 3 matching signatures, they will be signed with their respective release keys. - Detached signatures will then be committed to the bitcoin-detached-sigs repository, which can be combined with the unsigned apps to create signed binaries. + Wait for Windows/OS X detached signatures: + + Once the Windows/OS X builds each have 3 matching signatures, they will be signed with their respective release keys. + Detached signatures will then be committed to the [bitcoin-detached-sigs](https://github.com/bitcoin/bitcoin-detached-sigs) repository, which can be combined with the unsigned apps to create signed binaries. - Create the signed OSX binary: + Create (and optionally verify) the signed OS X binary: pushd ./gitian-builder ./bin/gbuild -i --commit signature=v${VERSION} ../bitcoin/contrib/gitian-descriptors/gitian-osx-signer.yml ./bin/gsign --signer $SIGNER --release ${VERSION}-osx-signed --destination ../gitian.sigs/ ../bitcoin/contrib/gitian-descriptors/gitian-osx-signer.yml + ./bin/gverify -v -d ../gitian.sigs/ -r ${VERSION}-osx-signed ../bitcoin/contrib/gitian-descriptors/gitian-osx-signer.yml mv build/out/bitcoin-osx-signed.dmg ../bitcoin-${VERSION}-osx.dmg popd - Create the signed Windows binaries: + Create (and optionally verify) the signed Windows binaries: pushd ./gitian-builder ./bin/gbuild -i --commit signature=v${VERSION} ../bitcoin/contrib/gitian-descriptors/gitian-win-signer.yml ./bin/gsign --signer $SIGNER --release ${VERSION}-win-signed --destination ../gitian.sigs/ ../bitcoin/contrib/gitian-descriptors/gitian-win-signer.yml + ./bin/gverify -v -d ../gitian.sigs/ -r ${VERSION}-win-signed ../bitcoin/contrib/gitian-descriptors/gitian-win-signer.yml mv build/out/bitcoin-*win64-setup.exe ../bitcoin-${VERSION}-win64-setup.exe mv build/out/bitcoin-*win32-setup.exe ../bitcoin-${VERSION}-win32-setup.exe popd -Commit your signature for the signed OSX/Windows binaries: +Commit your signature for the signed OS X/Windows binaries: pushd gitian.sigs git add ${VERSION}-osx-signed/${SIGNER} @@ -143,14 +175,14 @@ Note: check that SHA256SUMS itself doesn't end up in SHA256SUMS, which is a spur - Update bitcoin.org version - First, check to see if the Bitcoin.org maintainers have prepared a - release: https://github.com/bitcoin/bitcoin.org/labels/Releases + release: https://github.com/bitcoin-dot-org/bitcoin.org/labels/Releases - If they have, it will have previously failed their Travis CI checks because the final release files weren't uploaded. Trigger a Travis CI rebuild---if it passes, merge. - If they have not prepared a release, follow the Bitcoin.org release - instructions: https://github.com/bitcoin/bitcoin.org#release-notes + instructions: https://github.com/bitcoin-dot-org/bitcoin.org#release-notes - After the pull request is merged, the website will automatically show the newest version within 15 minutes, as well as update the OS download links. Ping @saivann/@harding (saivann/harding on Freenode) in case anything goes wrong diff --git a/doc/shared-libraries.md b/doc/shared-libraries.md index 1fc32112ce..f4ff53d6e9 100644 --- a/doc/shared-libraries.md +++ b/doc/shared-libraries.md @@ -40,3 +40,5 @@ The interface is defined in the C header `bitcoinconsensus.h` located in `src/s ### Example Implementations - [NBitcoin](https://github.com/NicolasDorier/NBitcoin/blob/master/NBitcoin/Script.cs#L814) (.NET Bindings) - [node-libbitcoinconsensus](https://github.com/bitpay/node-libbitcoinconsensus) (Node.js Bindings) +- [java-libbitcoinconsensus](https://github.com/dexX7/java-libbitcoinconsensus) (Java Bindings) +- [bitcoinconsensus-php](https://github.com/Bit-Wasp/bitcoinconsensus-php) (PHP Bindings) diff --git a/doc/tor.md b/doc/tor.md index 560f71fa27..594897f896 100644 --- a/doc/tor.md +++ b/doc/tor.md @@ -15,15 +15,15 @@ outgoing connections be anonymized, but more is possible. -proxy=ip:port Set the proxy server. If SOCKS5 is selected (default), this proxy server will be used to try to reach .onion addresses as well. - + -onion=ip:port Set the proxy server to use for tor hidden services. You do not need to set this if it's the same as -proxy. You can use -noonion to explicitly disable access to hidden service. - + -listen When using -proxy, listening is disabled by default. If you want to run a hidden service (see next section), you'll need to enable it explicitly. - + -connect=X When behind a Tor proxy, you can specify .onion addresses instead -addnode=X of IP addresses or hostnames in these parameters. It requires -seednode=X SOCKS5. In Tor mode, such addresses can also be exchanged with @@ -55,10 +55,10 @@ your bitcoind's P2P listen port (8333 by default). preference for your node to advertize itself with, for connections coming from unroutable addresses (such as 127.0.0.1, where the Tor proxy typically runs). - + -listen You'll need to enable listening for incoming connections, as this is off by default behind a proxy. - + -discover When -externalip is specified, no attempt is made to discover local IPv4 or IPv6 addresses. If you want to run a dual stack, reachable from both Tor and IPv4 (or IPv6), you'll need to either pass your @@ -70,9 +70,14 @@ In a typical situation, where you're only reachable via Tor, this should suffice ./bitcoind -proxy=127.0.0.1:9050 -externalip=57qr3yd1nyntf5k.onion -listen -(obviously, replace the Onion address with your own). If you don't care too much -about hiding your node, and want to be reachable on IPv4 as well, additionally -specify: +(obviously, replace the Onion address with your own). It should be noted that you still +listen on all devices and another node could establish a clearnet connection, when knowing +your address. To mitigate this, additionally bind the address of your Tor proxy: + + ./bitcoind ... -bind=127.0.0.1 + +If you don't care too much about hiding your node, and want to be reachable on IPv4 +as well, use `discover` instead: ./bitcoind ... -discover @@ -82,4 +87,3 @@ If you only want to use Tor to reach onion addresses, but not use it as a proxy for normal IPv4/IPv6 communication, use: ./bitcoin -onion=127.0.0.1:9050 -externalip=57qr3yd1nyntf5k.onion -discover - diff --git a/doc/translation_process.md b/doc/translation_process.md index 3653e53021..6389c5aced 100644 --- a/doc/translation_process.md +++ b/doc/translation_process.md @@ -1,7 +1,7 @@ Translations ============ -The Bitcoin-Core project has been designed to support multiple localisations. This makes adding new phrases, and completely new languages easily achievable. For managing all application translations, Bitcoin-Core makes use of the Transifex online translation management tool. +The Bitcoin-Core project has been designed to support multiple localisations. This makes adding new phrases, and completely new languages easily achievable. For managing all application translations, Bitcoin-Core makes use of the Transifex online translation management tool. ### Helping to translate (using Transifex) Transifex is setup to monitor the Github repo for updates, and when code containing new translations is found, Transifex will process any changes. It may take several hours after a pull-request has been merged, to appear in the Transifex web interface. @@ -106,6 +106,6 @@ To create a new language template, you will need to edit the languages manifest **Note:** that the language translation file **must end in `.qm`** (the compiled extension), and not `.ts`. ### Questions and general assistance -The Bitcoin-Core translation maintainers include *tcatm, seone, Diapolo, wumpus and luke-jr*.You can find them, and others, in the Freenode IRC chatroom - `irc.freenode.net #bitcoin-dev`. +The Bitcoin-Core translation maintainers include *tcatm, seone, Diapolo, wumpus and luke-jr*. You can find them, and others, in the Freenode IRC chatroom - `irc.freenode.net #bitcoin-core-dev`. If you are a translator, you should also subscribe to the mailing list, https://groups.google.com/forum/#!forum/bitcoin-translators. Announcements will be posted during application pre-releases to notify translators to check for updates. diff --git a/doc/translation_strings_policy.md b/doc/translation_strings_policy.md index cf72a55b20..936a6112c6 100644 --- a/doc/translation_strings_policy.md +++ b/doc/translation_strings_policy.md @@ -1,7 +1,7 @@ Translation Strings Policy =========================== -This document provides guidelines for internationalization of the Bitcoin Core software. +This document provides guidelines for internationalization of the Bitcoin Core software. How to translate? ------------------ @@ -107,4 +107,3 @@ The second example reduces the number of pluralized words that translators have During a string freeze (often before a major release), no translation strings are to be added, modified or removed. This can be checked by executing `make translate` in the `src` directory, then verifying that `bitcoin_en.ts` remains unchanged. - diff --git a/doc/zmq.md b/doc/zmq.md new file mode 100644 index 0000000000..902d1124c7 --- /dev/null +++ b/doc/zmq.md @@ -0,0 +1,101 @@ +# Block and Transaction Broadcasting With ZeroMQ + +[ZeroMQ](http://zeromq.org/) is a lightweight wrapper around TCP +connections, inter-process communication, and shared-memory, +providing various message-oriented semantics such as publish/subscribe, +request/reply, and push/pull. + +The Bitcoin Core daemon can be configured to act as a trusted "border +router", implementing the bitcoin wire protocol and relay, making +consensus decisions, maintaining the local blockchain database, +broadcasting locally generated transactions into the network, and +providing a queryable RPC interface to interact on a polled basis for +requesting blockchain related data. However, there exists only a +limited service to notify external software of events like the arrival +of new blocks or transactions. + +The ZeroMQ facility implements a notification interface through a set +of specific notifiers. Currently there are notifiers that publish +blocks and transactions. This read-only facility requires only the +connection of a corresponding ZeroMQ subscriber port in receiving +software; it is not authenticated nor is there any two-way protocol +involvement. Therefore, subscribers should validate the received data +since it may be out of date, incomplete or even invalid. + +ZeroMQ sockets are self-connecting and self-healing; that is, +connections made between two endpoints will be automatically restored +after an outage, and either end may be freely started or stopped in +any order. + +Because ZeroMQ is message oriented, subscribers receive transactions +and blocks all-at-once and do not need to implement any sort of +buffering or reassembly. + +## Prerequisites + +The ZeroMQ feature in Bitcoin Core requires ZeroMQ API version 4.x or +newer. Typically, it is packaged by distributions as something like +*libzmq3-dev*. The C++ wrapper for ZeroMQ is *not* needed. + +In order to run the example Python client scripts in contrib/ one must +also install *python-zmq*, though this is not necessary for daemon +operation. + +## Enabling + +By default, the ZeroMQ feature is automatically compiled in if the +necessary prerequisites are found. To disable, use --disable-zmq +during the *configure* step of building bitcoind: + + $ ./configure --disable-zmq (other options) + +To actually enable operation, one must set the appropriate options on +the commandline or in the configuration file. + +## Usage + +Currently, the following notifications are supported: + + -zmqpubhashtx=address + -zmqpubhashblock=address + -zmqpubrawblock=address + -zmqpubrawtx=address + +The socket type is PUB and the address must be a valid ZeroMQ socket +address. The same address can be used in more than one notification. + +For instance: + + $ bitcoind -zmqpubhashtx=tcp://127.0.0.1:28332 \ + -zmqpubrawtx=ipc:///tmp/bitcoind.tx.raw + +Each PUB notification has a topic and body, where the header +corresponds to the notification type. For instance, for the +notification `-zmqpubhashtx` the topic is `hashtx` (no null +terminator) and the body is the hexadecimal transaction hash (32 +bytes). + +These options can also be provided in bitcoin.conf. + +ZeroMQ endpoint specifiers for TCP (and others) are documented in the +[ZeroMQ API](http://api.zeromq.org/4-0:_start). + +Client side, then, the ZeroMQ subscriber socket must have the +ZMQ_SUBSCRIBE option set to one or either of these prefixes (for +instance, just `hash`); without doing so will result in no messages +arriving. Please see `contrib/zmq/zmq_sub.py` for a working example. + +## Remarks + +From the perspective of bitcoind, the ZeroMQ socket is write-only; PUB +sockets don't even have a read function. Thus, there is no state +introduced into bitcoind directly. Furthermore, no information is +broadcast that wasn't already received from the public P2P network. + +No authentication or authorization is done on connecting clients; it +is assumed that the ZeroMQ port is exposed only to trusted entities, +using other means such as firewalling. + +Note that when the block chain tip changes, a reorganisation may occur +and just the tip will be notified. It is up to the subscriber to +retrieve the chain from the last known block to the new tip. |
