diff options
Diffstat (limited to 'doc')
| -rw-r--r-- | doc/descriptors.md | 7 | ||||
| -rw-r--r-- | doc/developer-notes.md | 79 | ||||
| -rw-r--r-- | doc/fuzzing.md | 75 |
3 files changed, 126 insertions, 35 deletions
diff --git a/doc/descriptors.md b/doc/descriptors.md index a98f43737e..e31665b129 100644 --- a/doc/descriptors.md +++ b/doc/descriptors.md @@ -10,6 +10,13 @@ Supporting RPCs are: - `deriveaddresses` takes as input a descriptor and computes the corresponding addresses. - `listunspent` outputs a specialized descriptor for the reported unspent outputs. +- `getaddressinfo` outputs a descriptor for solvable addresses (since v0.18). +- `importmulti` takes as input descriptors to import into the wallet + (since v0.18). +- `generatetodescriptor` takes as input a descriptor and generates coins to it + (`regtest` only, since v0.19). +- `utxoupdatepsbt` takes as input descriptors to add information to the psbt + (since v0.19). This document describes the language. For the specifics on usage, see the RPC documentation for the functions mentioned above. diff --git a/doc/developer-notes.md b/doc/developer-notes.md index b50f552e92..3ef35cfcfa 100644 --- a/doc/developer-notes.md +++ b/doc/developer-notes.md @@ -9,6 +9,7 @@ Developer Notes - [Coding Style (C++)](#coding-style-c) - [Coding Style (Python)](#coding-style-python) - [Coding Style (Doxygen-compatible comments)](#coding-style-doxygen-compatible-comments) + - [Generating Documentation](#generating-documentation) - [Development tips and tricks](#development-tips-and-tricks) - [Compiling for debugging](#compiling-for-debugging) - [Compiling for gprof profiling](#compiling-for-gprof-profiling) @@ -35,6 +36,9 @@ Developer Notes - [Source code organization](#source-code-organization) - [GUI](#gui) - [Subtrees](#subtrees) + - [Upgrading LevelDB](#upgrading-leveldb) + - [File Descriptor Counts](#file-descriptor-counts) + - [Consensus Compatibility](#consensus-compatibility) - [Scripted diffs](#scripted-diffs) - [Suggestions and examples](#suggestions-and-examples) - [Release notes](#release-notes) @@ -138,12 +142,17 @@ For example, to describe a function use: ```c++ /** - * ... text ... - * @param[in] arg1 A description - * @param[in] arg2 Another argument description - * @pre Precondition for function... + * ... Description ... + * + * @param[in] arg1 input description... + * @param[in] arg2 input description... + * @param[out] arg3 output description... + * @return Return cases... + * @throws Error type and cases... + * @pre Pre-condition for function... + * @post Post-condition for function... */ -bool function(int arg1, const char *arg2) +bool function(int arg1, const char *arg2, std::string& arg3) ``` A complete list of `@xxx` commands can be found at http://www.doxygen.nl/manual/commands.html. @@ -158,44 +167,73 @@ To describe a class, use the same construct above the class definition: * @see GetWarnings() */ class CAlert -{ ``` To describe a member or variable use: ```c++ -int var; //!< Detailed description after the member +//! Description before the member +int var; ``` or ```c++ -//! Description before the member -int var; +int var; //!< Description after the member ``` Also OK: ```c++ /// -/// ... text ... +/// ... Description ... /// bool function2(int arg1, const char *arg2) ``` -Not OK (used plenty in the current source, but not picked up): +Not picked up by Doxygen: ```c++ // -// ... text ... +// ... Description ... // ``` +Also not picked up by Doxygen: +```c++ +/* + * ... Description ... + */ +``` + A full list of comment syntaxes picked up by Doxygen can be found at http://www.doxygen.nl/manual/docblocks.html, but the above styles are favored. -Documentation can be generated with `make docs` and cleaned up with `make clean-docs`. The resulting files are located in `doc/doxygen/html`; open `index.html` to view the homepage. +Recommendations: -Before running `make docs`, you will need to install dependencies `doxygen` and `dot`. For example, on macOS via Homebrew: -``` -brew install graphviz doxygen -``` +- Avoiding duplicating type and input/output information in function + descriptions. + +- Use backticks (``) to refer to `argument` names in function and + parameter descriptions. + +- Backticks aren't required when referring to functions Doxygen already knows + about; it will build hyperlinks for these automatically. See + http://www.doxygen.nl/manual/autolink.html for complete info. + +- Avoid linking to external documentation; links can break. + +- Javadoc and all valid Doxygen comments are stripped from Doxygen source code + previews (`STRIP_CODE_COMMENTS = YES` in [Doxyfile.in](doc/Doxyfile.in)). If + you want a comment to be preserved, it must instead use `//` or `/* */`. + +### Generating Documentation + +The documentation can be generated with `make docs` and cleaned up with `make +clean-docs`. The resulting files are located in `doc/doxygen/html`; open +`index.html` in that directory to view the homepage. + +Before running `make docs`, you'll need to install these dependencies: + +Linux: `sudo apt install doxygen graphviz` + +MacOS: `brew install doxygen graphviz` Development tips and tricks --------------------------- @@ -919,7 +957,7 @@ introduce accidental changes. Some good examples of scripted-diff: - [scripted-diff: Rename InitInterfaces to NodeContext](https://github.com/bitcoin/bitcoin/commit/301bd41a2e6765b185bd55f4c541f9e27aeea29d) -uses an elegant script to replace occurences of multiple terms in all source files. +uses an elegant script to replace occurrences of multiple terms in all source files. - [scripted-diff: Remove g_connman, g_banman globals](https://github.com/bitcoin/bitcoin/commit/301bd41a2e6765b185bd55f4c541f9e27aeea29d) replaces specific terms in a list of specific source files. @@ -1047,6 +1085,11 @@ A few guidelines for introducing and reviewing new RPC interfaces: new RPC is replacing a deprecated RPC, to avoid both RPCs confusingly showing up in the command list. +- Use *invalid* bech32 addresses for `RPCExamples` help documentation. + + - *Rationale*: Prevent accidental transactions by users and encourage the use + of bech32 addresses by default. + - Use the `UNIX_EPOCH_TIME` constant when describing UNIX epoch time or timestamps in the documentation. diff --git a/doc/fuzzing.md b/doc/fuzzing.md index 50e9251b8d..c34ca4cb59 100644 --- a/doc/fuzzing.md +++ b/doc/fuzzing.md @@ -7,11 +7,8 @@ describe how to use it with AFL and libFuzzer. ## Preparing fuzzing -AFL needs an input directory with examples, and an output directory where it -will place examples that it found. These can be anywhere in the file system, -we'll define environment variables to make it easy to reference them. - -libFuzzer will use the input directory as output directory. +The fuzzer needs some inputs to work on, but the inputs or seeds can be used +interchangeably between libFuzzer and AFL. Extract the example seeds (or other starting inputs) into the inputs directory before starting fuzzing. @@ -21,13 +18,19 @@ git clone https://github.com/bitcoin-core/qa-assets export DIR_FUZZ_IN=$PWD/qa-assets/fuzz_seed_corpus ``` -Only for AFL: +AFL needs an input directory with examples, and an output directory where it +will place examples that it found. These can be anywhere in the file system, +we'll define environment variables to make it easy to reference them. + +So, only for AFL you need to configure the outputs path: ``` mkdir outputs export AFLOUT=$PWD/outputs ``` +libFuzzer will use the input directory as output directory. + ## AFL ### Building AFL @@ -41,6 +44,9 @@ make export AFLPATH=$PWD ``` +For macOS you may need to ignore x86 compilation checks when running `make`: +`AFL_NO_X86=1 make`. + ### Instrumentation To build Bitcoin Core using AFL instrumentation (this assumes that the @@ -48,9 +54,15 @@ To build Bitcoin Core using AFL instrumentation (this assumes that the ``` ./configure --disable-ccache --disable-shared --enable-tests --enable-fuzz CC=${AFLPATH}/afl-gcc CXX=${AFLPATH}/afl-g++ export AFL_HARDEN=1 -cd src/ make ``` + +If you are using clang you will need to substitute `afl-gcc` with `afl-clang` +and `afl-g++` with `afl-clang++`, so the first line above becomes: +``` +./configure --disable-ccache --disable-shared --enable-tests --enable-fuzz CC=${AFLPATH}/afl-clang CXX=${AFLPATH}/afl-clang++ +``` + We disable ccache because we don't want to pollute the ccache with instrumented objects, and similarly don't want to use non-instrumented cached objects linked in. @@ -60,25 +72,32 @@ The fuzzing can be sped up significantly (~200x) by using `afl-clang-fast` and compiling using `afl-clang-fast`/`afl-clang-fast++` the resulting binary will be instrumented in such a way that the AFL features "persistent mode" and "deferred forkserver" can be used. See -https://github.com/mcarpenter/afl/tree/master/llvm_mode for details. +https://github.com/google/AFL/tree/master/llvm_mode for details. ### Fuzzing To start the actual fuzzing use: ``` -export FUZZ_TARGET=fuzz_target_foo # Pick a fuzz_target +export FUZZ_TARGET=bech32 # Pick a fuzz_target mkdir ${AFLOUT}/${FUZZ_TARGET} -$AFLPATH/afl-fuzz -i ${DIR_FUZZ_IN}/${FUZZ_TARGET} -o ${AFLOUT}/${FUZZ_TARGET} -m52 -- test/fuzz/${FUZZ_TARGET} +$AFLPATH/afl-fuzz -i ${DIR_FUZZ_IN}/${FUZZ_TARGET} -o ${AFLOUT}/${FUZZ_TARGET} -m52 -- src/test/fuzz/${FUZZ_TARGET} ``` You may have to change a few kernel parameters to test optimally - `afl-fuzz` will print an error and suggestion if so. +On macOS you may need to set `AFL_NO_FORKSRV=1` to get the target to run. +``` +export FUZZ_TARGET=bech32 # Pick a fuzz_target +mkdir ${AFLOUT}/${FUZZ_TARGET} +AFL_NO_FORKSRV=1 $AFLPATH/afl-fuzz -i ${DIR_FUZZ_IN}/${FUZZ_TARGET} -o ${AFLOUT}/${FUZZ_TARGET} -m52 -- src/test/fuzz/${FUZZ_TARGET} +``` + ## libFuzzer -A recent version of `clang`, the address/undefined sanitizers (ASan/UBSan) and libFuzzer is needed (all -found in the `compiler-rt` runtime libraries package). +A recent version of `clang`, the address/undefined sanitizers (ASan/UBSan) and +libFuzzer is needed (all found in the `compiler-rt` runtime libraries package). To build all fuzz targets with libFuzzer, run @@ -87,11 +106,33 @@ To build all fuzz targets with libFuzzer, run make ``` -The fuzzer needs some inputs to work on, but the inputs or seeds can be used -interchangeably between libFuzzer and AFL. - See https://llvm.org/docs/LibFuzzer.html#running on how to run the libFuzzer instrumented executable. -Alternatively run the script in `./test/fuzz/test_runner.py` and provide it -with the `${DIR_FUZZ_IN}` created earlier. +Alternatively, you can run the script through the fuzzing test harness (only +libFuzzer supported so far). You need to pass it the inputs directory and +the specific test target you want to run. + +``` +./test/fuzz/test_runner.py ${DIR_FUZZ_IN} bech32 +``` + +### macOS hints for libFuzzer + +The default clang/llvm version supplied by Apple on macOS does not include +fuzzing libraries, so macOS users will need to install a full version, for +example using `brew install llvm`. + +Should you run into problems with the address sanitizer, it is possible you +may need to run `./configure` with `--disable-asm` to avoid errors +with certain assembly code from Bitcoin Core's code. See [developer notes on sanitizers](https://github.com/bitcoin/bitcoin/blob/master/doc/developer-notes.md#sanitizers) +for more information. + +You may also need to take care of giving the correct path for clang and +clang++, like `CC=/path/to/clang CXX=/path/to/clang++` if the non-systems +clang does not come first in your path. + +Full configure that was tested on macOS Catalina with `brew` installed `llvm`: +``` +./configure --disable-ccache --enable-fuzz --with-sanitizers=fuzzer,address,undefined CC=/usr/local/opt/llvm/bin/clang CXX=/usr/local/opt/llvm/bin/clang++ --disable-asm +``` |