aboutsummaryrefslogtreecommitdiff
path: root/configure.ac
AgeCommit message (Collapse)Author
2021-03-29build: check if -lsocket is required with *ifaddrsfanquake
2021-03-27Merge #21505: build: Remove unused header from the build systemfanquake
0eabb2abede1230edcc0d7b660d9635135f0b4c1 build: Remove unused header from the build system (Hennadii Stepanov) Pull request description: The only `#include <miniupnpc/miniwget.h>` was removed in #16659. ACKs for top commit: practicalswift: cr ACK 0eabb2abede1230edcc0d7b660d9635135f0b4c1 fanquake: ACK 0eabb2abede1230edcc0d7b660d9635135f0b4c1 Tree-SHA512: 630da03875c851e80286561eae0f966c89624cbb17b90f70e2bec9a69146e79d088fc176e07a4906915770ac1cdb11341a7a431ea7cf6a59d2816e927486f335
2021-03-24Merge #17227: Qt: Add Android packaging supportWladimir J. van der Laan
246774e26459cb3652e308880abdd140e8e9d204 depends: fix Qt precompiled headers bug (Igor Cota) 8e7ad4146d55f472e3d1dacaabb6b7dee704a896 depends: disable Qt Vulkan support on Android (Igor Cota) ba46adaa1abd51798394b5bad3799021adc237d2 CI: add Android APK build to cirrus (Igor Cota) 7563720e30a3052b7ee390f1b3d2874856fd073a CI: add Android APK build script (Igor Cota) ebfb10cb75adb704418d08197681c1e742e63bd5 Qt: add Android packaging support (Igor Cota) Pull request description: ![bitcoin-qt](https://user-images.githubusercontent.com/762502/67396157-62f3d000-f5a7-11e9-8a6f-9425823fcd6c.gif) This PR is the third and final piece of the basic Android support puzzle - it depends on https://github.com/bitcoin/bitcoin/pull/16110 and is related to https://github.com/bitcoin/bitcoin/pull/16883. It introduces an `android` directory under `qt` and a simple way to build an Android package of `bitcoin-qt`: 1. Build depends for Android as described in the [README](https://github.com/bitcoin/bitcoin/blob/master/depends/README.md) 2. Configure with one of the resulting prefixes 3. Run `make && make apk` in `src/qt` The resulting APK files will be in `android/build/outputs/apk`. You can install them manually or with [adb](https://developer.android.com/studio/command-line/adb). One can also open the `android` directory in Android Studio for that integrated development and debugging experience. `BitcoinQtActivity` is your starting point. Under the hood makefile `apk` target: 1. Renames the `bitcoin-qt` binary to `libbitcoin-qt.so` and copies it over to a folder under `android/libs` depending on which prefix and corresponding [ABI](https://developer.android.com/ndk/guides/abis.html#sa) `bitcoin-qt` was built for 2. Takes `libc++_shared.so` from the Android NDK and puts in the same place. It [must be included](https://developer.android.com/ndk/guides/cpp-support) in the APK 3. Extracts Qt for Android Java support files from the `qtbase` archive in `depends/sources` to `android/src` There is also just a tiny bit of `ifdef`'d code to make the Qt Widgets menus usable. It's not pretty but it works and is a stepping stone towards https://github.com/bitcoin/bitcoin/pull/16883. ACKs for top commit: MarcoFalke: cr ACK 246774e264 laanwj: Code review ACK 246774e26459cb3652e308880abdd140e8e9d204 Tree-SHA512: ba30a746576a167545223c35a51ae60bb0838818779fc152c210f5af1413961b2a6ab6af520ff92cbc8dcd5dcb663e81ca960f021218430c1f76397ed4cead6c
2021-03-22build: Remove unused header from the build systemHennadii Stepanov
2021-03-21Qt: add Android packaging supportIgor Cota
Introduce an android directory under qt and allow one to package bitcoin-qt for Android by running make apk. Add bitcoin-qt Android build instructions.
2021-03-12build: don't try and use -fstack-clash-protection on Windowsfanquake
This has never worked with any of the mingw-w64 compilers we use, and the -O0 is causing issues for builders applying spectre mitigations. Recent discussion on https://gcc.gnu.org/bugzilla/show_bug.cgi?id=90458 also indicates that this should just not be used on Windows.
2021-03-11Merge #21007: bitcoind: Add -daemonwait option to wait for initializationWladimir J. van der Laan
e017a913d0d78ef0766cf73586fe7a38488e1a26 bitcoind: Add -daemonwait option to wait for initialization (Wladimir J. van der Laan) c3e6fdee6d39d3f52dec421b48a0ac8bad5006f7 shutdown: Use RAII TokenPipe in shutdown (Wladimir J. van der Laan) 612f746a8ffa265b6877bedbbe21fcbb392f1516 util: Add RAII TokenPipe (Wladimir J. van der Laan) Pull request description: This adds a `-daemonwait` flag that does the same as `-daemon` except that it, from a user perspective, backgrounds the process only after initialization is complete. This is similar to the behaviour of some other software such as c-lightning. This can be useful when the process launching bitcoind wants to guarantee that either the RPC server is running, or that initialization failed, before continuing. The exit code indicates the initialization result. The use of the libc function `daemon()` is replaced by a custom implementation which is inspired by the [glibc implementation](https://github.com/lattera/glibc/blob/master/misc/daemon.c#L44), but which also creates a pipe from the child to the parent process for communication. An additional advantage of having our own `daemon()` implementation is that no MACOS-specific pragmas are needed anymore to silence a deprecation warning. TODO: - [x] Factor out `token_read` and `token_write` to an utility, and use them in `shutdown.cpp` as well—this is exactly the same kind of communication mechanism. - [x] RAII-ify pipe endpoints. - [x] Improve granularity of the `configure.ac` checks. This currently still checks for the function `daemon()` which makes no sense as it's not used. It should check for individual functions such as `fork()` and `setsid()` etc—the former being required, the second optional. - [-] ~~Signal propagation during initialization: if say, pressing Ctrl-C during `-daemonwait` it would be good to pass this SIGINT on to the child process instead of detaching the parent process and letting the child run free.~~ This is not necessary, see https://github.com/bitcoin/bitcoin/pull/21007#issuecomment-769007341. Future: - Consider if it makes sense to use this in the RPC tests (there would be no more need for "is RPC ready" polling loops). I think this is out of scope for this PR. ACKs for top commit: jonatack: Tested ACK e017a913d0d78ef0766cf73586fe7a38488e1a26 checked change since previous review is move-only Tree-SHA512: 53369b8ca2247e4cf3af8cb2cfd5b3399e8e0e3296423d64be987004758162a7ddc1287b01a92d7692328edcb2da4cf05d279b1b4ef61a665b71440ab6a6dbe2
2021-03-04bitcoind: Add -daemonwait option to wait for initializationWladimir J. van der Laan
This adds a `-daemonwait` flag that does the same as `-daemon` except it, from a user perspective, backgrounds the process only after initialization is complete. This can be useful when the process launching bitcoind wants to guarantee that either the RPC server is running, or that initialization failed, before continuing. The exit code indicates the initialization result. This replaces the use of the libc function `daemon()` by a custom implementation which is inspired by the glibc implementation, but also creates a pipe from the child to the parent process for communication. An additional advantage of having our own `daemon()` implementation is that no MACOS-specific pragmas are needed anymore to silence a deprecation warning.
2021-03-04build: compile libnatpmp with -DNATPMP_STATICLIB on Windowsfanquake
This fixes linking issues and mirrors what we do with miniupnpc.
2021-03-02build: Make AM_CONDITIONAL([ENABLE_EXTERNAL_SIGNER]) unconditionalHennadii Stepanov
This change fixes an error when the value of the "use_boost" variable is equal to "no".
2021-03-02build, refactor: Fix indentation for if..then..fiHennadii Stepanov
This makes easier to spot conditional macros.
2021-02-28Merge #21286: build: Bump minimum Qt version to 5.9.5fanquake
faa06ecc9c357428f3fa7a8f86b41b1220809951 build: Bump minimum Qt version to 5.9.5 (Hennadii Stepanov) Pull request description: Close #20104. ACKs for top commit: laanwj: Code review ACK faa06ecc9c357428f3fa7a8f86b41b1220809951 jarolrod: ACK faa06ecc9c357428f3fa7a8f86b41b1220809951 fanquake: ACK faa06ecc9c357428f3fa7a8f86b41b1220809951 - this should be ok to do now. Tree-SHA512: 7295472b5fd37ffb30f044e88c39d375a5a5187d3f2d44d4e73d0eb0c7fd923cf9949c2ddab6cddd8c5da7e375fff38112b6ea9779da4fecce6f024d05ba9c08
2021-02-25Merge #21264: fuzz: Two scripted diff renamesMarcoFalke
fae216a73d71441432cf07212313d18771ce7deb scripted-diff: Rename MakeFuzzingContext to MakeNoLogFileContext (MarcoFalke) fa4fbec03e4395a6ab75ea98a696bee6d768e97e scripted-diff: Rename PROVIDE_MAIN_FUNCTION -> PROVIDE_FUZZ_MAIN_FUNCTION (MarcoFalke) Pull request description: Split out two renames from #21003: * `PROVIDE_FUZZ_MAIN_FUNCTION`. *Reason*: This in only used by fuzzing, so the name should indicate that. * `MakeNoLogFileContext`. *Reason*: Better reflects what the helper does. Also, prepares it to be used in non-fuzz tests in the future. ACKs for top commit: practicalswift: cr ACK fae216a73d71441432cf07212313d18771ce7deb: scripted-diff looks correct Tree-SHA512: e5d347746f5da72b0c86fd4f07ac2e4b3016e88e8c97a830c73bd79d0af6d0245fe7712487fc20344d6cc25958941716c1678124a123930407e3a437265b71df
2021-02-24build, doc: Fix configure script output indentation and typosHennadii Stepanov
2021-02-23build: Bump minimum Qt version to 5.9.5Hennadii Stepanov
2021-02-23Merge #21250: build: make HAVE_O_CLOEXEC available outside LevelDB (bugfix)Wladimir J. van der Laan
9bac71350d98580cc7441957fc7c3fa2f4158553 build: make HAVE_O_CLOEXEC available outside LevelDB (bugfix) (Sebastian Falbesoner) 584fd91d2d294883e6896dbd64a2176528e94581 init: only use pipe2 if availabile, check in configure (Sebastian Falbesoner) Pull request description: The result of the O_CLOEXEC availability check is currently only set in the Makefile and passed to LevelDB (see `LEVELDB_CPPFLAGS_INT` in `src/Makefile.leveldb.include`), but not defined to be used in our codebase. This means that code within the preprocessor conditional `#if HAVE_O_CLOEXEC` was actually never compiled. On the master branch this is currently used for pipe creation in `src/shutdown.cpp`, PR #21007 moves this part to a new module (I found the issue while testing that PR). The fix is similar to the one in #19803, which solved the same problem for HAVE_FDATASYNC. In the course of working on the PR it turned out that pipe2 is not available an all platforms, hence a configure check and a corresponding define HAVE_PIPE2 is introduced and used. The PR can be tested by anyone with a system that has pipe2 and O_CLOEXEC available by putting gibberish into the HAVE_O_CLOEXEC block: on master, everything should compile fine, on PR, the compiler should abort with an error. At least that's my naive way of testing preprocessor logic, happy to hear more sophisticated ways :-) ACKs for top commit: laanwj: Code review ACK 9bac71350d98580cc7441957fc7c3fa2f4158553 Tree-SHA512: aec89faf6ba52b6f014c610ebef7b725d9e967207d58b42a4a71afc9f1268fcb673ecc85b33a2a3debba8105a304dd7edaba4208c5373fcef2ab83e48a170051
2021-02-22build: make HAVE_O_CLOEXEC available outside LevelDB (bugfix)Sebastian Falbesoner
2021-02-22init: only use pipe2 if availabile, check in configureSebastian Falbesoner
2021-02-22scripted-diff: Rename PROVIDE_MAIN_FUNCTION -> PROVIDE_FUZZ_MAIN_FUNCTIONMarcoFalke
-BEGIN VERIFY SCRIPT- sed -i -e 's/PROVIDE_MAIN_FUNCTION/PROVIDE_FUZZ_MAIN_FUNCTION/g' $(git grep -l PROVIDE_MAIN_FUNCTION) -END VERIFY SCRIPT-
2021-02-21configure: add --enable-external-signerSjors Provoost
This option replaces --with-boost-process This prepares external signer support to be disabled by default. It adds a configure option to enable this feature and to check if Boost::Process is present. This also exposes ENABLE_EXTERNAL_SIGNER to the test suite via test/config.ini
2021-02-19Merge #21205: build: actually fail when Boost is missingfanquake
c5da2749e2f7375e292fb0982e8e252ae1adbce3 build: actually stop configure if Boost isn't available (fanquake) cad8b527eaf7a93877e2249960866fd4db2d1c14 build: explicitly install libboost-dev package (fanquake) Pull request description: If Boost is not found via AX_BOOST_BASE, we don't actually stop configuring, only a warning is emitted: ```bash checking for boostlib >= 1.58.0 (105800)... configure: We could not detect the boost libraries (version MINIMUM_REQUIRED_BOOST or higher). If you have a staged boost library (still not installed) please specify $BOOST_ROOT in your environment and do not give a PATH to --with-boost option. If you are sure you have boost installed, then check your version number looking in <boost/version.hpp>. See http://randspringer.de/boost for more documentation. ``` Instead we usually fail when one of the other AX_BOOST_* macros fails to find a library. These macros are slowly being removed, and in any case, it makes more sense to fail earlier if Boost is missing. If Boost is unavailable, the failure now looks like: ```bash checking for boostlib >= 1.58.0 (105800)... configure: We could not detect the boost libraries (version 1.58.0 or higher). If you have a staged boost library (still not installed) please specify $BOOST_ROOT in your environment and do not give a PATH to --with-boost option. If you are sure you have boost installed, then check your version number looking in <boost/version.hpp>. See http://randspringer.de/boost for more documentation. configure: error: Boost is not available! ``` Note that we now just pass the version into AX_BOOST_BASE, which fixes it's display in the output (rather than showing `MINIMUM_REQUIRED_BOOST`). This PR also has a commit that adds `libboost-dev` to our install instructions and CI. This package is currently installed as a side-effect of installing our other libboost-*-dev packages. However as those continue to disappear, it makes sense to install boost-dev explicitly. ACKs for top commit: laanwj: Code review ACK c5da2749e2f7375e292fb0982e8e252ae1adbce3 MarcoFalke: Concept ACK c5da2749e2f7375e292fb0982e8e252ae1adbce3 Tree-SHA512: f866062f9d7d3a2316b6c887f17c664b9cfff41fdc0cb99ca79d641240fb01a5ae0d34140e515bc465219e1b43d5ca84f7c55f48b9c5b45a80ff2795dafd072b
2021-02-17build: actually stop configure if Boost isn't availablefanquake
If Boost is not found via AX_BOOST_BASE, we don't actually stop configuring, only a warning is emitted: ```bash checking for boostlib >= 1.58.0 (105800)... configure: We could not detect the boost libraries (version MINIMUM_REQUIRED_BOOST or higher). If you have a staged boost library (still not installed) please specify $BOOST_ROOT in your environment and do not give a PATH to --with-boost option. If you are sure you have boost installed, then check your version number looking in <boost/version.hpp>. See http://randspringer.de/boost for more documentation. ``` Instead we would usually fail when one of the other AX_BOOST_* macros fails to find a library. These macros are slowly being removed, and in any case, it makes more sense to fail earlier if Boost is missing. If Boost is unavailable, the failure now looks like: ```bash checking for boostlib >= 1.58.0 (105800)... configure: We could not detect the boost libraries (version 1.58.0 or higher). If you have a staged boost library (still not installed) please specify $BOOST_ROOT in your environment and do not give a PATH to --with-boost option. If you are sure you have boost installed, then check your version number looking in <boost/version.hpp>. See http://randspringer.de/boost for more documentation. configure: error: Boost is not available! ``` Note that we now just pass the version into AX_BOOST_BASE, which fixes it's display in the output (rather than MINIMUM_REQUIRED_BOOST).
2021-02-17build: remove mostly pointless BOOST_PROCESS macrofanquake
Performing a series of link checks for a Boost component that is header-only doesn't make much sense, and currently means we just have another confusing Boost macro in our tree. I'm not sure why this was originally done this way; maybe Sjors or luke-jr can elaborate (#15382 (929cda5470f98d1ef85c05b1cad4e2fb9227e3b0))? The macro also has the side-effect of producing confusing error messages. i.e in #20744, the CI is currently failing with: ```bash checking for boostlib >= 1.58.0 (105800) lib path in "/tmp/cirrus-ci-build/depends/x86_64-pc-linux-gnu/lib"... yes checking for boostlib >= 1.58.0 (105800)... yes checking whether the Boost::Process library is available... yes configure: error: Could not find a version of the Boost::Process library! ``` This isn't useful, given there is no such thing as a `Boost::Process` library. This PR just removes the macro entirely, but maintains a `--with-boost-process` (defaulting to off), flag to configure. Hopefully this will also be removed, in favour of `--enable-disable-external-signer` if/when #16546 is merged.
2021-02-12Merge #21064: refactor: use std::shared_mutex & remove Boost ThreadWladimir J. van der Laan
060a2a64d40d75fecb60b7d2b9946a67e46aa6fc ci: remove boost thread installation (fanquake) 06e1d7d81d5a56d136c6fc88f09a2b0654a164f9 build: don't build or use Boost Thread (fanquake) 7097add83c8596f81be9edd66971ffd2486357eb refactor: replace Boost shared_mutex with std shared_mutex in sigcache (fanquake) 8e55981ef834490c438436719f95cbaf888c4914 refactor: replace Boost shared_mutex with std shared_mutex in cuckoocache tests (fanquake) Pull request description: This replaces `boost::shared_mutex` and `boost::unique_lock` with [`std::shared_mutex`](https://en.cppreference.com/w/cpp/thread/shared_mutex) & [`std::unique_lock`](https://en.cppreference.com/w/cpp/thread/unique_lock). Even though [some concerns were raised](https://github.com/bitcoin/bitcoin/issues/16684#issuecomment-726214696) in #16684 with regard to `std::shared_mutex` being unsafe to use across some glibc versions, I still think this change is an improvement. As I mentioned in #21022, I also think trying to restrict standard library feature usage based on bugs in glibc is not only hard to do, but it's not currently clear exactly how we do that in practice (does it also extend to patching out use in our dependencies, should we be implementing more runtime checks for features we are using, when do we consider an affected glibc "old enough" not to worry about? etc). If you take a look through the [glibc bug tracker](https://sourceware.org/bugzilla/describecomponents.cgi?product=glibc) you'll no doubt find plenty of (active) bug reports for standard library code we already using. Obviously not to say we shouldn't try and avoid buggy code where possible. Two other points: [Cory mentioned in #21022](https://github.com/bitcoin/bitcoin/pull/21022#issuecomment-769274179): > It also seems reasonable to me to worry that boost hits the same underlying glibc bug, and we've just not happened to trigger the right conditions yet. Moving away from Boost to the standard library also removes the potential for differences related to Boosts configuration. Boost has multiple versions of `shared_mutex`, and what you end up using, and what it's backed by depends on: * The version of Boost. * The platform you're building for. * Which version of `BOOST_THREAD_VERSION` is defined: (2,3,4 or 5) default=2. (see [here](https://www.boost.org/doc/libs/1_70_0/doc/html/thread/build.html#thread.build.configuration) for some of the differences). * Is `BOOST_THREAD_V2_SHARED_MUTEX` defined? (not by default). If so, you might get the ["less performant, but more robust"](https://github.com/boostorg/thread/issues/230#issuecomment-475937761) version of `shared_mutex`. A lot of these factors are eliminated by our use of depends, but users will have varying configurations. It's also not inconceivable to think that a distro, or some package manager might start defining something like `BOOST_THREAD_VERSION=3`. Boost tried to change the default from 2 to 3 at one point. With this change, we no longer use Boost Thread, so this PR also removes it from depends, the build system, CI etc. Previous similar PRs were #19183 & #20922. The authors are included in the commits here. Also related to #21022 - pthread sanity checking. ACKs for top commit: laanwj: Code review ACK 060a2a64d40d75fecb60b7d2b9946a67e46aa6fc vasild: ACK 060a2a64d40d75fecb60b7d2b9946a67e46aa6fc Tree-SHA512: 572d14d8c9de20bc434511f20d3f431836393ff915b2fe9de5a47a02dca76805ad5c3fc4cceecb4cd43f3ba939a0508178c4e60e62abdbaaa6b3e8db20b75b03
2021-02-12build: consolidate reduced export checksfanquake
2021-02-12build: add building libconsensus to end-of-configure outputfanquake
2021-02-12build: remove AX_GCC_FUNC_ATTRIBUTE test for dllimportfanquake
The result of this test isn't currently used anywhere (we use dllimport based on MSC_VER in libconsensus).
2021-02-12build: test for __declspec(dllexport) in configurefanquake
This should work for GCC and Clang when building for Windows targets.
2021-02-12build: remove duplicate visibility attribute detectionfanquake
We are already testing for this, and our test works correctly with a Darwin target, where the macro does not. Darwin targets do not support "protected" visibility.
2021-02-08Merge #20936: build: build fuzz tests by defaultMarcoFalke
32cbb06676e2957705d3ca7ff5710c9c3ff22c14 build: build fuzz tests by default. (Dan Benjamin) Pull request description: This fixes issue #19388. The changes are as follows: - Add a new flag to configure, --enable-fuzz-binary, which allows building test/fuzz/fuzz regardless of whether we are building to do actual fuzzing - Set -DPROVIDE_MAIN_FUNCTION whenever --enable-fuzz is no - Add the following libraries to FUZZ_SUITE_LD_COMMON: - LIBBITCOIN_WALLET - SQLLITE_LIBS - BDB_LIBS - if necessary, some or all of: - NATPMP_LIBS - MINIUPNPC_LIBS - LIBBITCOIN_ZMQ / ZMQ_LIBS Fixes #19388 ACKs for top commit: MarcoFalke: review ACK 32cbb06676e2957705d3ca7ff5710c9c3ff22c14 📭 Tree-SHA512: c91d713ffe54a3d055daaec02c4317d7e13eed6688821ddc10d894224950b18e276fbdd4acc758c7103b50f34a132b1882b68bc8b60409f97438e0759ced77e1
2021-02-08Merge #20720: build: more robustly check for fcf-protection supportfanquake
e9189a750b237eba1befc6b16c12c2cee3e0176c build: more robustly check for fcf-protection support (fanquake) Pull request description: When using Clang 7, we may end up trying to use the flag when it won't work properly, which can lead to confusing errors. i.e: ```bash /usr/bin/ld: error: ... <corrupt x86 feature size: 0x8> ``` Use `AX_CHECK_LINK_FLAG` & `--fatal-warnings` to ensure we wont use the flag in this case. We do this as even when the error is emitted, compilation succeeds, and the binaries produced will run. This means we can't just check if the compiler accepts the flag, or if compilation succeeds (without or without `-Werror`, and/or passing `-Wl,--fatal-warnings`, which may not be passed through to the linker). This was reported by someone configuring for fuzzing, on Debian 10, where Clang 7 is the default. See here for a minimal example of the problematic behaviour: https://gist.github.com/fanquake/9b33555fcfebef8eb8c0795a71732bc6 ACKs for top commit: pstratem: tested ACK e9189a750b237eba1befc6b16c12c2cee3e0176c MarcoFalke: not an ACK e9189a750b237eba1befc6b16c12c2cee3e0176c , I only tested configure on my system (gcc-10, clang-11): hebasto: ACK e9189a750b237eba1befc6b16c12c2cee3e0176c, tested with clang-7, clang-10 and gcc: the `-fcf-protection=full` is not applied for clang-7, but applied for others compilers. Tree-SHA512: ec24b0cc5523b90139c96cbb33bb98d1e6a24d858c466aa7dfb3c474caf8c50aca53e570fdbc0ff88378406b0ac5d687542452637b1b5fa062e829291b886fc1
2021-02-05build: build fuzz tests by default.Dan Benjamin
This fixes issue #19388. The changes are as follows: - Add a new flag to configure, --enable-fuzz-binary, which allows building test/fuzz/fuzz regardless of whether we are building to do actual fuzzing - Set -DPROVIDE_MAIN_FUNCTION whenever --enable-fuzz is no - Add the following libraries to FUZZ_SUITE_LD_COMMON: - LIBBITCOIN_WALLET - SQLLITE_LIBS - BDB_LIBS - if necessary, some or all of: - NATPMP_LIBS - MINIUPNPC_LIBS - LIBBITCOIN_ZMQ / ZMQ_LIBS
2021-02-04fuzz: Configure check for main function (take 2)MarcoFalke
2021-02-02build: don't build or use Boost Threadfanquake
2021-02-01build: Add -Werror=mismatched-tagsHennadii Stepanov
2021-01-19Support glibc-back-compat on 64-bit POWERLuke Dashjr
2021-01-18Merge #19866: eBPF Linux tracepointsWladimir J. van der Laan
22eb7930a6ae021438aa0b8e750170534944f296 tracing: add tracing framework (William Casarin) 933ab8a720cb9b3341adec4109cffb6dc5b322a5 build: detect sys/sdt.h for eBPF tracing (William Casarin) Pull request description: Instead of writing ad-hoc logging everywhere (eg: #19509), we can take advantage of linux user static defined traces, aka. USDTs ( not the stablecoin :sweat_smile: ) The linux kernel can hook into these tracepoints at runtime, but otherwise they have little to no performance impact. Traces can pass data which can be printed externally via tools such as bpftrace. For example, here's one that prints incoming and outgoing network messages: # Examples ## Network Messages ``` #!/usr/bin/env bpftrace BEGIN { printf("bitcoin net msgs\n"); @start = nsecs; } usdt:./src/bitcoind:net:push_message { $ip = str(arg0); $peer_id = (int64)arg1; $command = str(arg2); $data_len = arg3; $data = buf(arg3,arg4); $t = (nsecs - @start) / 100000; printf("%zu outbound %s %s %zu %d %r\n", $t, $command, $ip, $peer_id, $data_len, $data); @outbound[$command]++; } usdt:./src/bitcoind:net:process_message { $ip = str(arg0); $peer_id = (int64)arg1; $command = str(arg2); $data_len = arg3; $data = buf(arg3,arg4); $t = (nsecs - @start) / 100000; printf("%zu inbound %s %s %zu %d %r\n", $t, $command, $ip, $peer_id, $data_len, $data); @inbound[$ip, $command]++; } ``` $ sudo bpftrace netmsg.bt output: https://jb55.com/s/b11312484b601fb3.txt if you look at the bottom of the output you can see a histogram of all the messages grouped by message type and IP. nice! ## IBD Benchmarking ``` #!/usr/bin/env bpftrace BEGIN { printf("IBD to 500,000 bench\n"); } usdt:./src/bitcoind:CChainState:ConnectBlock { $height = (uint32)arg0; if ($height == 1) { printf("block 1 found, starting benchmark\n"); @start = nsecs; } if ($height >= 500000) { @end = nsecs; @duration = @end - @start; exit(); } } END { printf("duration %d ms\n", @duration / 1000000) } ``` This one hooks into ConnectBlock and prints the IBD time to height 500,000 starting from the first call to ConnectBlock Userspace static tracepoints give lots of flexibility without invasive logging code. It's also more flexible than ad-hoc logging code, allowing you to instrument many different aspects of the system without having to enable per-subsystem logging. Other ideas: tracepoints for lock contention, threads, what else? Let me know what ya'll think and if this is worth adding to bitcoin. ## TODO - [ ] docs? - [x] Integrate systemtap-std-dev/libsystemtap into build (provides the <sys/sdt.h> header) - [x] ~dtrace macos support? (is this still a thing?)~ going to focus on linux for now ACKs for top commit: laanwj: Tested ACK 22eb7930a6ae021438aa0b8e750170534944f296 0xB10C: Tested ACK 22eb7930a6ae021438aa0b8e750170534944f296 Tree-SHA512: 69242242112b679c8a12a22b3bc50252c305894fb3055ae6e13d5f56221d858e58af1d698af55e23b69bdb7abedb5565ac6b45fa5144087b77a17acd04646a75
2021-01-12Merge #19937: signet mining utilityWladimir J. van der Laan
595a34dbea01954cb0372b0210d2fd64357a1762 contrib/signet: Document miner script in README.md (Anthony Towns) ff7dbdc08a11e999e7718b6ac7645ecceef81188 contrib/signet: Add script for generating a signet chain (Anthony Towns) 13762bcc9618138dd28b53c2031defdc9d762d26 Add bitcoin-util command line utility (Anthony Towns) 95d5d5e6257825bb385cee318d5681597f7f7646 rpc: allow getblocktemplate for test chains when unconnected or in IBD (Anthony Towns) 81c54dec20891f2627a49b2e3e785fdaf2a1e664 rpc: update getblocktemplate with signet rule, include signet_challenge (Anthony Towns) Pull request description: Adds `contrib/signet/miner` for mining signet blocks. Adds `bitcoin-util` cli utility, with the idea being it can provide bitcoin related functionality that does not rely on the ability to access a running node. Only subcommand currently is "grind" which takes a hex-encoded header and grinds its nonce until its nBits is satisfied. Updates `getblocktemplate` to include `signet_challenge` field, and makes `getblocktemplate` require the signet rule when invoked on the signet change. Removes connectivity and IBD checks from `getblocktemplate` when applied to a test chain (regtest, testnet, signet). ACKs for top commit: laanwj: code review ACK 595a34dbea01954cb0372b0210d2fd64357a1762 Tree-SHA512: 8d43297710fdc1edc58acd9b53e1bd1671e5724f7097b40ab73653715dc8becc70534c4496cbba9290f4dd6538a7a3d5830eb85f83391ea31a3bb5b9d3378cc3
2021-01-12Add bitcoin-util command line utilityAnthony Towns
2021-01-07Merge #18077: net: Add NAT-PMP port forwarding supportWladimir J. van der Laan
a191e23b8e7f0e19fc0359825eb7ca0d47966fa9 doc: Add release notes (Hennadii Stepanov) ae749d12ddbaf592fbdb65d98ca35a0ff5566992 doc: Add libnatpmp stuff (Hennadii Stepanov) e28f9be87a0f3c59a9184d602fe7947526df6a97 ci: Add libnatpmp-dev package to some builds (Hennadii Stepanov) 5a0185b6c9c838290103314916190a0330ed9a82 gui: Add NAT-PMP network option (Hennadii Stepanov) a39f7336a3b493d46a4486c4c94fdca1b3151370 net: Add -natpmp command line option (Hennadii Stepanov) 28acffd9d53ec437e908abb8c84497a4f41b91ed net: Add NAT-PMP to port mapping loop (Hennadii Stepanov) a8d9f275d0ca64797cc89627f8003b48b3efef63 net: Add libnatpmp support (Hennadii Stepanov) 58e8364dcdc4e57b0caac09f8402e6535301de9b gui: Apply port mapping changes on dialog exit (Hennadii Stepanov) cf151cc68c95a8943e43e3fa4061e176262779e7 scripted-diff: Rename UPnP stuff (Hennadii Stepanov) 4e91b1e24d96e0cdccdd2a3ed034413f3ba6bae6 net: Add flags for port mapping protocols (Hennadii Stepanov) 8b50d1b5bb29b7d1ea0245ba75a8df3144e312dc net: Keep trying to use UPnP when -upnp=1 (Hennadii Stepanov) 28e2961fd6a2a9101fc08fb748430989291aaf7e refactor: Replace magic number with named constant (Hennadii Stepanov) 02ccf69dd6b772423acb343d16ef2bdbb3e3da03 refactor: Move port mapping code to its own module (Hennadii Stepanov) Pull request description: Close #11902 This PR is an alternative to: - #12288 - #15717 To compile with NAT-PMP support on Ubuntu [`libnatpmp-dev`](https://packages.ubuntu.com/source/bionic/libnatpmp) should be available. Log excerpt: ``` 2020-02-05T20:12:28Z [mapport] NAT-PMP: public address = 95.164.65.194 2020-02-05T20:12:28Z [mapport] AddLocal(95.164.65.194:18333,3) 2020-02-05T20:12:28Z [mapport] NAT-PMP: port mapping successful. ``` See: [`libnatpmp`](https://miniupnp.tuxfamily.org/libnatpmp.html) --- Some follow-ups are out of this PR's scope: - mention NAT-PMP library in the version message - ~integrate NAT-PMP into the GUI~ (already [added](https://github.com/bitcoin/bitcoin/pull/18077#issuecomment-589405068)) ACKs for top commit: laanwj: Tested and code review ACK a191e23b8e7f0e19fc0359825eb7ca0d47966fa9 Tree-SHA512: 10e19267c21bf30f20ff1abfc882d526049f0e790b95e12f109dc2bed7c0aef45de03eaf967f4e667e7509be04f1873a5c508087393d947205f3aab2ad6d7cf1
2021-01-07Merge #19055: Add MuHash3072 implementationWladimir J. van der Laan
9815332d5158d69a94abeaf465a2c07bd8e43359 test: Change MuHash Python implementation to match cpp version again (Fabian Jahr) 01297fb3ca57e4b8cbc5a89fc7c6367de33b0bc6 fuzz: Add MuHash consistency fuzz test (Fabian Jahr) b111410914041b72961536c3e4037eba103a8085 test: Add MuHash3072 fuzz test (Fabian Jahr) c1225273857f9fa2e2276396e3f8b3ea48306df3 bench: Add Muhash benchmarks (Fabian Jahr) 7b1242229d1fcc9277238a3aefb3431061c82bfa test: Add MuHash3072 unit tests (Fabian Jahr) adc708c98dbf03b1735edc91f813a36580781a95 crypto: Add MuHash3072 implementation (Fabian Jahr) 0b4d290bf5b0a4d156c523431bf89aaa9ffe92e5 crypto: Add Num3072 implementation (Fabian Jahr) 589f958662a2dcaacdb9a66f1088c74828a39577 build: Check for 128 bit integer support (Fabian Jahr) Pull request description: This is the first split of #18000 which implements the Muhash algorithm and uses it to calculate the UTXO set hash in `gettxoutsetinfo`. ACKs for top commit: laanwj: Code review ACK 9815332d5158d69a94abeaf465a2c07bd8e43359 Tree-SHA512: 4bc090738f0e3d80b74bdd8122e24a8ce80121120fd37c7e4335a73e7ba4fcd7643f2a2d559e2eebf54b8e3a3bd5f12cfb27ba61ded135fda210a07a233eae45
2021-01-07net: Add libnatpmp supportHennadii Stepanov
2021-01-05Merge #19846: build: enable unused member function diagnosticMarcoFalke
819d03b932134ee91df3b0fe98a481a331ce57bf refactor: took out unused member functions (Zero) ed69213c2b2a99023bdee5168614cb8b71990f5f build: enable unused member function diagnostic (Zero) Pull request description: This PR enables the `-Wunused-member-function` compiler diagnostic, as discussed in #19702. > **Notice**: The `unused-member-function` diagnostic is only available on clang. Therefore, clang should be used to test this PR. - [x] Include the `-Wunused-member-function`diagnostic in `./configure.ac`. (ed69213c2b2a99023bdee5168614cb8b71990f5f) - [x] Resolve the reported warnings. (819d03b932134ee91df3b0fe98a481a331ce57bf) Currently, enabling this flag no longer reports the following warnings: > **Note**: output from `make 2>&1 | grep "warning: unused member function" | sort | uniq -c` ``` 1 index/blockfilterindex.cpp:54:5: warning: unused member function 'DBHeightKey' [-Wunused-member-function] 2 script/bitcoinconsensus.cpp:50:9: warning: unused member function 'GetType' [-Wunused-member-function] 1 test/util_tests.cpp:1975:14: warning: unused member function 'operator=' [-Wunused-member-function] ``` All tests have passed locally (from `make check` & `src/test/test_bitcoin`). This PR closes #19702. ACKs for top commit: practicalswift: ACK 819d03b932134ee91df3b0fe98a481a331ce57bf - patch still looks correct :) MarcoFalke: ACK 819d03b932134ee91df3b0fe98a481a331ce57bf pox: Tested ACK 819d03b932134ee91df3b0fe98a481a331ce57bf with clang after `make clean`. No unused member function warnings. theStack: tested ACK 819d03b932134ee91df3b0fe98a481a331ce57bf Tree-SHA512: 5fdfbbb02b3dc618a90a874a5caa5e01e596fc1d14a209e75a6981f01b253f9bca0cfac8fdd758dd7151986609fb76571c3745124a29cfd4f8cbb8d82a07272e
2020-12-30doc: Update license year range to 2021Emil Engler
2020-12-20build: Check for 128 bit integer supportFabian Jahr
Used in MuHash3072 implementation. Co-authored-by: Pieter Wuille <pieter.wuille@gmail.com>
2020-12-20build: more robustly check for fcf-protection supportfanquake
When using Clang 7, we may end up trying to use the flag when it won't work properly, which can lead to confusing errors. i.e: ```bash /usr/bin/ld: error: ... <corrupt x86 feature size: 0x8> ``` Use CHECK_LINK_FLAG & --fatal-warnings to ensure we wont use the flag in this case.
2020-12-18Merge #20434: contrib: Parse ELF directly for symbol and security checksWladimir J. van der Laan
a0a771843fc39c3cc2574a51f009c3391e1808e9 contrib: Changes to checks for PowerPC64 (Luke Dashjr) 634f6ec4eb9997d7bd0f8209fad49a4171d42384 contrib: Parse ELF directly for symbol and security checks (Wladimir J. van der Laan) Pull request description: Instead of the ever-messier text parsing of the output of the readelf tool (which is clearly meant for human consumption not to be machine parseable), parse the ELF binaries directly. Add a small dependency-less ELF parser specific to the checks. This is slightly more secure, too, because it removes potential ambiguity due to misparsing and changes in the output format of `elfread`. It also allows for stricter and more specific ELF format checks in the future. This removes the build-time dependency for `readelf`. It passes the test-security-check for me locally, ~~though I haven't checked on all platforms~~. I've checked that this works on the cross-compile output for all ELF platforms supported by Bitcoin Core at the moment, as well as PPC64 LE and BE. Top commit has no ACKs. Tree-SHA512: 7f9241fec83ee512642fecf5afd90546964561efd8c8c0f99826dcf6660604a4db2b7255e1afb1e9bb0211fd06f5dbad18a6175dfc03e39761a40025118e7bfc
2020-12-16Merge #20470: build: Replace genisoimage with xorrisoWladimir J. van der Laan
7587d11ec959f15f469bd396d4ad2697729b4ccd build: remove cdrkit package from depends (fanquake) 0df98191268fe641dd5d12e3a9f517c0c5cfacdd build: Replace genisoimage with xorriso (fanquake) 22437fc72e78ba3845a3953853d40093de32c395 build: Run libdmg-hfsplus's DMG tool in make deploy (Carl Dong) Pull request description: This is a redo of fanquake's https://github.com/bitcoin/bitcoin/pull/18151, which, aside from switching us from the deprecated `genisoimage` to the maintained `xorriso`, is also necessary for Guix to achieve determinism without using faketime. > xorriso and its mkisofs/genisoimage emulation alter-ego xorrisofs are > more maintained, and has the right toggles for us to achieve output > determinism without using blunt tools like faketime. > > In this commit, we use xorrisofs from the build environment rather than > building it ourselves using depends. This is not necessary and can be > changed in the future. > > From wiki.debian.org/genisoimage?action=recall&rev=11 : > > > The classical command line interface for production of ISO 9660 > > filesystem images is the option set established by program mkisofs. > > For reasons of licensing and other problems with its author, Debian > > ships a fork of mkisofs, called genisoimage, which was split off in > > 2006 and then developed independently. > > > > Meanwhile, genisoimage gets no new features and not even bug fixes. It > > is first choice only if its options -udf or -hfs are needed. > > > > Replacement in most uses cases, especially for bootable ISO 9660 > > filesystems, archiving, and backup, is xorrisofs which starts the -as > > mkisofs emulation mode of program xorriso. ACKs for top commit: laanwj: ACK 7587d11ec959f15f469bd396d4ad2697729b4ccd Tree-SHA512: 62f3aad08fa8bf21192e951d7dd33b24975586d76834cfa3498f4b8cdb586cefec8cab2c073d1951a0884b5e182fd71ef2cf3accad98f84455016776ad3c5422
2020-12-16build: Replace genisoimage with xorrisofanquake
xorriso and its mkisofs/genisoimage emulation alter-ego xorrisofs are more maintained, and has the right toggles for us to achieve output determinism without using blunt tools like faketime. In this commit, we use xorrisofs from the build environment rather than building it ourselves using depends. This is not necessary and can be changed in the future. From https://wiki.debian.org/genisoimage?action=recall&rev=11 : > The classical command line interface for production of ISO 9660 > filesystem images is the option set established by program mkisofs. > For reasons of licensing and other problems with its author, Debian > ships a fork of mkisofs, called genisoimage, which was split off in > 2006 and then developed independently. > > Meanwhile, genisoimage gets no new features and not even bug fixes. It > is first choice only if its options -udf or -hfs are needed. > > Replacement in most uses cases, especially for bootable ISO 9660 > filesystems, archiving, and backup, is xorrisofs which starts the -as > mkisofs emulation mode of program xorriso.
2020-12-15Merge #20560: fuzz: Link all targets onceMarcoFalke
fa13e1b0c52738492310b6b421d8e38cb04da5b1 build: Add option --enable-danger-fuzz-link-all (MarcoFalke) 44444ba759480237172d83f42374c5c29c76eda0 fuzz: Link all targets once (MarcoFalke) Pull request description: Currently the linker is invoked more than 150 times when compiling with `--enable-fuzz`. This is problematic for several reasons: * It wastes disk space north of 20 GB, as all libraries and sanitizers are linked more than 150 times * It wastes CPU time, as the link step can practically not be cached (similar to ccache for object files) * It makes it a blocker to compile the fuzz tests by default for non-fuzz builds #19388, for the aforementioned reasons * The build file is several thousand lines of code, without doing anything meaningful except listing each fuzz target in a highly verbose manner * It makes writing new fuzz tests unnecessarily hard, as build system knowledge is required; Compare that to boost unit tests, which can be added by simply editing an existing cpp file * It encourages fuzz tests that re-use the `buffer` or assume the `buffer` to be concatenations of seeds, which increases complexity of seeds and complexity for the fuzz engine to explore; Thus reducing the effectiveness of the affected fuzz targets Fixes #20088 ACKs for top commit: practicalswift: Tested ACK fa13e1b0c52738492310b6b421d8e38cb04da5b1 sipa: ACK fa13e1b0c52738492310b6b421d8e38cb04da5b1. Reviewed the code changes, and tested the 3 different test_runner.py modes (run once, merge, generate). I also tested building with the new --enable-danger-fuzz-link-all Tree-SHA512: 962ab33269ebd51810924c51266ecc62edd6ddf2fcd9a8c359ed906766f58c3f73c223f8d3cc49f2c60f0053f65e8bdd86ce9c19e673f8c2b3cd676e913f2642
generated by cgit v1.2.3 (git 2.26.2) at 2024-11-20 14:28:42 +0000