aboutsummaryrefslogtreecommitdiff
path: root/configure.ac
AgeCommit message (Collapse)Author
2021-02-19Merge #21205: build: actually fail when Boost is missingfanquake
c5da2749e2f7375e292fb0982e8e252ae1adbce3 build: actually stop configure if Boost isn't available (fanquake) cad8b527eaf7a93877e2249960866fd4db2d1c14 build: explicitly install libboost-dev package (fanquake) Pull request description: If Boost is not found via AX_BOOST_BASE, we don't actually stop configuring, only a warning is emitted: ```bash checking for boostlib >= 1.58.0 (105800)... configure: We could not detect the boost libraries (version MINIMUM_REQUIRED_BOOST or higher). If you have a staged boost library (still not installed) please specify $BOOST_ROOT in your environment and do not give a PATH to --with-boost option. If you are sure you have boost installed, then check your version number looking in <boost/version.hpp>. See http://randspringer.de/boost for more documentation. ``` Instead we usually fail when one of the other AX_BOOST_* macros fails to find a library. These macros are slowly being removed, and in any case, it makes more sense to fail earlier if Boost is missing. If Boost is unavailable, the failure now looks like: ```bash checking for boostlib >= 1.58.0 (105800)... configure: We could not detect the boost libraries (version 1.58.0 or higher). If you have a staged boost library (still not installed) please specify $BOOST_ROOT in your environment and do not give a PATH to --with-boost option. If you are sure you have boost installed, then check your version number looking in <boost/version.hpp>. See http://randspringer.de/boost for more documentation. configure: error: Boost is not available! ``` Note that we now just pass the version into AX_BOOST_BASE, which fixes it's display in the output (rather than showing `MINIMUM_REQUIRED_BOOST`). This PR also has a commit that adds `libboost-dev` to our install instructions and CI. This package is currently installed as a side-effect of installing our other libboost-*-dev packages. However as those continue to disappear, it makes sense to install boost-dev explicitly. ACKs for top commit: laanwj: Code review ACK c5da2749e2f7375e292fb0982e8e252ae1adbce3 MarcoFalke: Concept ACK c5da2749e2f7375e292fb0982e8e252ae1adbce3 Tree-SHA512: f866062f9d7d3a2316b6c887f17c664b9cfff41fdc0cb99ca79d641240fb01a5ae0d34140e515bc465219e1b43d5ca84f7c55f48b9c5b45a80ff2795dafd072b
2021-02-17build: actually stop configure if Boost isn't availablefanquake
If Boost is not found via AX_BOOST_BASE, we don't actually stop configuring, only a warning is emitted: ```bash checking for boostlib >= 1.58.0 (105800)... configure: We could not detect the boost libraries (version MINIMUM_REQUIRED_BOOST or higher). If you have a staged boost library (still not installed) please specify $BOOST_ROOT in your environment and do not give a PATH to --with-boost option. If you are sure you have boost installed, then check your version number looking in <boost/version.hpp>. See http://randspringer.de/boost for more documentation. ``` Instead we would usually fail when one of the other AX_BOOST_* macros fails to find a library. These macros are slowly being removed, and in any case, it makes more sense to fail earlier if Boost is missing. If Boost is unavailable, the failure now looks like: ```bash checking for boostlib >= 1.58.0 (105800)... configure: We could not detect the boost libraries (version 1.58.0 or higher). If you have a staged boost library (still not installed) please specify $BOOST_ROOT in your environment and do not give a PATH to --with-boost option. If you are sure you have boost installed, then check your version number looking in <boost/version.hpp>. See http://randspringer.de/boost for more documentation. configure: error: Boost is not available! ``` Note that we now just pass the version into AX_BOOST_BASE, which fixes it's display in the output (rather than MINIMUM_REQUIRED_BOOST).
2021-02-17build: remove mostly pointless BOOST_PROCESS macrofanquake
Performing a series of link checks for a Boost component that is header-only doesn't make much sense, and currently means we just have another confusing Boost macro in our tree. I'm not sure why this was originally done this way; maybe Sjors or luke-jr can elaborate (#15382 (929cda5470f98d1ef85c05b1cad4e2fb9227e3b0))? The macro also has the side-effect of producing confusing error messages. i.e in #20744, the CI is currently failing with: ```bash checking for boostlib >= 1.58.0 (105800) lib path in "/tmp/cirrus-ci-build/depends/x86_64-pc-linux-gnu/lib"... yes checking for boostlib >= 1.58.0 (105800)... yes checking whether the Boost::Process library is available... yes configure: error: Could not find a version of the Boost::Process library! ``` This isn't useful, given there is no such thing as a `Boost::Process` library. This PR just removes the macro entirely, but maintains a `--with-boost-process` (defaulting to off), flag to configure. Hopefully this will also be removed, in favour of `--enable-disable-external-signer` if/when #16546 is merged.
2021-02-12Merge #21064: refactor: use std::shared_mutex & remove Boost ThreadWladimir J. van der Laan
060a2a64d40d75fecb60b7d2b9946a67e46aa6fc ci: remove boost thread installation (fanquake) 06e1d7d81d5a56d136c6fc88f09a2b0654a164f9 build: don't build or use Boost Thread (fanquake) 7097add83c8596f81be9edd66971ffd2486357eb refactor: replace Boost shared_mutex with std shared_mutex in sigcache (fanquake) 8e55981ef834490c438436719f95cbaf888c4914 refactor: replace Boost shared_mutex with std shared_mutex in cuckoocache tests (fanquake) Pull request description: This replaces `boost::shared_mutex` and `boost::unique_lock` with [`std::shared_mutex`](https://en.cppreference.com/w/cpp/thread/shared_mutex) & [`std::unique_lock`](https://en.cppreference.com/w/cpp/thread/unique_lock). Even though [some concerns were raised](https://github.com/bitcoin/bitcoin/issues/16684#issuecomment-726214696) in #16684 with regard to `std::shared_mutex` being unsafe to use across some glibc versions, I still think this change is an improvement. As I mentioned in #21022, I also think trying to restrict standard library feature usage based on bugs in glibc is not only hard to do, but it's not currently clear exactly how we do that in practice (does it also extend to patching out use in our dependencies, should we be implementing more runtime checks for features we are using, when do we consider an affected glibc "old enough" not to worry about? etc). If you take a look through the [glibc bug tracker](https://sourceware.org/bugzilla/describecomponents.cgi?product=glibc) you'll no doubt find plenty of (active) bug reports for standard library code we already using. Obviously not to say we shouldn't try and avoid buggy code where possible. Two other points: [Cory mentioned in #21022](https://github.com/bitcoin/bitcoin/pull/21022#issuecomment-769274179): > It also seems reasonable to me to worry that boost hits the same underlying glibc bug, and we've just not happened to trigger the right conditions yet. Moving away from Boost to the standard library also removes the potential for differences related to Boosts configuration. Boost has multiple versions of `shared_mutex`, and what you end up using, and what it's backed by depends on: * The version of Boost. * The platform you're building for. * Which version of `BOOST_THREAD_VERSION` is defined: (2,3,4 or 5) default=2. (see [here](https://www.boost.org/doc/libs/1_70_0/doc/html/thread/build.html#thread.build.configuration) for some of the differences). * Is `BOOST_THREAD_V2_SHARED_MUTEX` defined? (not by default). If so, you might get the ["less performant, but more robust"](https://github.com/boostorg/thread/issues/230#issuecomment-475937761) version of `shared_mutex`. A lot of these factors are eliminated by our use of depends, but users will have varying configurations. It's also not inconceivable to think that a distro, or some package manager might start defining something like `BOOST_THREAD_VERSION=3`. Boost tried to change the default from 2 to 3 at one point. With this change, we no longer use Boost Thread, so this PR also removes it from depends, the build system, CI etc. Previous similar PRs were #19183 & #20922. The authors are included in the commits here. Also related to #21022 - pthread sanity checking. ACKs for top commit: laanwj: Code review ACK 060a2a64d40d75fecb60b7d2b9946a67e46aa6fc vasild: ACK 060a2a64d40d75fecb60b7d2b9946a67e46aa6fc Tree-SHA512: 572d14d8c9de20bc434511f20d3f431836393ff915b2fe9de5a47a02dca76805ad5c3fc4cceecb4cd43f3ba939a0508178c4e60e62abdbaaa6b3e8db20b75b03
2021-02-12build: consolidate reduced export checksfanquake
2021-02-12build: add building libconsensus to end-of-configure outputfanquake
2021-02-12build: remove AX_GCC_FUNC_ATTRIBUTE test for dllimportfanquake
The result of this test isn't currently used anywhere (we use dllimport based on MSC_VER in libconsensus).
2021-02-12build: test for __declspec(dllexport) in configurefanquake
This should work for GCC and Clang when building for Windows targets.
2021-02-12build: remove duplicate visibility attribute detectionfanquake
We are already testing for this, and our test works correctly with a Darwin target, where the macro does not. Darwin targets do not support "protected" visibility.
2021-02-08Merge #20936: build: build fuzz tests by defaultMarcoFalke
32cbb06676e2957705d3ca7ff5710c9c3ff22c14 build: build fuzz tests by default. (Dan Benjamin) Pull request description: This fixes issue #19388. The changes are as follows: - Add a new flag to configure, --enable-fuzz-binary, which allows building test/fuzz/fuzz regardless of whether we are building to do actual fuzzing - Set -DPROVIDE_MAIN_FUNCTION whenever --enable-fuzz is no - Add the following libraries to FUZZ_SUITE_LD_COMMON: - LIBBITCOIN_WALLET - SQLLITE_LIBS - BDB_LIBS - if necessary, some or all of: - NATPMP_LIBS - MINIUPNPC_LIBS - LIBBITCOIN_ZMQ / ZMQ_LIBS Fixes #19388 ACKs for top commit: MarcoFalke: review ACK 32cbb06676e2957705d3ca7ff5710c9c3ff22c14 📭 Tree-SHA512: c91d713ffe54a3d055daaec02c4317d7e13eed6688821ddc10d894224950b18e276fbdd4acc758c7103b50f34a132b1882b68bc8b60409f97438e0759ced77e1
2021-02-08Merge #20720: build: more robustly check for fcf-protection supportfanquake
e9189a750b237eba1befc6b16c12c2cee3e0176c build: more robustly check for fcf-protection support (fanquake) Pull request description: When using Clang 7, we may end up trying to use the flag when it won't work properly, which can lead to confusing errors. i.e: ```bash /usr/bin/ld: error: ... <corrupt x86 feature size: 0x8> ``` Use `AX_CHECK_LINK_FLAG` & `--fatal-warnings` to ensure we wont use the flag in this case. We do this as even when the error is emitted, compilation succeeds, and the binaries produced will run. This means we can't just check if the compiler accepts the flag, or if compilation succeeds (without or without `-Werror`, and/or passing `-Wl,--fatal-warnings`, which may not be passed through to the linker). This was reported by someone configuring for fuzzing, on Debian 10, where Clang 7 is the default. See here for a minimal example of the problematic behaviour: https://gist.github.com/fanquake/9b33555fcfebef8eb8c0795a71732bc6 ACKs for top commit: pstratem: tested ACK e9189a750b237eba1befc6b16c12c2cee3e0176c MarcoFalke: not an ACK e9189a750b237eba1befc6b16c12c2cee3e0176c , I only tested configure on my system (gcc-10, clang-11): hebasto: ACK e9189a750b237eba1befc6b16c12c2cee3e0176c, tested with clang-7, clang-10 and gcc: the `-fcf-protection=full` is not applied for clang-7, but applied for others compilers. Tree-SHA512: ec24b0cc5523b90139c96cbb33bb98d1e6a24d858c466aa7dfb3c474caf8c50aca53e570fdbc0ff88378406b0ac5d687542452637b1b5fa062e829291b886fc1
2021-02-05build: build fuzz tests by default.Dan Benjamin
This fixes issue #19388. The changes are as follows: - Add a new flag to configure, --enable-fuzz-binary, which allows building test/fuzz/fuzz regardless of whether we are building to do actual fuzzing - Set -DPROVIDE_MAIN_FUNCTION whenever --enable-fuzz is no - Add the following libraries to FUZZ_SUITE_LD_COMMON: - LIBBITCOIN_WALLET - SQLLITE_LIBS - BDB_LIBS - if necessary, some or all of: - NATPMP_LIBS - MINIUPNPC_LIBS - LIBBITCOIN_ZMQ / ZMQ_LIBS
2021-02-04fuzz: Configure check for main function (take 2)MarcoFalke
2021-02-02build: don't build or use Boost Threadfanquake
2021-02-01build: Add -Werror=mismatched-tagsHennadii Stepanov
2021-01-19Support glibc-back-compat on 64-bit POWERLuke Dashjr
2021-01-18Merge #19866: eBPF Linux tracepointsWladimir J. van der Laan
22eb7930a6ae021438aa0b8e750170534944f296 tracing: add tracing framework (William Casarin) 933ab8a720cb9b3341adec4109cffb6dc5b322a5 build: detect sys/sdt.h for eBPF tracing (William Casarin) Pull request description: Instead of writing ad-hoc logging everywhere (eg: #19509), we can take advantage of linux user static defined traces, aka. USDTs ( not the stablecoin :sweat_smile: ) The linux kernel can hook into these tracepoints at runtime, but otherwise they have little to no performance impact. Traces can pass data which can be printed externally via tools such as bpftrace. For example, here's one that prints incoming and outgoing network messages: # Examples ## Network Messages ``` #!/usr/bin/env bpftrace BEGIN { printf("bitcoin net msgs\n"); @start = nsecs; } usdt:./src/bitcoind:net:push_message { $ip = str(arg0); $peer_id = (int64)arg1; $command = str(arg2); $data_len = arg3; $data = buf(arg3,arg4); $t = (nsecs - @start) / 100000; printf("%zu outbound %s %s %zu %d %r\n", $t, $command, $ip, $peer_id, $data_len, $data); @outbound[$command]++; } usdt:./src/bitcoind:net:process_message { $ip = str(arg0); $peer_id = (int64)arg1; $command = str(arg2); $data_len = arg3; $data = buf(arg3,arg4); $t = (nsecs - @start) / 100000; printf("%zu inbound %s %s %zu %d %r\n", $t, $command, $ip, $peer_id, $data_len, $data); @inbound[$ip, $command]++; } ``` $ sudo bpftrace netmsg.bt output: https://jb55.com/s/b11312484b601fb3.txt if you look at the bottom of the output you can see a histogram of all the messages grouped by message type and IP. nice! ## IBD Benchmarking ``` #!/usr/bin/env bpftrace BEGIN { printf("IBD to 500,000 bench\n"); } usdt:./src/bitcoind:CChainState:ConnectBlock { $height = (uint32)arg0; if ($height == 1) { printf("block 1 found, starting benchmark\n"); @start = nsecs; } if ($height >= 500000) { @end = nsecs; @duration = @end - @start; exit(); } } END { printf("duration %d ms\n", @duration / 1000000) } ``` This one hooks into ConnectBlock and prints the IBD time to height 500,000 starting from the first call to ConnectBlock Userspace static tracepoints give lots of flexibility without invasive logging code. It's also more flexible than ad-hoc logging code, allowing you to instrument many different aspects of the system without having to enable per-subsystem logging. Other ideas: tracepoints for lock contention, threads, what else? Let me know what ya'll think and if this is worth adding to bitcoin. ## TODO - [ ] docs? - [x] Integrate systemtap-std-dev/libsystemtap into build (provides the <sys/sdt.h> header) - [x] ~dtrace macos support? (is this still a thing?)~ going to focus on linux for now ACKs for top commit: laanwj: Tested ACK 22eb7930a6ae021438aa0b8e750170534944f296 0xB10C: Tested ACK 22eb7930a6ae021438aa0b8e750170534944f296 Tree-SHA512: 69242242112b679c8a12a22b3bc50252c305894fb3055ae6e13d5f56221d858e58af1d698af55e23b69bdb7abedb5565ac6b45fa5144087b77a17acd04646a75
2021-01-12Merge #19937: signet mining utilityWladimir J. van der Laan
595a34dbea01954cb0372b0210d2fd64357a1762 contrib/signet: Document miner script in README.md (Anthony Towns) ff7dbdc08a11e999e7718b6ac7645ecceef81188 contrib/signet: Add script for generating a signet chain (Anthony Towns) 13762bcc9618138dd28b53c2031defdc9d762d26 Add bitcoin-util command line utility (Anthony Towns) 95d5d5e6257825bb385cee318d5681597f7f7646 rpc: allow getblocktemplate for test chains when unconnected or in IBD (Anthony Towns) 81c54dec20891f2627a49b2e3e785fdaf2a1e664 rpc: update getblocktemplate with signet rule, include signet_challenge (Anthony Towns) Pull request description: Adds `contrib/signet/miner` for mining signet blocks. Adds `bitcoin-util` cli utility, with the idea being it can provide bitcoin related functionality that does not rely on the ability to access a running node. Only subcommand currently is "grind" which takes a hex-encoded header and grinds its nonce until its nBits is satisfied. Updates `getblocktemplate` to include `signet_challenge` field, and makes `getblocktemplate` require the signet rule when invoked on the signet change. Removes connectivity and IBD checks from `getblocktemplate` when applied to a test chain (regtest, testnet, signet). ACKs for top commit: laanwj: code review ACK 595a34dbea01954cb0372b0210d2fd64357a1762 Tree-SHA512: 8d43297710fdc1edc58acd9b53e1bd1671e5724f7097b40ab73653715dc8becc70534c4496cbba9290f4dd6538a7a3d5830eb85f83391ea31a3bb5b9d3378cc3
2021-01-12Add bitcoin-util command line utilityAnthony Towns
2021-01-07Merge #18077: net: Add NAT-PMP port forwarding supportWladimir J. van der Laan
a191e23b8e7f0e19fc0359825eb7ca0d47966fa9 doc: Add release notes (Hennadii Stepanov) ae749d12ddbaf592fbdb65d98ca35a0ff5566992 doc: Add libnatpmp stuff (Hennadii Stepanov) e28f9be87a0f3c59a9184d602fe7947526df6a97 ci: Add libnatpmp-dev package to some builds (Hennadii Stepanov) 5a0185b6c9c838290103314916190a0330ed9a82 gui: Add NAT-PMP network option (Hennadii Stepanov) a39f7336a3b493d46a4486c4c94fdca1b3151370 net: Add -natpmp command line option (Hennadii Stepanov) 28acffd9d53ec437e908abb8c84497a4f41b91ed net: Add NAT-PMP to port mapping loop (Hennadii Stepanov) a8d9f275d0ca64797cc89627f8003b48b3efef63 net: Add libnatpmp support (Hennadii Stepanov) 58e8364dcdc4e57b0caac09f8402e6535301de9b gui: Apply port mapping changes on dialog exit (Hennadii Stepanov) cf151cc68c95a8943e43e3fa4061e176262779e7 scripted-diff: Rename UPnP stuff (Hennadii Stepanov) 4e91b1e24d96e0cdccdd2a3ed034413f3ba6bae6 net: Add flags for port mapping protocols (Hennadii Stepanov) 8b50d1b5bb29b7d1ea0245ba75a8df3144e312dc net: Keep trying to use UPnP when -upnp=1 (Hennadii Stepanov) 28e2961fd6a2a9101fc08fb748430989291aaf7e refactor: Replace magic number with named constant (Hennadii Stepanov) 02ccf69dd6b772423acb343d16ef2bdbb3e3da03 refactor: Move port mapping code to its own module (Hennadii Stepanov) Pull request description: Close #11902 This PR is an alternative to: - #12288 - #15717 To compile with NAT-PMP support on Ubuntu [`libnatpmp-dev`](https://packages.ubuntu.com/source/bionic/libnatpmp) should be available. Log excerpt: ``` 2020-02-05T20:12:28Z [mapport] NAT-PMP: public address = 95.164.65.194 2020-02-05T20:12:28Z [mapport] AddLocal(95.164.65.194:18333,3) 2020-02-05T20:12:28Z [mapport] NAT-PMP: port mapping successful. ``` See: [`libnatpmp`](https://miniupnp.tuxfamily.org/libnatpmp.html) --- Some follow-ups are out of this PR's scope: - mention NAT-PMP library in the version message - ~integrate NAT-PMP into the GUI~ (already [added](https://github.com/bitcoin/bitcoin/pull/18077#issuecomment-589405068)) ACKs for top commit: laanwj: Tested and code review ACK a191e23b8e7f0e19fc0359825eb7ca0d47966fa9 Tree-SHA512: 10e19267c21bf30f20ff1abfc882d526049f0e790b95e12f109dc2bed7c0aef45de03eaf967f4e667e7509be04f1873a5c508087393d947205f3aab2ad6d7cf1
2021-01-07Merge #19055: Add MuHash3072 implementationWladimir J. van der Laan
9815332d5158d69a94abeaf465a2c07bd8e43359 test: Change MuHash Python implementation to match cpp version again (Fabian Jahr) 01297fb3ca57e4b8cbc5a89fc7c6367de33b0bc6 fuzz: Add MuHash consistency fuzz test (Fabian Jahr) b111410914041b72961536c3e4037eba103a8085 test: Add MuHash3072 fuzz test (Fabian Jahr) c1225273857f9fa2e2276396e3f8b3ea48306df3 bench: Add Muhash benchmarks (Fabian Jahr) 7b1242229d1fcc9277238a3aefb3431061c82bfa test: Add MuHash3072 unit tests (Fabian Jahr) adc708c98dbf03b1735edc91f813a36580781a95 crypto: Add MuHash3072 implementation (Fabian Jahr) 0b4d290bf5b0a4d156c523431bf89aaa9ffe92e5 crypto: Add Num3072 implementation (Fabian Jahr) 589f958662a2dcaacdb9a66f1088c74828a39577 build: Check for 128 bit integer support (Fabian Jahr) Pull request description: This is the first split of #18000 which implements the Muhash algorithm and uses it to calculate the UTXO set hash in `gettxoutsetinfo`. ACKs for top commit: laanwj: Code review ACK 9815332d5158d69a94abeaf465a2c07bd8e43359 Tree-SHA512: 4bc090738f0e3d80b74bdd8122e24a8ce80121120fd37c7e4335a73e7ba4fcd7643f2a2d559e2eebf54b8e3a3bd5f12cfb27ba61ded135fda210a07a233eae45
2021-01-07net: Add libnatpmp supportHennadii Stepanov
2021-01-05Merge #19846: build: enable unused member function diagnosticMarcoFalke
819d03b932134ee91df3b0fe98a481a331ce57bf refactor: took out unused member functions (Zero) ed69213c2b2a99023bdee5168614cb8b71990f5f build: enable unused member function diagnostic (Zero) Pull request description: This PR enables the `-Wunused-member-function` compiler diagnostic, as discussed in #19702. > **Notice**: The `unused-member-function` diagnostic is only available on clang. Therefore, clang should be used to test this PR. - [x] Include the `-Wunused-member-function`diagnostic in `./configure.ac`. (ed69213c2b2a99023bdee5168614cb8b71990f5f) - [x] Resolve the reported warnings. (819d03b932134ee91df3b0fe98a481a331ce57bf) Currently, enabling this flag no longer reports the following warnings: > **Note**: output from `make 2>&1 | grep "warning: unused member function" | sort | uniq -c` ``` 1 index/blockfilterindex.cpp:54:5: warning: unused member function 'DBHeightKey' [-Wunused-member-function] 2 script/bitcoinconsensus.cpp:50:9: warning: unused member function 'GetType' [-Wunused-member-function] 1 test/util_tests.cpp:1975:14: warning: unused member function 'operator=' [-Wunused-member-function] ``` All tests have passed locally (from `make check` & `src/test/test_bitcoin`). This PR closes #19702. ACKs for top commit: practicalswift: ACK 819d03b932134ee91df3b0fe98a481a331ce57bf - patch still looks correct :) MarcoFalke: ACK 819d03b932134ee91df3b0fe98a481a331ce57bf pox: Tested ACK 819d03b932134ee91df3b0fe98a481a331ce57bf with clang after `make clean`. No unused member function warnings. theStack: tested ACK 819d03b932134ee91df3b0fe98a481a331ce57bf Tree-SHA512: 5fdfbbb02b3dc618a90a874a5caa5e01e596fc1d14a209e75a6981f01b253f9bca0cfac8fdd758dd7151986609fb76571c3745124a29cfd4f8cbb8d82a07272e
2020-12-30doc: Update license year range to 2021Emil Engler
2020-12-20build: Check for 128 bit integer supportFabian Jahr
Used in MuHash3072 implementation. Co-authored-by: Pieter Wuille <pieter.wuille@gmail.com>
2020-12-20build: more robustly check for fcf-protection supportfanquake
When using Clang 7, we may end up trying to use the flag when it won't work properly, which can lead to confusing errors. i.e: ```bash /usr/bin/ld: error: ... <corrupt x86 feature size: 0x8> ``` Use CHECK_LINK_FLAG & --fatal-warnings to ensure we wont use the flag in this case.
2020-12-18Merge #20434: contrib: Parse ELF directly for symbol and security checksWladimir J. van der Laan
a0a771843fc39c3cc2574a51f009c3391e1808e9 contrib: Changes to checks for PowerPC64 (Luke Dashjr) 634f6ec4eb9997d7bd0f8209fad49a4171d42384 contrib: Parse ELF directly for symbol and security checks (Wladimir J. van der Laan) Pull request description: Instead of the ever-messier text parsing of the output of the readelf tool (which is clearly meant for human consumption not to be machine parseable), parse the ELF binaries directly. Add a small dependency-less ELF parser specific to the checks. This is slightly more secure, too, because it removes potential ambiguity due to misparsing and changes in the output format of `elfread`. It also allows for stricter and more specific ELF format checks in the future. This removes the build-time dependency for `readelf`. It passes the test-security-check for me locally, ~~though I haven't checked on all platforms~~. I've checked that this works on the cross-compile output for all ELF platforms supported by Bitcoin Core at the moment, as well as PPC64 LE and BE. Top commit has no ACKs. Tree-SHA512: 7f9241fec83ee512642fecf5afd90546964561efd8c8c0f99826dcf6660604a4db2b7255e1afb1e9bb0211fd06f5dbad18a6175dfc03e39761a40025118e7bfc
2020-12-16Merge #20470: build: Replace genisoimage with xorrisoWladimir J. van der Laan
7587d11ec959f15f469bd396d4ad2697729b4ccd build: remove cdrkit package from depends (fanquake) 0df98191268fe641dd5d12e3a9f517c0c5cfacdd build: Replace genisoimage with xorriso (fanquake) 22437fc72e78ba3845a3953853d40093de32c395 build: Run libdmg-hfsplus's DMG tool in make deploy (Carl Dong) Pull request description: This is a redo of fanquake's https://github.com/bitcoin/bitcoin/pull/18151, which, aside from switching us from the deprecated `genisoimage` to the maintained `xorriso`, is also necessary for Guix to achieve determinism without using faketime. > xorriso and its mkisofs/genisoimage emulation alter-ego xorrisofs are > more maintained, and has the right toggles for us to achieve output > determinism without using blunt tools like faketime. > > In this commit, we use xorrisofs from the build environment rather than > building it ourselves using depends. This is not necessary and can be > changed in the future. > > From wiki.debian.org/genisoimage?action=recall&rev=11 : > > > The classical command line interface for production of ISO 9660 > > filesystem images is the option set established by program mkisofs. > > For reasons of licensing and other problems with its author, Debian > > ships a fork of mkisofs, called genisoimage, which was split off in > > 2006 and then developed independently. > > > > Meanwhile, genisoimage gets no new features and not even bug fixes. It > > is first choice only if its options -udf or -hfs are needed. > > > > Replacement in most uses cases, especially for bootable ISO 9660 > > filesystems, archiving, and backup, is xorrisofs which starts the -as > > mkisofs emulation mode of program xorriso. ACKs for top commit: laanwj: ACK 7587d11ec959f15f469bd396d4ad2697729b4ccd Tree-SHA512: 62f3aad08fa8bf21192e951d7dd33b24975586d76834cfa3498f4b8cdb586cefec8cab2c073d1951a0884b5e182fd71ef2cf3accad98f84455016776ad3c5422
2020-12-16build: Replace genisoimage with xorrisofanquake
xorriso and its mkisofs/genisoimage emulation alter-ego xorrisofs are more maintained, and has the right toggles for us to achieve output determinism without using blunt tools like faketime. In this commit, we use xorrisofs from the build environment rather than building it ourselves using depends. This is not necessary and can be changed in the future. From https://wiki.debian.org/genisoimage?action=recall&rev=11 : > The classical command line interface for production of ISO 9660 > filesystem images is the option set established by program mkisofs. > For reasons of licensing and other problems with its author, Debian > ships a fork of mkisofs, called genisoimage, which was split off in > 2006 and then developed independently. > > Meanwhile, genisoimage gets no new features and not even bug fixes. It > is first choice only if its options -udf or -hfs are needed. > > Replacement in most uses cases, especially for bootable ISO 9660 > filesystems, archiving, and backup, is xorrisofs which starts the -as > mkisofs emulation mode of program xorriso.
2020-12-15Merge #20560: fuzz: Link all targets onceMarcoFalke
fa13e1b0c52738492310b6b421d8e38cb04da5b1 build: Add option --enable-danger-fuzz-link-all (MarcoFalke) 44444ba759480237172d83f42374c5c29c76eda0 fuzz: Link all targets once (MarcoFalke) Pull request description: Currently the linker is invoked more than 150 times when compiling with `--enable-fuzz`. This is problematic for several reasons: * It wastes disk space north of 20 GB, as all libraries and sanitizers are linked more than 150 times * It wastes CPU time, as the link step can practically not be cached (similar to ccache for object files) * It makes it a blocker to compile the fuzz tests by default for non-fuzz builds #19388, for the aforementioned reasons * The build file is several thousand lines of code, without doing anything meaningful except listing each fuzz target in a highly verbose manner * It makes writing new fuzz tests unnecessarily hard, as build system knowledge is required; Compare that to boost unit tests, which can be added by simply editing an existing cpp file * It encourages fuzz tests that re-use the `buffer` or assume the `buffer` to be concatenations of seeds, which increases complexity of seeds and complexity for the fuzz engine to explore; Thus reducing the effectiveness of the affected fuzz targets Fixes #20088 ACKs for top commit: practicalswift: Tested ACK fa13e1b0c52738492310b6b421d8e38cb04da5b1 sipa: ACK fa13e1b0c52738492310b6b421d8e38cb04da5b1. Reviewed the code changes, and tested the 3 different test_runner.py modes (run once, merge, generate). I also tested building with the new --enable-danger-fuzz-link-all Tree-SHA512: 962ab33269ebd51810924c51266ecc62edd6ddf2fcd9a8c359ed906766f58c3f73c223f8d3cc49f2c60f0053f65e8bdd86ce9c19e673f8c2b3cd676e913f2642
2020-12-14Merge #20594: Fix getauxval calls in randomenv.cppWladimir J. van der Laan
836a3dc02c72f917db5be386b9b4787a59d48610 Avoid weak-linked getauxval support on non-linux platforms (like macOS) (Jonas Schnelli) 41a413b31746cc749f3c64ed8070cea9cc6cfdbe Define correct symbols for getauxval (Jonas Schnelli) Pull request description: PR #20358 made use of the two preprocessor symbols `HAVE_STRONG_GETAUXVAL` as well as `HAVE_WEAK_GETAUXVAL`. These symbols have not been defined in configure.ac. They where only passed selective as CRC32 CPPFLAGS in https://github.com/bitcoin/bitcoin/blob/master/src/Makefile.crc32c.include#L16. PR #20358 would have broken the macOS build since `getauxval` is not supported on macOS (but weak-linking does pass). This PR defines the two symbols correctly and reduces calls to `getauxval` to linux. ACKs for top commit: laanwj: Code review ACK 836a3dc02c72f917db5be386b9b4787a59d48610 jonatack: utACK 836a3dc02c72f917db5be386b9b4787a59d48610 Tree-SHA512: 6527f4a617b937f4c368a3cb1c162f1ac38a6f5e6341295554961eaf322906e9b27398a6f7b00819854ceebb5c828d3e6ce0a779edd769adc4053ce8beda3739
2020-12-14build: Add option --enable-danger-fuzz-link-allMarcoFalke
2020-12-11build: Run libdmg-hfsplus's DMG tool in make deployCarl Dong
Previously, the compression of the .iso file to a .dmg file was done outside of `make deploy' in order to use the faketime-wrapped version of libdmg-hfsplus's DMG tool. Specifying the faketime-wrapped version of the DMG tool to ./configure fixes this and simplifies build scripts.
2020-12-10Merge #20527: build: Do not ignore Homebrew's SQLite on macOSJonas Schnelli
c932e0d67e4b369e4265267da6c8bebac2b6fb53 doc: Update wallet database installation guide for macOS (Hennadii Stepanov) ee7b84e63cbeadd5e680d69ff0548275581e9241 build: Use Homebrew's sqlite package if it is available (Hennadii Stepanov) c96d1f65a552712f8476269ad64a415717ead50d build, refactor: Check that Homebrew's qt5 package is actually installed (Hennadii Stepanov) Pull request description: On master (7ae86b3c6845873ca96650fc69beb4ae5285c801) installed Homebrew `sqlite` package is ignored during build on macOS. This PR fixes this issue and update macOS build docs. Closes #20498. ACKs for top commit: willcl-ark: > > That said, another tACK of [c932e0d](https://github.com/bitcoin/bitcoin/commit/c932e0d67e4b369e4265267da6c8bebac2b6fb53) hebasto: > That said, another tACK of [c932e0d](https://github.com/bitcoin/bitcoin/commit/c932e0d67e4b369e4265267da6c8bebac2b6fb53) laanwj: Code review ACK c932e0d67e4b369e4265267da6c8bebac2b6fb53 jonasschnelli: code review re-ACK c932e0d67e4b369e4265267da6c8bebac2b6fb53 Tree-SHA512: 2563f25534d065556b17ee8c0fca957aea61b5ae288a2aa72743e77607843a45c39f209321e0f05b34283a74d2edcf961cf1dc54a35ed0cc21182304bb961505
2020-12-10Merge #20549: Support make src/bitcoin-node and src/bitcoin-guiWladimir J. van der Laan
206f74e88cfa343d228c1d6596d3846863824ca5 Support make src/bitcoin-node and src/bitcoin-gui (João Barbosa) Pull request description: This change adds the following configure output variables ``` dnl Multi Process BITCOIN_MP_NODE_NAME=bitcoin-node BITCOIN_MP_GUI_NAME=bitcoin-gui ``` and adds support for ```sh make src/bitcoin-node src/bitcoin-gui ``` ACKs for top commit: laanwj: Code review ACK 206f74e88cfa343d228c1d6596d3846863824ca5 Tree-SHA512: 4d1a694b9010ecc267ee955f4475127a58e6da72f30179ec740285ee6fe03cd91dcb6847317a47460dbd548edb88b7da6c7a98eac10f0dabe3ce4e83e0aa8093
2020-12-09configure: output notice that test binary is disabled by fuzzingAndrew Poelstra
2020-12-07Avoid weak-linked getauxval support on non-linux platforms (like macOS)Jonas Schnelli
2020-12-07Define correct symbols for getauxvalJonas Schnelli
2020-12-07build: Use Homebrew's sqlite package if it is availableHennadii Stepanov
2020-12-07build: detect sys/sdt.h for eBPF tracingWilliam Casarin
2020-12-07Merge #20476: contrib: Add test for ELF symbol-checkfanquake
ed1bbcefeaafef460b6e5609de85e13c12e0d2a4 contrib: add MACHO tests to symbol-check tests (fanquake) 5bab08df175db3a4283596515e498fc5a4d0dab9 contrib: Add test for ELF symbol-check (Wladimir J. van der Laan) Pull request description: Check both failure cases: - Use a glibc symbol from a version that is too new - Use a symbol from a library that is not in the allowlist And also check a conforming binary. Adding a similar check for Windows PE can be done in a separate PR. ACKs for top commit: fanquake: ACK ed1bbcefeaafef460b6e5609de85e13c12e0d2a4 Tree-SHA512: fd437612e003922465fe1396efa1fa3a64bd1c7b0a514d2a0a7a0caaaa9fb5cb43e0ed7caec15eb0a3508692c9eb3212d7ba3c7e8180b942dd3e50616ad6e557
2020-12-04build, refactor: Check that Homebrew's qt5 package is actually installedHennadii Stepanov
This change unifies Homebrew packages workflow, and does not change behavior.
2020-12-04Merge #20255: util: Add Assume() identity functionMarcoFalke
faa05854f832405231c9198787a4eafe2cd4c5f0 util: Remove probably misleading TODO (MarcoFalke) fac5efe730ab5b8694920547203deefc5252d294 util: Add Assume() identity function (MarcoFalke) fa861569dcfff9e72686dc5f30b1faa645b4d54e util: Allow Assert(...) to be used in all contexts (practicalswift) Pull request description: This is needed for #20138. Please refer to the added documentation for motivation. ACKs for top commit: practicalswift: cr ACK faa05854f832405231c9198787a4eafe2cd4c5f0 jnewbery: utACK faa05854f832405231c9198787a4eafe2cd4c5f0 hebasto: ACK faa05854f832405231c9198787a4eafe2cd4c5f0, I have reviewed the code and it looks OK, I agree it can be merged. Tree-SHA512: 72165fbd898b92ab9a79b070993fa1faa86c2e3545b6645e72c652bda295d5107bc298d0482bf3aaf0926fc0c3e6418a445c0e073b08568c44231f547f76a688
2020-12-03build: Check that Homebrew's berkeley-db4 package is actually installedHennadii Stepanov
2020-12-03contrib: Add test for ELF symbol-checkWladimir J. van der Laan
Check both failure cases: - Use a glibc symbol from a version that is too new - Use a symbol from a library that is not in the allowlist And also check a conforming binary. Adding a similar check for Windows PE can be done in a separate PR.
2020-12-02Support make src/bitcoin-node and src/bitcoin-guiJoão Barbosa
2020-11-24Don't set BDB flags when configuring withoutJonas Schnelli
2020-11-24util: Add Assume() identity functionMarcoFalke
2020-11-23Merge #20202: wallet: Make BDB support optionalWladimir J. van der Laan
d52f502b1ea1cafa7d58c5517f01dba26ecb7269 Fix mock SQLiteDatabases (Andrew Chow) 99309ab3e96a290359b84f9b657c5115aa3470dd Allow disabling BDB in configure with --without-bdb (Andrew Chow) ee47f11f7399ec3a4330ea1f2fc388c7e32959d6 GUI: Force descriptor wallets when BDB is not compiled (Andrew Chow) 71e40b33bd1e72ccf5d82e1d3f8b481f8e965492 RPC: Require descriptors=True for createwallet when BDB is not compiled (Andrew Chow) 6ebc41bf9cb0184554923e84e1935195d356f2b3 Enforce salvage is only for BDB wallets (Andrew Chow) a58b719cf75e2d97205ec260bcff0d4780fe4fb8 Do not compile BDB things when USE_BDB is defined (Andrew Chow) b33af48210c117a734fc3e1bebeb1c2057645775 Include wallet/bdb.h where it is actually being used (Andrew Chow) Pull request description: Adds a `--without-bdb` option to `configure` which disables the compilation of the BDB stuff. Legacy wallets will not be created when BDB is not compiled. A legacy-sqlite wallet can be loaded, but we will not create them. Based on #20156 to resolve the situation where both `--without-sqlite` and `--without-bdb` are provided. In that case, the wallet is disabled and `--disable-wallet` is effectively set. ACKs for top commit: laanwj: Code review ACK d52f502b1ea1cafa7d58c5517f01dba26ecb7269 Tree-SHA512: 5a92ba7a542acc2e27003e9d4e5940e0d02d5c1f110db06cdcab831372bfd83e8d89c269caff31dd5bff062c1cf5f04683becff12bd23a33be731676f346553d
2020-11-22contrib: Parse ELF directly for symbol and security checksWladimir J. van der Laan
Instead of the ever-messier text parsing of the output of the readelf tool (which is clearly meant for human consumption not to be machine parseable), parse the ELF binaries directly. Add a small dependency-less ELF parser specific to the checks. This is slightly more secure, too, because it removes potential ambiguity due to misparsing and changes in the output format of `elfread`. It also allows for stricter and more specific ELF format checks in the future. This removes the build-time dependency for `readelf`. It passes the test-security-check for me locally, though I haven't checked on all platforms.
generated by cgit v1.2.3 (git 2.26.2) at 2024-11-07 23:14:34 +0000