aboutsummaryrefslogtreecommitdiff
path: root/contrib/devtools
AgeCommit message (Collapse)Author
2022-05-04Merge bitcoin/bitcoin#22235: script: add script to generate example bitcoin.conflaanwj
b42643c2537ffbe99d6d94fb1fb3b7f9d5234f93 doc: update init.cpp -conf help text (josibake) 970b9987ad5bcb72e581c40a7cdd408d94a48c81 doc: update devtools, release-process readmes (josibake) 50635d27b45d125b6264ac2abfbd6a1129c7228f build: include bitcoin.conf in build outputs (josibake) 6aac946f49aea243de1dc50631bb72f0186bbf58 doc: update bitcoin-conf.md (Josiah Baker) 1c7e820ded0846ef6ab4be9616b0de452336ef64 script: add script to generate example bitcoin.conf (josibake) b483084d866c16d97a34251ae652bac94f85f61d doc: replace bitcoin.conf with placeholder file (josibake) Pull request description: create a script for parsing the output from `bitcoind --help` to create an example conf file for new users ## problem per #10746 , `bitcoin.conf` not being put into the data directory during installation causes some confusion for users when running bitcoin. in the discussion on the issue, one proposed solution was to have an example config file and instruct users to `cp` it into their data directory after startup. in addition to #10746 , there have been other requests for a "skeleton config file" (https://github.com/bitcoin/bitcoin/issues/19641) to help users get started with configuring bitcoind. the main issue with an example config file is that it creates a second source of truth regarding what options are available for configuring bitcoind. this means any changes to the options (including the addition or removal of options) would have to be updated for the command line and also updated in the example file. this PR addresses this issue by providing a script to generate an example file directly from the `bitcoind --help` on-demand by running `contrib/devtools/gen-bitcoin-conf.sh`. this solution was originally proposed on #10746 and would also solve #19641 . this guarantees any changes made to the command-line options or the command-line options help would also be reflected in the example file after compiling and running the script. the main purpose of this script is to generate a config file to be included with releases, same as `gen-manpages.sh`. this ensures every release also includes an up-to-date, full example config file for users to edit. the script is also available for users who compile from source for generating an example config for their compiled binary. ## special considerations this removes the `bitcoin.conf` example file from the repo as it is now generated by this script. the original example file did contain extra text related to how to use certain options but going forward all option help docs should be moved into `init.cpp` this also edits `init.cpp` to have the option help indicate that `-conf` is not usable from the config file. this is similar to how `-includeconf` 's help indicates it cannot be used from the command line ACKs for top commit: laanwj: Tested and code review ACK b42643c2537ffbe99d6d94fb1fb3b7f9d5234f93 Tree-SHA512: 4546e0cef92aa1398da553294ce4712d02e616dd72dcbe0b921af474e54f24750464ec813661f1283802472d1e8774e634dd1cc26fbf1f13286d3e0406c02c09
2022-05-04doc: update devtools, release-process readmesjosibake
include running `gen-bitcoin-conf.sh` as part of the release process.
2022-05-04Merge bitcoin/bitcoin#24993: test, contrib, refactor: use `with` when ↵laanwj
opening a file 027aab663aaca32818051d456c3900326377281c test, contrib, refactor: use `with` when opening a file (brunoerg) Pull request description: When manipulating a file in Python without using `with()`, you have to close the file manually, so this PR does it in `get_block_hashes` (`contrib/linearize/linearize-data.py`). Edit: this PR does it for all occurances that previously weren't using `with`. ACKs for top commit: laanwj: Code review ACK 027aab663aaca32818051d456c3900326377281c Tree-SHA512: 879400968e0013e8678ec16f1fe5d0963a73c1e0d442ca34802d885214f0783d2e9a9b500fc6be7c3b93560a367b6a3d685eee24d2f9ce53fddf064ea6feecf8
2022-05-02script: add script to generate example bitcoin.confjosibake
this ensures bitcoind option help is the source of truth and also gives an example conf file for users to customize and copy to their data directory. closes #10746
2022-04-27test, contrib, refactor: use `with` when opening a filebrunoerg
2022-04-20tidy: Add include-what-you-usefanquake
2022-03-28contrib: use LIEF 0.12.0 for symbol and security checksfanquake
2022-03-07contrib: fix implicit function decleration in win symbol checkfanquake
```bash test3.c: In function 'main': test3.c:6:21: warning: implicit declaration of function 'CoFreeUnusedLibrariesEx' [-Wimplicit-function-declaration] 6 | CoFreeUnusedLibrariesEx(0,0); ```
2022-02-22contrib: address gen-manpages feedback from #24263fanquake
Co-authored-by: Carl Dong <contact@carldong.me>
2022-02-04doc: Fix gen-manpages, rewrite in Pythonlaanwj
Rewrite the manual page generation script in Python. This: - Solves '-' stripping issue (fixes #22681) - Makes that copyright footer is generated again
2022-01-26contrib: support arm64 darwin in security checksfanquake
2022-01-26build: use macOS 11 SDK (Xcode 12.2)fanquake
This should be sufficient to support building for Apple ARM when cross-compiling.
2022-01-25Merge bitcoin/bitcoin#23839: Linux: build with and test for control flow ↵laanwj
instrumentation on x86_64 5a8f907c93f158c54e58706d6d8c6b0a1a3205ba scripts: add CONTROL_FLOW to ELF security checks (fanquake) e13f8f775d9e8da04db0e072040d11237c5d1089 build: build x86_64 Linux Boost with -fcf-protection=full (fanquake) 6ca5efa8ed1ed9d5978bed0baf3c63dc2d406ec5 script rename control flow check to MACHO specific (fanquake) Pull request description: Closes #21888. TODO: * Duplication in security-check-tests Guix build: ```bash bash-5.1# find guix-build-$(git rev-parse --short=12 HEAD)/output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum 8025e2e6859247eaf865a4a5009d0a39570ec5e8ab51739fa4da0d8ee4ab2117 guix-build-5a8f907c93f1/output/aarch64-linux-gnu/SHA256SUMS.part 75576482994493262dbf7d3567b0450c0804bdc75c186d6b4c6a856bd0d8f0a3 guix-build-5a8f907c93f1/output/aarch64-linux-gnu/bitcoin-5a8f907c93f1-aarch64-linux-gnu-debug.tar.gz 7cb89c9327cc540859334b597c041a2250156e3e83ce2aa7d16177376056302f guix-build-5a8f907c93f1/output/aarch64-linux-gnu/bitcoin-5a8f907c93f1-aarch64-linux-gnu.tar.gz e264053e4c7a5d65340dd7bbd2c664bcd596292ab80f00a0bf0026bfe0c480bc guix-build-5a8f907c93f1/output/arm-linux-gnueabihf/SHA256SUMS.part 6c3ee9b3c0c4583620301b183226678f1618605dd07dbed0bbdf7d06d3591314 guix-build-5a8f907c93f1/output/arm-linux-gnueabihf/bitcoin-5a8f907c93f1-arm-linux-gnueabihf-debug.tar.gz f217cb8d8e233a8dbdbfde7dabf12c5d867d7de53e8c652b8ed1d4a55da82fd9 guix-build-5a8f907c93f1/output/arm-linux-gnueabihf/bitcoin-5a8f907c93f1-arm-linux-gnueabihf.tar.gz 48b4b2a1b52b3098f4e92c11cb60f1e5e9696a2a960560cd6adea72277eaa4a4 guix-build-5a8f907c93f1/output/dist-archive/bitcoin-5a8f907c93f1.tar.gz 40832db2446e129879caa9fbc9d682c53069876dbb2e0d4d76592e5dcb40bb12 guix-build-5a8f907c93f1/output/powerpc64-linux-gnu/SHA256SUMS.part 26fdfa9b7b77d5db415ef34054cf6f6d020a5dab73239db6dd05539f654bc5d5 guix-build-5a8f907c93f1/output/powerpc64-linux-gnu/bitcoin-5a8f907c93f1-powerpc64-linux-gnu-debug.tar.gz 3adbdb9d3eb1cb5f9adc38b29450054f286bd6d74cef8619adaee89494853605 guix-build-5a8f907c93f1/output/powerpc64-linux-gnu/bitcoin-5a8f907c93f1-powerpc64-linux-gnu.tar.gz 8ec2baf82483a698350bfdabf530cd9b5241690c916f597c746210e95ac451de guix-build-5a8f907c93f1/output/powerpc64le-linux-gnu/SHA256SUMS.part 1797e75e1f66ec9068fa4e57e0108960475e863f8f054fbe854358b1f995c4df guix-build-5a8f907c93f1/output/powerpc64le-linux-gnu/bitcoin-5a8f907c93f1-powerpc64le-linux-gnu-debug.tar.gz 448bd289ef26c777a1fc4498e7ba7fb17d0f6a932dcac91b2f89cbba63704bb8 guix-build-5a8f907c93f1/output/powerpc64le-linux-gnu/bitcoin-5a8f907c93f1-powerpc64le-linux-gnu.tar.gz a23ee91eeae515c2a6a31eb25d659fab833839aaafa4676ccee364bdad2a468c guix-build-5a8f907c93f1/output/riscv64-linux-gnu/SHA256SUMS.part 82df1b6d5020d0af8268ecc8e823f752f20dec308277763b2dd675804dfa4bbd guix-build-5a8f907c93f1/output/riscv64-linux-gnu/bitcoin-5a8f907c93f1-riscv64-linux-gnu-debug.tar.gz 09a6098ce83896a6ee6d5c8aff12eaca51595bd724c8e0b2a6f90b6410dc168c guix-build-5a8f907c93f1/output/riscv64-linux-gnu/bitcoin-5a8f907c93f1-riscv64-linux-gnu.tar.gz 4fac2951f80eaa2bd1747a263be6be1b76282cac5062f7d86db631a2fb80f8db guix-build-5a8f907c93f1/output/x86_64-apple-darwin/SHA256SUMS.part 3392f417b09efca5916c384f0b2d0c177a72ec4921399c62e84484f0054cc8c4 guix-build-5a8f907c93f1/output/x86_64-apple-darwin/bitcoin-5a8f907c93f1-osx-unsigned.dmg bfb4f8ade6107996ec4bc9efdb53959151b8fb19b6790c34472fe218dd02383d guix-build-5a8f907c93f1/output/x86_64-apple-darwin/bitcoin-5a8f907c93f1-osx-unsigned.tar.gz 4de92e149bad46fc863efb3b650753d194aad96be991d020e0b859452cf27457 guix-build-5a8f907c93f1/output/x86_64-apple-darwin/bitcoin-5a8f907c93f1-osx64.tar.gz 9329549a2f275a59e329afc8744ff8cbc297f3042e0ad03b47626805c7aae2e8 guix-build-5a8f907c93f1/output/x86_64-linux-gnu/SHA256SUMS.part d87deb1eca8a1d3780f02edde78350d27f700e13c7ff444be0bfb34e7369904a guix-build-5a8f907c93f1/output/x86_64-linux-gnu/bitcoin-5a8f907c93f1-x86_64-linux-gnu-debug.tar.gz 76f3bc2fad010d9373e854d941e8205b68fa6c9a8ecaff34c4978ae3ae76c806 guix-build-5a8f907c93f1/output/x86_64-linux-gnu/bitcoin-5a8f907c93f1-x86_64-linux-gnu.tar.gz 2948631081c4bd475529da2b7bc2b32b5aa2e44de46dfdaa6cf9432b3c9fe869 guix-build-5a8f907c93f1/output/x86_64-w64-mingw32/SHA256SUMS.part c5d56fac163b73c00e9745aa7b0f9a0bd9fcac7517e39b677869b6e76faf7218 guix-build-5a8f907c93f1/output/x86_64-w64-mingw32/bitcoin-5a8f907c93f1-win-unsigned.tar.gz f12c7cd72511544c67f4934981c090cee0d9c17e931c059edbbbef6e843dd651 guix-build-5a8f907c93f1/output/x86_64-w64-mingw32/bitcoin-5a8f907c93f1-win64-debug.zip eb952cc4554f92ba6787353f4305d7cbcb1c6dafc4a3867b3088106252a573b8 guix-build-5a8f907c93f1/output/x86_64-w64-mingw32/bitcoin-5a8f907c93f1-win64-setup-unsigned.exe 7bd2b071f1cdf9410535e6a10dd1da519f942bd2c1e47ed52c5b8c4f977e8f27 guix-build-5a8f907c93f1/output/x86_64-w64-mingw32/bitcoin-5a8f907c93f1-win64.zip ``` ACKs for top commit: laanwj: Code review ACK 5a8f907c93f158c54e58706d6d8c6b0a1a3205ba hebasto: ACK 5a8f907c93f158c54e58706d6d8c6b0a1a3205ba, I have reviewed the code and it looks OK, I agree it can be merged. Tree-SHA512: 75702ac01175ccf08e73b5c3bce266cee9dd15ccf8fb38d46108cfada156de9a8c18e27d996f9343ae33f7a75a6904d335cbe25256d90af1f4ccbe72ce1788bb
2022-01-05guix: use GCC 10 (over GCC 8) to build releasesfanquake
This currently points to the version-1.4.0 branch.
2022-01-04scripts: add CONTROL_FLOW to ELF security checksfanquake
2022-01-04script rename control flow check to MACHO specificfanquake
2022-01-03scripts: make security checks architecture independentfanquake
This paves the way for using and checking for architecture dependent flags like -fcf-protection on x86_64 Linux and -mbranch-protection on 64 bit ARM.
2021-12-30scripted-diff: Bump copyright headersHennadii Stepanov
-BEGIN VERIFY SCRIPT- ./contrib/devtools/copyright_header.py update ./ -END VERIFY SCRIPT- Commits of previous years: * 2020: fa0074e2d82928016a43ca408717154a1c70a4db * 2019: aaaaad6ac95b402fe18d019d67897ced6b316ee0
2021-12-30script: Fix copyright_header.pyHennadii Stepanov
This change prevents updating copyright years from "2021" to "2021-2017".
2021-12-18scripts: match on exe type over str in symbol-check.pyfanquake
2021-12-18scripts: match on exe type over str in security-check.pyfanquake
2021-12-03build: qt 5.15.2fanquake
2021-11-17scripts: test for PE control flow instrumentationfanquake
2021-11-16build: require glibc 2.18+ for release buildsfanquake
From what I can see the only platform this drops support for is CentOS 7. CentOS 7 reached the end of it's "full update" support at the end of 2020. It does receive maintenance updates until 2024, however I don't think supporting glibc 2.17 until 2024 is realistic. Note that anyone wanting to self-compile and target a glibc 2.17 runtime could build with --disable-threadlocal. glibc 2.18 was released in August 2013. https://sourceware.org/legacy-ml/libc-alpha/2013-08/msg00160.html
2021-11-13test: Enable SC2086 shellcheck ruleHennadii Stepanov
2021-10-21Add minisketch dependencyGleb Naumenko
2021-10-16lint: enable mypy checking for missing importsjosibake
Achieve this by adding some ignore, and making data/ importable. Co-authored-by: João Barbosa <joao.paulo.barbosa@gmail.com>
2021-10-13symbol-check: Check requested ELF interpreterCarl Dong
It is important that binaries request a standard interpreter location where most distros would place the linker-loader. Otherwise, the user would be met with a very confusing message: bash: <path>/<to>/bitcoind: No such file or directory When really it's the interpreter that's not found.
2021-10-12scripts: remove pixie.pyfanquake
2021-10-12scripts: only parse the binary once in security-check.pyfanquake
2021-10-12scripts: use LIEF for ELF checks in security-check.pyfanquake
2021-10-12scripts: only parse the binary once in symbol-check.pyfanquake
2021-10-12scripts: use LIEF for ELF checks in symbol-check.pyfanquake
Co-authored-by: Carl Dong <contact@carldong.me>
2021-10-12test-*-check: Pass in *FLAGS and compile with themCarl Dong
These test-*-check scripts should compile "test" binaries in a way that is as close to what autotools would do, since the goal is to make sure that if we run the *-check script, they can correctly detect flaws in binaries which are compiled by our autotools-based system. Therefore, we should emulate what happens when the binary is linked in autotools, meaning that for C binaries, we need to supply the CFLAGS, CPPFLAGS, and LDFLAGS flags in that order. Note to future developers: perhaps it'd be nice to have these test-*-check scripts be part of configure.ac to avoid having to manually replicate autoconf-like behaviour every time we find a discrepancy. Of course, that would also mean you'd have to write more m4...
2021-09-16build: set OSX_MIN_VERSION to 10.15fanquake
This is required to use std::filesystem on macOS as support for it only landed in the libc++ dylib shipped with 10.15. See also: https://developer.apple.com/documentation/xcode-release-notes/xcode-11-release-notes Clang now supports the C++17 <filesystem> library for iOS 13, macOS 10.15, watchOS 6, and tvOS 13.
2021-08-31release: remove gitianfanquake
2021-08-09scripts: prevent GCC optimising test symbols in test-symbol-checkfanquake
2021-07-07scripts: adjust test-symbol-check for guix release environmentfanquake
Now that our release binaries are build in a glibc 2.24 and 2.27 environment, we can't use a symbol from glibc 2.28 to test our checks. Replace renameat2() with nextup(), which was introduced in 2.24. Note that this also means re-disabling the test for RISC-V, however RISC-V is built in a glibc 2.27 environment, and our minimum required glibc for that binary is 2.27.
2021-07-07scripts: more robustly test macOS symbol checksfanquake
2021-07-07build: Use and test PE binutils with --reloc-sectionCarl Dong
Also fix test-security-check.py to account for new PE PIE failure indication.
2021-07-07devtools: Improve *-check.py tool detectionCarl Dong
This is important to make sure that we're not testing tools different from the one we're building with. Introduce determine_wellknown_cmd, which encapsulates how we should handle well-known tools specification (IFS splitting, env override, etc.).
2021-07-06script: remove gitian reference from symbol-check.pyfanquake
2021-06-21Merge bitcoin/bitcoin#22244: devtools: Correctly extract symbol versions in ↵W. J. van der Laan
symbol-check e8cd3700eeb27437f5ea435869c9d61214285fdd devtools: Integrate ARCH_MIN_GLIBC_VER table into MAX_VERSIONS in symbol-check.py (W. J. van der Laan) a33381acf5ae2b43616fffaf26b1c8962e8ef0bb devtools: Add xkb version to symbol-check (W. J. van der Laan) 19e598bab0a1cb5ad93321eb9fa25d1a58d5e276 devtools: Fix verneed section parsing in pixie (W. J. van der Laan) Pull request description: I misunderstood the ELF specification for version symbols (verneed): The `vn_aux` pointer is relative to the main verneed record, not the start of the section. This caused many symbols to not be versioned properly in the return value of `elf.dyn_symbols`. This was discovered in #21454. Fix it by correcting the offset computation. - xkb versions symbols (using the prefix `V`), as this library is used by bitcoin-qt, add it to the valid versions in `symbol-check.py` This unfortunately brings to light some symbols that have been introduced since and weren't caught (from a gitian compile of master): ``` bitcoin-cli: symbol getrandom from unsupported version GLIBC_2.25 bitcoin-cli: failed IMPORTED_SYMBOLS bitcoind: symbol getrandom from unsupported version GLIBC_2.25 bitcoind: symbol log from unsupported version GLIBC_2.29 bitcoind: symbol fcntl64 from unsupported version GLIBC_2.28 bitcoind: symbol pow from unsupported version GLIBC_2.29 bitcoind: symbol exp from unsupported version GLIBC_2.29 bitcoind: failed IMPORTED_SYMBOLS bitcoin-qt: symbol exp from unsupported version GLIBC_2.29 bitcoin-qt: symbol fcntl64 from unsupported version GLIBC_2.28 bitcoin-qt: symbol log from unsupported version GLIBC_2.29 bitcoin-qt: symbol pow from unsupported version GLIBC_2.29 bitcoin-qt: symbol statx from unsupported version GLIBC_2.28 bitcoin-qt: symbol getrandom from unsupported version GLIBC_2.25 bitcoin-qt: symbol renameat2 from unsupported version GLIBC_2.28 bitcoin-qt: symbol getentropy from unsupported version GLIBC_2.25 bitcoin-qt: failed IMPORTED_SYMBOLS bitcoin-wallet: symbol exp from unsupported version GLIBC_2.29 bitcoin-wallet: symbol log from unsupported version GLIBC_2.29 bitcoin-wallet: symbol fcntl64 from unsupported version GLIBC_2.28 bitcoin-wallet: failed IMPORTED_SYMBOLS test_bitcoin: symbol getrandom from unsupported version GLIBC_2.25 test_bitcoin: symbol log from unsupported version GLIBC_2.29 test_bitcoin: symbol fcntl64 from unsupported version GLIBC_2.28 test_bitcoin: symbol pow from unsupported version GLIBC_2.29 test_bitcoin: symbol exp from unsupported version GLIBC_2.29 test_bitcoin: failed IMPORTED_SYMBOLS ``` ACKs for top commit: hebasto: ACK e8cd3700eeb27437f5ea435869c9d61214285fdd Tree-SHA512: 8c15e3478eb642f01a1ddaadef03f80583f088f9fa8e3bf171ce16b0ec05ffb4675ec147d7ffc6a4360637ed47fca517c6ca2bac7bb30d794c03783cfb964b79
2021-06-18devtools: Integrate ARCH_MIN_GLIBC_VER table into MAX_VERSIONS in ↵W. J. van der Laan
symbol-check.py The (ancient) versions specified here were deceptive. Entries older than MAX_VERSIONS['GLIBC'], which is 2.17, are ignored here. So reorganize the code to avoid confusion for other people reading this code.
2021-06-14devtools: Add xkb version to symbol-checkW. J. van der Laan
xkb versions symbols (using the prefix `V`), as this library is used by bitcoin-qt, add it to the valid versions in `symbol-check.py`.
2021-06-14devtools: Fix verneed section parsing in pixieW. J. van der Laan
I misunderstood the ELF specification for version symbols (verneed): The `vn_aux` pointer is relative to the main verneed record, not the start of the section. This caused many symbols to not be versioned properly in the return value of `elf.dyn_symbols`. This was discovered in #21454. Fix it by correcting the offset computation.
2021-06-10scripts: check macOS SDK version is setfanquake
Clangs Darwin driver should infer the SDK version used during compilation, and forward that through to the linker. Add a check that this has been done, and the expected SDK version is set. Should help prevent issues like #21771 in future.
2021-06-10scripts: check minimum required Windows version is setfanquake
We use linker flags (-Wl,--major/minor-subsystem-version) to set the minimum required version of Windows needed to run our binaries. This adds a sanity check that the version is being set as expected.
2021-06-10scripts: check minimum required macOS vesion is setfanquake
We use a compile flag (-mmacosx-version-min) to set the minimum required version of macOS needed to run our binaries. This adds a sanity check that the version is being set as expected.
2021-05-09scripts: test for MACHO control flow instrumentationfanquake
generated by cgit v1.2.3 (git 2.26.2) at 2024-11-01 22:59:57 +0000