aboutsummaryrefslogtreecommitdiff
path: root/contrib/devtools
AgeCommit message (Collapse)Author
2022-02-22contrib: address gen-manpages feedback from #24263fanquake
Co-authored-by: Carl Dong <contact@carldong.me>
2022-02-04doc: Fix gen-manpages, rewrite in Pythonlaanwj
Rewrite the manual page generation script in Python. This: - Solves '-' stripping issue (fixes #22681) - Makes that copyright footer is generated again
2022-01-26contrib: support arm64 darwin in security checksfanquake
2022-01-26build: use macOS 11 SDK (Xcode 12.2)fanquake
This should be sufficient to support building for Apple ARM when cross-compiling.
2022-01-25Merge bitcoin/bitcoin#23839: Linux: build with and test for control flow ↵laanwj
instrumentation on x86_64 5a8f907c93f158c54e58706d6d8c6b0a1a3205ba scripts: add CONTROL_FLOW to ELF security checks (fanquake) e13f8f775d9e8da04db0e072040d11237c5d1089 build: build x86_64 Linux Boost with -fcf-protection=full (fanquake) 6ca5efa8ed1ed9d5978bed0baf3c63dc2d406ec5 script rename control flow check to MACHO specific (fanquake) Pull request description: Closes #21888. TODO: * Duplication in security-check-tests Guix build: ```bash bash-5.1# find guix-build-$(git rev-parse --short=12 HEAD)/output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum 8025e2e6859247eaf865a4a5009d0a39570ec5e8ab51739fa4da0d8ee4ab2117 guix-build-5a8f907c93f1/output/aarch64-linux-gnu/SHA256SUMS.part 75576482994493262dbf7d3567b0450c0804bdc75c186d6b4c6a856bd0d8f0a3 guix-build-5a8f907c93f1/output/aarch64-linux-gnu/bitcoin-5a8f907c93f1-aarch64-linux-gnu-debug.tar.gz 7cb89c9327cc540859334b597c041a2250156e3e83ce2aa7d16177376056302f guix-build-5a8f907c93f1/output/aarch64-linux-gnu/bitcoin-5a8f907c93f1-aarch64-linux-gnu.tar.gz e264053e4c7a5d65340dd7bbd2c664bcd596292ab80f00a0bf0026bfe0c480bc guix-build-5a8f907c93f1/output/arm-linux-gnueabihf/SHA256SUMS.part 6c3ee9b3c0c4583620301b183226678f1618605dd07dbed0bbdf7d06d3591314 guix-build-5a8f907c93f1/output/arm-linux-gnueabihf/bitcoin-5a8f907c93f1-arm-linux-gnueabihf-debug.tar.gz f217cb8d8e233a8dbdbfde7dabf12c5d867d7de53e8c652b8ed1d4a55da82fd9 guix-build-5a8f907c93f1/output/arm-linux-gnueabihf/bitcoin-5a8f907c93f1-arm-linux-gnueabihf.tar.gz 48b4b2a1b52b3098f4e92c11cb60f1e5e9696a2a960560cd6adea72277eaa4a4 guix-build-5a8f907c93f1/output/dist-archive/bitcoin-5a8f907c93f1.tar.gz 40832db2446e129879caa9fbc9d682c53069876dbb2e0d4d76592e5dcb40bb12 guix-build-5a8f907c93f1/output/powerpc64-linux-gnu/SHA256SUMS.part 26fdfa9b7b77d5db415ef34054cf6f6d020a5dab73239db6dd05539f654bc5d5 guix-build-5a8f907c93f1/output/powerpc64-linux-gnu/bitcoin-5a8f907c93f1-powerpc64-linux-gnu-debug.tar.gz 3adbdb9d3eb1cb5f9adc38b29450054f286bd6d74cef8619adaee89494853605 guix-build-5a8f907c93f1/output/powerpc64-linux-gnu/bitcoin-5a8f907c93f1-powerpc64-linux-gnu.tar.gz 8ec2baf82483a698350bfdabf530cd9b5241690c916f597c746210e95ac451de guix-build-5a8f907c93f1/output/powerpc64le-linux-gnu/SHA256SUMS.part 1797e75e1f66ec9068fa4e57e0108960475e863f8f054fbe854358b1f995c4df guix-build-5a8f907c93f1/output/powerpc64le-linux-gnu/bitcoin-5a8f907c93f1-powerpc64le-linux-gnu-debug.tar.gz 448bd289ef26c777a1fc4498e7ba7fb17d0f6a932dcac91b2f89cbba63704bb8 guix-build-5a8f907c93f1/output/powerpc64le-linux-gnu/bitcoin-5a8f907c93f1-powerpc64le-linux-gnu.tar.gz a23ee91eeae515c2a6a31eb25d659fab833839aaafa4676ccee364bdad2a468c guix-build-5a8f907c93f1/output/riscv64-linux-gnu/SHA256SUMS.part 82df1b6d5020d0af8268ecc8e823f752f20dec308277763b2dd675804dfa4bbd guix-build-5a8f907c93f1/output/riscv64-linux-gnu/bitcoin-5a8f907c93f1-riscv64-linux-gnu-debug.tar.gz 09a6098ce83896a6ee6d5c8aff12eaca51595bd724c8e0b2a6f90b6410dc168c guix-build-5a8f907c93f1/output/riscv64-linux-gnu/bitcoin-5a8f907c93f1-riscv64-linux-gnu.tar.gz 4fac2951f80eaa2bd1747a263be6be1b76282cac5062f7d86db631a2fb80f8db guix-build-5a8f907c93f1/output/x86_64-apple-darwin/SHA256SUMS.part 3392f417b09efca5916c384f0b2d0c177a72ec4921399c62e84484f0054cc8c4 guix-build-5a8f907c93f1/output/x86_64-apple-darwin/bitcoin-5a8f907c93f1-osx-unsigned.dmg bfb4f8ade6107996ec4bc9efdb53959151b8fb19b6790c34472fe218dd02383d guix-build-5a8f907c93f1/output/x86_64-apple-darwin/bitcoin-5a8f907c93f1-osx-unsigned.tar.gz 4de92e149bad46fc863efb3b650753d194aad96be991d020e0b859452cf27457 guix-build-5a8f907c93f1/output/x86_64-apple-darwin/bitcoin-5a8f907c93f1-osx64.tar.gz 9329549a2f275a59e329afc8744ff8cbc297f3042e0ad03b47626805c7aae2e8 guix-build-5a8f907c93f1/output/x86_64-linux-gnu/SHA256SUMS.part d87deb1eca8a1d3780f02edde78350d27f700e13c7ff444be0bfb34e7369904a guix-build-5a8f907c93f1/output/x86_64-linux-gnu/bitcoin-5a8f907c93f1-x86_64-linux-gnu-debug.tar.gz 76f3bc2fad010d9373e854d941e8205b68fa6c9a8ecaff34c4978ae3ae76c806 guix-build-5a8f907c93f1/output/x86_64-linux-gnu/bitcoin-5a8f907c93f1-x86_64-linux-gnu.tar.gz 2948631081c4bd475529da2b7bc2b32b5aa2e44de46dfdaa6cf9432b3c9fe869 guix-build-5a8f907c93f1/output/x86_64-w64-mingw32/SHA256SUMS.part c5d56fac163b73c00e9745aa7b0f9a0bd9fcac7517e39b677869b6e76faf7218 guix-build-5a8f907c93f1/output/x86_64-w64-mingw32/bitcoin-5a8f907c93f1-win-unsigned.tar.gz f12c7cd72511544c67f4934981c090cee0d9c17e931c059edbbbef6e843dd651 guix-build-5a8f907c93f1/output/x86_64-w64-mingw32/bitcoin-5a8f907c93f1-win64-debug.zip eb952cc4554f92ba6787353f4305d7cbcb1c6dafc4a3867b3088106252a573b8 guix-build-5a8f907c93f1/output/x86_64-w64-mingw32/bitcoin-5a8f907c93f1-win64-setup-unsigned.exe 7bd2b071f1cdf9410535e6a10dd1da519f942bd2c1e47ed52c5b8c4f977e8f27 guix-build-5a8f907c93f1/output/x86_64-w64-mingw32/bitcoin-5a8f907c93f1-win64.zip ``` ACKs for top commit: laanwj: Code review ACK 5a8f907c93f158c54e58706d6d8c6b0a1a3205ba hebasto: ACK 5a8f907c93f158c54e58706d6d8c6b0a1a3205ba, I have reviewed the code and it looks OK, I agree it can be merged. Tree-SHA512: 75702ac01175ccf08e73b5c3bce266cee9dd15ccf8fb38d46108cfada156de9a8c18e27d996f9343ae33f7a75a6904d335cbe25256d90af1f4ccbe72ce1788bb
2022-01-05guix: use GCC 10 (over GCC 8) to build releasesfanquake
This currently points to the version-1.4.0 branch.
2022-01-04scripts: add CONTROL_FLOW to ELF security checksfanquake
2022-01-04script rename control flow check to MACHO specificfanquake
2022-01-03scripts: make security checks architecture independentfanquake
This paves the way for using and checking for architecture dependent flags like -fcf-protection on x86_64 Linux and -mbranch-protection on 64 bit ARM.
2021-12-30scripted-diff: Bump copyright headersHennadii Stepanov
-BEGIN VERIFY SCRIPT- ./contrib/devtools/copyright_header.py update ./ -END VERIFY SCRIPT- Commits of previous years: * 2020: fa0074e2d82928016a43ca408717154a1c70a4db * 2019: aaaaad6ac95b402fe18d019d67897ced6b316ee0
2021-12-30script: Fix copyright_header.pyHennadii Stepanov
This change prevents updating copyright years from "2021" to "2021-2017".
2021-12-18scripts: match on exe type over str in symbol-check.pyfanquake
2021-12-18scripts: match on exe type over str in security-check.pyfanquake
2021-12-03build: qt 5.15.2fanquake
2021-11-17scripts: test for PE control flow instrumentationfanquake
2021-11-16build: require glibc 2.18+ for release buildsfanquake
From what I can see the only platform this drops support for is CentOS 7. CentOS 7 reached the end of it's "full update" support at the end of 2020. It does receive maintenance updates until 2024, however I don't think supporting glibc 2.17 until 2024 is realistic. Note that anyone wanting to self-compile and target a glibc 2.17 runtime could build with --disable-threadlocal. glibc 2.18 was released in August 2013. https://sourceware.org/legacy-ml/libc-alpha/2013-08/msg00160.html
2021-11-13test: Enable SC2086 shellcheck ruleHennadii Stepanov
2021-10-21Add minisketch dependencyGleb Naumenko
2021-10-16lint: enable mypy checking for missing importsjosibake
Achieve this by adding some ignore, and making data/ importable. Co-authored-by: João Barbosa <joao.paulo.barbosa@gmail.com>
2021-10-13symbol-check: Check requested ELF interpreterCarl Dong
It is important that binaries request a standard interpreter location where most distros would place the linker-loader. Otherwise, the user would be met with a very confusing message: bash: <path>/<to>/bitcoind: No such file or directory When really it's the interpreter that's not found.
2021-10-12scripts: remove pixie.pyfanquake
2021-10-12scripts: only parse the binary once in security-check.pyfanquake
2021-10-12scripts: use LIEF for ELF checks in security-check.pyfanquake
2021-10-12scripts: only parse the binary once in symbol-check.pyfanquake
2021-10-12scripts: use LIEF for ELF checks in symbol-check.pyfanquake
Co-authored-by: Carl Dong <contact@carldong.me>
2021-10-12test-*-check: Pass in *FLAGS and compile with themCarl Dong
These test-*-check scripts should compile "test" binaries in a way that is as close to what autotools would do, since the goal is to make sure that if we run the *-check script, they can correctly detect flaws in binaries which are compiled by our autotools-based system. Therefore, we should emulate what happens when the binary is linked in autotools, meaning that for C binaries, we need to supply the CFLAGS, CPPFLAGS, and LDFLAGS flags in that order. Note to future developers: perhaps it'd be nice to have these test-*-check scripts be part of configure.ac to avoid having to manually replicate autoconf-like behaviour every time we find a discrepancy. Of course, that would also mean you'd have to write more m4...
2021-09-16build: set OSX_MIN_VERSION to 10.15fanquake
This is required to use std::filesystem on macOS as support for it only landed in the libc++ dylib shipped with 10.15. See also: https://developer.apple.com/documentation/xcode-release-notes/xcode-11-release-notes Clang now supports the C++17 <filesystem> library for iOS 13, macOS 10.15, watchOS 6, and tvOS 13.
2021-08-31release: remove gitianfanquake
2021-08-09scripts: prevent GCC optimising test symbols in test-symbol-checkfanquake
2021-07-07scripts: adjust test-symbol-check for guix release environmentfanquake
Now that our release binaries are build in a glibc 2.24 and 2.27 environment, we can't use a symbol from glibc 2.28 to test our checks. Replace renameat2() with nextup(), which was introduced in 2.24. Note that this also means re-disabling the test for RISC-V, however RISC-V is built in a glibc 2.27 environment, and our minimum required glibc for that binary is 2.27.
2021-07-07scripts: more robustly test macOS symbol checksfanquake
2021-07-07build: Use and test PE binutils with --reloc-sectionCarl Dong
Also fix test-security-check.py to account for new PE PIE failure indication.
2021-07-07devtools: Improve *-check.py tool detectionCarl Dong
This is important to make sure that we're not testing tools different from the one we're building with. Introduce determine_wellknown_cmd, which encapsulates how we should handle well-known tools specification (IFS splitting, env override, etc.).
2021-07-06script: remove gitian reference from symbol-check.pyfanquake
2021-06-21Merge bitcoin/bitcoin#22244: devtools: Correctly extract symbol versions in ↵W. J. van der Laan
symbol-check e8cd3700eeb27437f5ea435869c9d61214285fdd devtools: Integrate ARCH_MIN_GLIBC_VER table into MAX_VERSIONS in symbol-check.py (W. J. van der Laan) a33381acf5ae2b43616fffaf26b1c8962e8ef0bb devtools: Add xkb version to symbol-check (W. J. van der Laan) 19e598bab0a1cb5ad93321eb9fa25d1a58d5e276 devtools: Fix verneed section parsing in pixie (W. J. van der Laan) Pull request description: I misunderstood the ELF specification for version symbols (verneed): The `vn_aux` pointer is relative to the main verneed record, not the start of the section. This caused many symbols to not be versioned properly in the return value of `elf.dyn_symbols`. This was discovered in #21454. Fix it by correcting the offset computation. - xkb versions symbols (using the prefix `V`), as this library is used by bitcoin-qt, add it to the valid versions in `symbol-check.py` This unfortunately brings to light some symbols that have been introduced since and weren't caught (from a gitian compile of master): ``` bitcoin-cli: symbol getrandom from unsupported version GLIBC_2.25 bitcoin-cli: failed IMPORTED_SYMBOLS bitcoind: symbol getrandom from unsupported version GLIBC_2.25 bitcoind: symbol log from unsupported version GLIBC_2.29 bitcoind: symbol fcntl64 from unsupported version GLIBC_2.28 bitcoind: symbol pow from unsupported version GLIBC_2.29 bitcoind: symbol exp from unsupported version GLIBC_2.29 bitcoind: failed IMPORTED_SYMBOLS bitcoin-qt: symbol exp from unsupported version GLIBC_2.29 bitcoin-qt: symbol fcntl64 from unsupported version GLIBC_2.28 bitcoin-qt: symbol log from unsupported version GLIBC_2.29 bitcoin-qt: symbol pow from unsupported version GLIBC_2.29 bitcoin-qt: symbol statx from unsupported version GLIBC_2.28 bitcoin-qt: symbol getrandom from unsupported version GLIBC_2.25 bitcoin-qt: symbol renameat2 from unsupported version GLIBC_2.28 bitcoin-qt: symbol getentropy from unsupported version GLIBC_2.25 bitcoin-qt: failed IMPORTED_SYMBOLS bitcoin-wallet: symbol exp from unsupported version GLIBC_2.29 bitcoin-wallet: symbol log from unsupported version GLIBC_2.29 bitcoin-wallet: symbol fcntl64 from unsupported version GLIBC_2.28 bitcoin-wallet: failed IMPORTED_SYMBOLS test_bitcoin: symbol getrandom from unsupported version GLIBC_2.25 test_bitcoin: symbol log from unsupported version GLIBC_2.29 test_bitcoin: symbol fcntl64 from unsupported version GLIBC_2.28 test_bitcoin: symbol pow from unsupported version GLIBC_2.29 test_bitcoin: symbol exp from unsupported version GLIBC_2.29 test_bitcoin: failed IMPORTED_SYMBOLS ``` ACKs for top commit: hebasto: ACK e8cd3700eeb27437f5ea435869c9d61214285fdd Tree-SHA512: 8c15e3478eb642f01a1ddaadef03f80583f088f9fa8e3bf171ce16b0ec05ffb4675ec147d7ffc6a4360637ed47fca517c6ca2bac7bb30d794c03783cfb964b79
2021-06-18devtools: Integrate ARCH_MIN_GLIBC_VER table into MAX_VERSIONS in ↵W. J. van der Laan
symbol-check.py The (ancient) versions specified here were deceptive. Entries older than MAX_VERSIONS['GLIBC'], which is 2.17, are ignored here. So reorganize the code to avoid confusion for other people reading this code.
2021-06-14devtools: Add xkb version to symbol-checkW. J. van der Laan
xkb versions symbols (using the prefix `V`), as this library is used by bitcoin-qt, add it to the valid versions in `symbol-check.py`.
2021-06-14devtools: Fix verneed section parsing in pixieW. J. van der Laan
I misunderstood the ELF specification for version symbols (verneed): The `vn_aux` pointer is relative to the main verneed record, not the start of the section. This caused many symbols to not be versioned properly in the return value of `elf.dyn_symbols`. This was discovered in #21454. Fix it by correcting the offset computation.
2021-06-10scripts: check macOS SDK version is setfanquake
Clangs Darwin driver should infer the SDK version used during compilation, and forward that through to the linker. Add a check that this has been done, and the expected SDK version is set. Should help prevent issues like #21771 in future.
2021-06-10scripts: check minimum required Windows version is setfanquake
We use linker flags (-Wl,--major/minor-subsystem-version) to set the minimum required version of Windows needed to run our binaries. This adds a sanity check that the version is being set as expected.
2021-06-10scripts: check minimum required macOS vesion is setfanquake
We use a compile flag (-mmacosx-version-min) to set the minimum required version of macOS needed to run our binaries. This adds a sanity check that the version is being set as expected.
2021-05-09scripts: test for MACHO control flow instrumentationfanquake
2021-05-04contrib: consolidate PIE and NX security checksfanquake
2021-05-04contrib: use LIEF in PE symbol checksfanquake
2021-05-04contrib: use LIEF in macOS symbol checksfanquake
2021-05-04contrib: use f strings in symbol-check.pyfanquake
2021-05-04contrib: add PE PIE check to security checksfanquake
2021-05-04contrib: use LIEF for PE security checksfanquake
2021-05-04contrib: use LIEF for macOS security checksfanquake
2021-05-04contrib: use f strings in security-check.pyfanquake
generated by cgit v1.2.3 (git 2.26.2) at 2024-11-20 09:01:46 +0000