aboutsummaryrefslogtreecommitdiff
path: root/ci
AgeCommit message (Collapse)Author
2021-10-16lint: install pyzmq (22.3.0) into linter environmentjosibake
mypy stubs were introduced in 21.0.1
2021-10-15ci, refactor: Disable binaries for Android task explicitlyHennadii Stepanov
No behavior change.
2021-10-11Integrate univalue into our buildsystemCory Fields
This addresses issues like the one in #12467, where some of our compiler flags end up being dropped during the subconfigure of Univalue. Specifically, we're still using the compiler-default c++ version rather than forcing c++17. We can drop the need subconfigure completely in favor of a tighter build integration, where the sources are listed separately from the build recipes, so that they may be included directly by upstream projects. This is similar to the way leveldb build integration works in Core. Core benefits of this approach include: - Better caching (for ex. ccache and autoconf) - No need for a slow subconfigure - Faster autoconf - No more missing compile flags - Compile only the objects needed There are no benefits to Univalue itself that I can think of. These changes should be a no-op there, and to downstreams as well until they take advantage of the new sources.mk. This also removes the option to use an external univalue to avoid similar ABI issues with mystery binaries. Co-authored-by: fanquake <fanquake@gmail.com>
2021-10-04Merge bitcoin/bitcoin#20487: Add syscall sandboxing using seccomp-bpf (Linux ↵W. J. van der Laan
secure computing mode) 4747da3a5b639b5a336b737e7e3cbf060cf2efcf Add syscall sandboxing (seccomp-bpf) (practicalswift) Pull request description: Add experimental syscall sandboxing using seccomp-bpf (Linux secure computing mode). Enable filtering of system calls using seccomp-bpf: allow only explicitly allowlisted (expected) syscalls to be called. The syscall sandboxing implemented in this PR is an experimental feature currently available only under Linux x86-64. To enable the experimental syscall sandbox the `-sandbox=<mode>` option must be passed to `bitcoind`: ``` -sandbox=<mode> Use the experimental syscall sandbox in the specified mode (-sandbox=log-and-abort or -sandbox=abort). Allow only expected syscalls to be used by bitcoind. Note that this is an experimental new feature that may cause bitcoind to exit or crash unexpectedly: use with caution. In the "log-and-abort" mode the invocation of an unexpected syscall results in a debug handler being invoked which will log the incident and terminate the program (without executing the unexpected syscall). In the "abort" mode the invocation of an unexpected syscall results in the entire process being killed immediately by the kernel without executing the unexpected syscall. ``` The allowed syscalls are defined on a per thread basis. I've used this feature since summer 2020 and I find it to be a helpful testing/debugging addition which makes it much easier to reason about the actual capabilities required of each type of thread in Bitcoin Core. --- Quick start guide: ``` $ ./configure $ src/bitcoind -regtest -debug=util -sandbox=log-and-abort … 2021-06-09T12:34:56Z Experimental syscall sandbox enabled (-sandbox=log-and-abort): bitcoind will terminate if an unexpected (not allowlisted) syscall is invoked. … 2021-06-09T12:34:56Z Syscall filter installed for thread "addcon" 2021-06-09T12:34:56Z Syscall filter installed for thread "dnsseed" 2021-06-09T12:34:56Z Syscall filter installed for thread "net" 2021-06-09T12:34:56Z Syscall filter installed for thread "msghand" 2021-06-09T12:34:56Z Syscall filter installed for thread "opencon" 2021-06-09T12:34:56Z Syscall filter installed for thread "init" … # A simulated execve call to show the sandbox in action: 2021-06-09T12:34:56Z ERROR: The syscall "execve" (syscall number 59) is not allowed by the syscall sandbox in thread "msghand". Please report. … Aborted (core dumped) $ ``` --- [About seccomp and seccomp-bpf](https://en.wikipedia.org/wiki/Seccomp): > In computer security, seccomp (short for secure computing mode) is a facility in the Linux kernel. seccomp allows a process to make a one-way transition into a "secure" state where it cannot make any system calls except exit(), sigreturn(), and read() and write() to already-open file descriptors. Should it attempt any other system calls, the kernel will terminate the process with SIGKILL or SIGSYS. In this sense, it does not virtualize the system's resources but isolates the process from them entirely. > > […] > > seccomp-bpf is an extension to seccomp that allows filtering of system calls using a configurable policy implemented using Berkeley Packet Filter rules. It is used by OpenSSH and vsftpd as well as the Google Chrome/Chromium web browsers on Chrome OS and Linux. (In this regard seccomp-bpf achieves similar functionality, but with more flexibility and higher performance, to the older systrace—which seems to be no longer supported for Linux.) ACKs for top commit: laanwj: Code review and lightly tested ACK 4747da3a5b639b5a336b737e7e3cbf060cf2efcf Tree-SHA512: e1c28e323eb4409a46157b7cc0fc29a057ba58d1ee2de268962e2ade28ebd4421b5c2536c64a3af6e9bd3f54016600fec88d016adb49864b63edea51ad838e17
2021-10-01Add syscall sandboxing (seccomp-bpf)practicalswift
2021-09-22ci: update minimum compiler requirements for std::filesystemfanquake
2021-09-21Merge bitcoin/bitcoin#22993: build: set OSX_MIN_VERSION to 10.15fanquake
a43b8e955558483d8893996cc3a67bc74cbaf358 build: set OSX_MIN_VERSION to 10.15 (fanquake) Pull request description: Taken out of #20744, as splitting up some of the build changes was mentioned [here](https://github.com/bitcoin/bitcoin/pull/22937#discussion_r707303172). This is required to use `std::filesystem` on macOS, as support for it only landed in the libc++.dylib shipped with 10.15. So if we want to move to using `std::filesystem` for `23.0`, this bump is required. See also: https://developer.apple.com/documentation/xcode-release-notes/xcode-11-release-notes > Clang now supports the C++17 \<filesystem\> library for iOS 13, macOS 10.15, watchOS 6, and tvOS 13. macOS 10.15 was released in October 2019. macOS OS's seem to have a life of about 3 years, so it's possible that 10.14 will become officially unsupported by the end of 2021 and prior to the release of 23.0. Guix builds: ```bash bash-5.1# find guix-build-$(git rev-parse --short=12 HEAD)/output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum abc8b749be65f1339dcdf44bd1ed6ade2533b8e3b5030ad1dde0ae0cede78136 guix-build-a43b8e955558/output/dist-archive/bitcoin-a43b8e955558.tar.gz 1edcc301eb4c02f3baa379beb8d4c78e661abc24a293813bc9d900cf7255b790 guix-build-a43b8e955558/output/x86_64-apple-darwin19/SHA256SUMS.part e9dbb5594a664519da778dde9ed861c3f0f631525672e17a67eeda599f16ff44 guix-build-a43b8e955558/output/x86_64-apple-darwin19/bitcoin-a43b8e955558-osx-unsigned.dmg 11b23a17c630dddc7594c25625eea3de42db50f355733b9ce9ade2d8eba3a8f3 guix-build-a43b8e955558/output/x86_64-apple-darwin19/bitcoin-a43b8e955558-osx-unsigned.tar.gz 257ba64a327927f94d9aa0a68da3a2695cf880b3ed1a0113c5a966dcc426eb5e guix-build-a43b8e955558/output/x86_64-apple-darwin19/bitcoin-a43b8e955558-osx64.tar.gz ``` ACKs for top commit: hebasto: ACK a43b8e955558483d8893996cc3a67bc74cbaf358 jarolrod: ACK a43b8e9 Tree-SHA512: 9ac77be7cb56c068578860a3b2b8b7487c9e18b71b14aedd77a9c663f5d4bb19756d551770c02ddd12f1797beea5757b261588e7b67fb53509bb998ee8022369
2021-09-17ci: Update valgrind configMarcoFalke
* Set missing DOCKER_NAME_TAG * Update TEST_RUNNER_EXTRA
2021-09-17ci: Update s390x configMarcoFalke
* Bump to debian:bookworm to avoid crash in the zmq functional test (bitcoind: line 2: 33011 Illegal instruction (core dumped) qemu-s390x) * Remove RUN_UNIT_TESTS=true, because it is the default * Add TEST_RUNNER_EXTRA --exclude to skip failing tests
2021-09-16Merge bitcoin/bitcoin#22930: build: remove glibc back compatW. J. van der Laan
76630cbfb9247a78c7760aa8dca62c6b599a0c7c doc: add glibc to dependencies.md (fanquake) a907704e7f0f1989e70b9b073c25b740e9d5003d compat: remove glibc_compat.cpp (fanquake) 62e45da94537ffaecc51441896242ef5123106a5 build: remove glibc-back-compat from build system (fanquake) 2ef0accefc1653ea0845683db8256dddf3b55e04 remove --enable-glibc-back-compat from CI and docs (fanquake) Pull request description: This removes our glibc backwards compatibility code (glibcxx sanity checks remain), which is [no-longer used for release builds](https://github.com/bitcoin/bitcoin/pull/22405). With these changes our minimum required glibc remains at 2.17, as Guix builds [are done with `--disable-thread-local`](https://github.com/bitcoin/bitcoin/blob/d2dd1697cee9d6d0f13f9cb351bce84eaa4a72b4/contrib/guix/libexec/build.sh#L242). Guix builds: ```bash d9024376ac06b836800b942fcbe8a6454866ded652e41c43d605255f840e8183 guix-build-76630cbfb924/output/aarch64-linux-gnu/SHA256SUMS.part bea9607f111b7a501e2410e59ccfca6f83ea075edb299d2f9b878860a4f1ad50 guix-build-76630cbfb924/output/aarch64-linux-gnu/bitcoin-76630cbfb924-aarch64-linux-gnu-debug.tar.gz b41d7eaec5b598f146ead32a40c0e2281d22138c02a7000fb154fe8ff341ab9d guix-build-76630cbfb924/output/aarch64-linux-gnu/bitcoin-76630cbfb924-aarch64-linux-gnu.tar.gz 9ec0e1b2cd4ca55d5c12354325d7c4552333f1bad7e620db247eb24e15500210 guix-build-76630cbfb924/output/arm-linux-gnueabihf/SHA256SUMS.part dbb3e92a3b7d6460e0d5319f49c91eb65593d8604807c1b3084c9d657b198271 guix-build-76630cbfb924/output/arm-linux-gnueabihf/bitcoin-76630cbfb924-arm-linux-gnueabihf-debug.tar.gz ff2fa70b01f92b9fba9f0216deebc3e511d84ff93a7316f6766a405160a20483 guix-build-76630cbfb924/output/arm-linux-gnueabihf/bitcoin-76630cbfb924-arm-linux-gnueabihf.tar.gz 486586b4f3e81855a52e5410127b92f6dde9d1eede2720de260b13552a4b4823 guix-build-76630cbfb924/output/dist-archive/bitcoin-76630cbfb924.tar.gz 466ccc848c39c0ca85c4575c2bfcd1bc7aba2caa5ba8a42147cce60be5120bf6 guix-build-76630cbfb924/output/powerpc64-linux-gnu/SHA256SUMS.part 4c739da127df3738e993dcef48ec8f005b7d7938060197c718e771cdd18c2087 guix-build-76630cbfb924/output/powerpc64-linux-gnu/bitcoin-76630cbfb924-powerpc64-linux-gnu-debug.tar.gz 2294fc23c170d63fc03085d56ba8653e3cf0cff15c8bd5680faa7b7552e1db07 guix-build-76630cbfb924/output/powerpc64-linux-gnu/bitcoin-76630cbfb924-powerpc64-linux-gnu.tar.gz 7be025478c51a1ba7884a7c2f090a1311651fc218d706051b1830cbc4e82ee4d guix-build-76630cbfb924/output/powerpc64le-linux-gnu/SHA256SUMS.part 30600ad6d9bfe70b68a2a4bedc733b56b26232a45d4491b1a6fa30b76cd3f690 guix-build-76630cbfb924/output/powerpc64le-linux-gnu/bitcoin-76630cbfb924-powerpc64le-linux-gnu-debug.tar.gz 70528805fae16b95d1df46011511ca1c2616fb89422c0164e30d02fa0193b6a1 guix-build-76630cbfb924/output/powerpc64le-linux-gnu/bitcoin-76630cbfb924-powerpc64le-linux-gnu.tar.gz 20f4daaa095803e9c34ed8b119f948e3e61d03b4d5814e4b15ca285c2ba5a109 guix-build-76630cbfb924/output/riscv64-linux-gnu/SHA256SUMS.part 6f969d6b6561ea87d1e54a3a643640da54e4e33d2470dbc607ad27f3a87a0f1d guix-build-76630cbfb924/output/riscv64-linux-gnu/bitcoin-76630cbfb924-riscv64-linux-gnu-debug.tar.gz 7c1e925d199d21781c96e9ee8d63d5eac995cbe2574ccedd81c0938531f694c1 guix-build-76630cbfb924/output/riscv64-linux-gnu/bitcoin-76630cbfb924-riscv64-linux-gnu.tar.gz d0f13a7a9fcd95669cbab3b637940c2014632405037959989768ec2ffa7fd861 guix-build-76630cbfb924/output/x86_64-apple-darwin18/SHA256SUMS.part ff23fbdd72fd42140bf11773866061cd5571b66d65bbd769b1345969d195b7ce guix-build-76630cbfb924/output/x86_64-apple-darwin18/bitcoin-76630cbfb924-osx-unsigned.dmg 1b51f4a7611b85d4f599291f42887643ddd59babfc6fb71284793123d8a6be33 guix-build-76630cbfb924/output/x86_64-apple-darwin18/bitcoin-76630cbfb924-osx-unsigned.tar.gz e0dc9be66c3dc9ba30c5224cf0499ef02b3b7e213081561a6d57d5cc76b3e9a7 guix-build-76630cbfb924/output/x86_64-apple-darwin18/bitcoin-76630cbfb924-osx64.tar.gz ae3fb0320a0f39dae48354fe74f3d071b16b33920b3b7b27244658439fdfbc93 guix-build-76630cbfb924/output/x86_64-linux-gnu/SHA256SUMS.part 9545ccf26a930a3f876058bb0c4965e1320a354ee2d9ce27d95f8bdd2e8679a6 guix-build-76630cbfb924/output/x86_64-linux-gnu/bitcoin-76630cbfb924-x86_64-linux-gnu-debug.tar.gz 4827206220eaa2b36ebb44b68035c6bda0cec0c2f1b27c08b8349f2f7b3f56d5 guix-build-76630cbfb924/output/x86_64-linux-gnu/bitcoin-76630cbfb924-x86_64-linux-gnu.tar.gz c2ad3ed62b2ac41a25a43e1c38869a588bf93853cae4502880adf0b66ce30369 guix-build-76630cbfb924/output/x86_64-w64-mingw32/SHA256SUMS.part a88ee3a0cb715d32cf12cb164d8fe4d9c4c810cc417426f3aacc4e7f08460271 guix-build-76630cbfb924/output/x86_64-w64-mingw32/bitcoin-76630cbfb924-win-unsigned.tar.gz 7673ac8df641d185ea7a150ed27eeee9645168e126d186c6ae935eefdff9edae guix-build-76630cbfb924/output/x86_64-w64-mingw32/bitcoin-76630cbfb924-win64-debug.zip b90d243d292d1b603d744639e5061917035bd8fba6acd0bb61f10479a200f5aa guix-build-76630cbfb924/output/x86_64-w64-mingw32/bitcoin-76630cbfb924-win64-setup-unsigned.exe fe69d000da647e0fb7ab19252149be4b45af742223b4c37630200b1d5b43de33 guix-build-76630cbfb924/output/x86_64-w64-mingw32/bitcoin-76630cbfb924-win64.zip ``` ACKs for top commit: laanwj: Code review ACK 76630cbfb9247a78c7760aa8dca62c6b599a0c7c Tree-SHA512: 6bd1e344f0f37cfb0017fb8b2b0eee41c6a043f23f8ccb2ad1d59e7f2a47f4b84fe431d7d059c409b63263a0af38be955961c4b2ba7b03538f77a0597abb7880
2021-09-16build: set OSX_MIN_VERSION to 10.15fanquake
This is required to use std::filesystem on macOS as support for it only landed in the libc++ dylib shipped with 10.15. See also: https://developer.apple.com/documentation/xcode-release-notes/xcode-11-release-notes Clang now supports the C++17 <filesystem> library for iOS 13, macOS 10.15, watchOS 6, and tvOS 13.
2021-09-16ci: use Debian Bullseye in ARM CIfanquake
This works around an issue when trying to use `std::filesystem::remove_all` with the ARM GCC on Buster. Has been split out of #20744. See comments starting here: https://github.com/bitcoin/bitcoin/pull/20744#issuecomment-810279549. Also: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=93201.
2021-09-10remove --enable-glibc-back-compat from CI and docsfanquake
2021-09-08ci: Switch multiprocess to i686 buildMarcoFalke
2021-09-06Merge bitcoin/bitcoin#22841: ci: Fuzz with -ftrivial-auto-var-init=patternMarcoFalke
fa0a5fa744108d81bee9600c80bfda6ca11e5255 ci: Fuzz with -ftrivial-auto-var-init=pattern (MarcoFalke) Pull request description: This makes memory bugs deterministic. `-ftrivial-auto-var-init=pattern` is incompatible with other memory sanitizers (like valgrind and msan), but that is irrelevant here, because the address sanitizer in this fuzz CI config is already incompatible with them. `-ftrivial-auto-var-init=pattern` goes well with `-fsanitize=bool` and `-fsanitize=enum`, but those are already enabled via `-fsanitize=undefined`. See https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html#available-checks ACKs for top commit: practicalswift: cr ACK fa0a5fa744108d81bee9600c80bfda6ca11e5255 Tree-SHA512: ed6be953cd99eadb1ba245ba30170747eff66be54d2773c8d26a3a6aee0fdcd6967c596f4f4ab1d238de6a6526623dac5211f0ba77f1986639395d7921bdc19f
2021-09-02test: Update test README and lint scriptMarcoFalke
2021-09-02Merge bitcoin/bitcoin#22418: release: Remove gitianMarcoFalke
ab9c34237ab7b056394e0bd1f7cb131ffd95754c release: remove gitian (fanquake) Pull request description: Note that this doesn't yet touch any glibc back compat related code. ACKs for top commit: laanwj: Code review ACK ab9c34237ab7b056394e0bd1f7cb131ffd95754c Tree-SHA512: 8e2fe3ec1097f54bb11ab9136b43818d90eab5dbb0a663ad6a552966ada4bdb49cc12ff4e66f0ec0ec5400bda5c81f3a3ce70a9ebb6fe1e0db612da9f00a51a7
2021-08-31Merge bitcoin/bitcoin#22744: ci: Re-enable verify-commits.py checkfanquake
fa001602cd5ac61b9258e998ee2b236688c19ef7 ci: Re-enable verify-commits.py check (MarcoFalke) fa880b10d67542b8eb476a0e1f3ffb67e88d5e53 ci: Unconditionally set the global git author name in cirrys.yml (MarcoFalke) Pull request description: Might be useful to detect bugs in the script itself or an accidentally missed signature. ACKs for top commit: josibake: ACK https://github.com/bitcoin/bitcoin/pull/22744/commits/fa001602cd5ac61b9258e998ee2b236688c19ef7 Zero-1729: tACK fa001602cd5ac61b9258e998ee2b236688c19ef7 fanquake: untested ACK fa001602cd5ac61b9258e998ee2b236688c19ef7 Tree-SHA512: 8a13a67d325f2477f4088d1034f0d5e4e04937a01ee3c738435fe66394c02b9f33225529952ad331b0ba19b63ca4b2f26911cb5d264890159840cf3e09085969
2021-08-31ci: Fuzz with -ftrivial-auto-var-init=patternMarcoFalke
2021-08-31release: remove gitianfanquake
2021-08-27Merge bitcoin/bitcoin#20586: Fix Windows build with --enable-werrorW. J. van der Laan
b367745cfe19f6de3f44b3adc90fa08e36e44bb6 ci: Make Cirrus CI Windows build with --enable-werror (Hennadii Stepanov) c713bb2b243881a771ab288340ffeb623c82d7f6 Fix Windows build with --enable-werror on Ubuntu Focal (Hennadii Stepanov) Pull request description: This PR makes possible to cross-compile Windows build with `--enable-werror --enable-suppress-external-warnings`. Some problems are fixed, others are silenced. Also `--enable-werror` is enabled for Cirrus CI Windows build (the last one on Cirrus CI without `--enable-werror`). ACKs for top commit: practicalswift: cr ACK b367745cfe19f6de3f44b3adc90fa08e36e44bb6: patch looks correct laanwj: Code review ACK b367745cfe19f6de3f44b3adc90fa08e36e44bb6 vasild: ACK b367745cfe19f6de3f44b3adc90fa08e36e44bb6 jarolrod: ACK b367745cfe19f6de3f44b3adc90fa08e36e44bb6 Tree-SHA512: 64f5c99b7dad4c0efce80cd45d7074f275bd8411235dc9e0841287bdab64b812c6f8f9d632c35531d0b8210148531f53aaaac77be7699b29d2d6aaae304dbee0
2021-08-26ci: Re-enable verify-commits.py checkMarcoFalke
2021-08-17contrib: use hkps://keys.openpgp.org to retrieve builder keysfanquake
hkps://hkps.pool.sks-keyservers.net is essentially no-longer functional, and a number of distributions and GPG tools have since switched to using this key server as their default. See this Debian patch for additional context: https://salsa.debian.org/debian/gnupg2/-/blob/debian/main/debian/patches/Use-hkps-keys.openpgp.org-as-the-default-keyserver.patch Switch to using keys.openpgp.org in the CI as well.
2021-07-18ci: Bump Android NDK to r22 which supports std::filesystemHennadii Stepanov
2021-07-14Merge bitcoin/bitcoin#20354: test: Add feature_taproot.py --previous_releaseMarcoFalke
fa80e10d94dbf86da84fc761b09fb631155a5b25 test: Add feature_taproot.py --previous_release (MarcoFalke) 85ccffa26686c6c9adbd18bdde37fc1747281bab test: move releases download incantation to README (Sjors Provoost) 29d6b1da2a862bfbb14e7821979c97416c5400e8 test: previous releases: add v0.20.1 (Sjors Provoost) Pull request description: Disabling the new consensus code at runtime is fine, but potentially fragile and incomplete. Fix that by giving the option to run with a version that has been compiled without any taproot code. ACKs for top commit: Sjors: tACK fa80e10 NelsonGaldeman: tACK fa80e10d94dbf86da84fc761b09fb631155a5b25 Tree-SHA512: 1a1feef823f08c05268759645a8974e1b2d39a024258f5e6acecbe25097aae3fa9302c27262978b40f1aa8e7b525b60c0047199010f2a5d6017dd6434b4066f0
2021-07-07ci: skip running the Linux test-security-check target for nowfanquake
The CI environment is a moving target, and these tests are somewhat fragile, so for now, disable them.
2021-07-02ci: use Ubuntu 20.04 as the default Docker containerfanquake
All but 2 of the Ubuntu CIs are already using 20.04 or 21.04.
2021-06-30Do not clone qa-assets git repository if not necessaryKiminuo
2021-06-28ci: Upgrading pip version in macos environmentTushar Singla
During each CI run, for macos native environment, python packages lief and zmq are rebuilt everytime which wastes a lot of resources and time. The latest version of pip directly fetches pre-built binaries. Through this commit pip version is upgraded in macos environment before installation of these packages.
2021-06-16build: enable external signer by defaultSjors Provoost
2021-05-10Merge bitcoin/bitcoin#21749: test: Bump shellcheck versionW. J. van der Laan
08f3dbb1b0cd5ca01d87e488a2fa905adf7df057 test: Bump shellcheck version (Hennadii Stepanov) Pull request description: The changelog for v0.7.2 is available [here](https://github.com/koalaman/shellcheck/blob/v0.7.2/CHANGELOG.md). Only [SC2268](https://github.com/koalaman/shellcheck/wiki/SC2268) requires to update our code. ACKs for top commit: jarolrod: ACK 08f3dbb1b0cd5ca01d87e488a2fa905adf7df057 Tree-SHA512: 4585cd1f4d9def2fbaafe5a2a57761288d432781eb8c6c6d37064727d7ca8fc3f35c552e6a2ffdf0820d753d4bde2c8e43e5f3f57d242f5f57591a9b1b03558d
2021-05-05ci: Properly pass msan cflagsMarcoFalke
2021-05-05fix permissions on 00_setup_env_native_fuzz_with_msanglozow
2021-05-05Merge bitcoin/bitcoin#21740: test: add new python linter to check file names ↵W. J. van der Laan
and permissions 46b025e00df40724175735eb5606ac73067cb3b8 test: add new python linter to check file names and permissions (windsok) 6f6bb3ebc7cb8e17a5dfc8ef55aa2d3f2dc6bdea test: fix file permissions on various scripts (windsok) Pull request description: Adds a new python linter test which tests for correct filenames and file permissions in the repository. Replaces the existing tests in the `test/lint/lint-filenames.sh` and `test/lint/lint-shebang.sh` linter tests, as well as adding some new and increased testing. This increased coverage is intended to catch issues such as in #21728 and https://github.com/bitcoin/bitcoin/pull/16807/files#r345547050 Summary of tests: * Checks every file in the repository against an allowed regexp to make sure only lowercase or uppercase alphanumerics (a-zA-Z0-9), underscores (_), hyphens (-), at (@) and dots (.) are used in repository filenames. * Checks only source files (*.cpp, *.h, *.py, *.sh) against a stricter allowed regexp to make sure only lowercase alphanumerics (a-z0-9), underscores (_), hyphens (-) and dots (.) are used in source code filenames. Additionally there is an exception regexp for directories or files which are excepted from matching this regexp (This should replicate the existing `test/lint/lint-filenames.sh` test) * Checks all files in the repository match an allowed executable or non-executable file permission octal. Additionally checks that for executable files, the file contains a shebang line. * Checks that for executable `.py` and `.sh` files, the shebang line used matches an allowable list of shebangs (This should replicate the existing `test/lint/lint-shebang.sh` test) * Checks every file that contains a shebang line to ensure it has an executable permission Additionally updates the permissions on various files to comply with the new tests. Fixes #21729 ACKs for top commit: practicalswift: cr re-ACK 46b025e00df40724175735eb5606ac73067cb3b8: patch still looks correct kiminuo: code review ACK 46b025e00df40724175735eb5606ac73067cb3b8 if `contrib/gitian-descriptors/assign_DISTNAME` permission change is deemed OK. laanwj: Code review ACK 46b025e00df40724175735eb5606ac73067cb3b8 Tree-SHA512: 1c8201a2cee0d9cbce15652b68cec9a6458a8b493fcd5392f98560aca0b1a12e668baab65a47100f116f626dadc3f591deb47f7368468c6a46c6c712c2533455
2021-05-05Merge bitcoin/bitcoin#21664: contrib: use LIEF for macOS and Windows symbol ↵W. J. van der Laan
& security checks 7fc5e865b93af59364e9c8bf75ec68b4decc7e5d test: install lief in CI (fanquake) 955140b3265d3bcb9504c61d73fbfdadfff8a2b2 contrib: consolidate PIE and NX security checks (fanquake) 2aa1631822b2fdbc6cf7a3dcd99adaf4d2745ed4 contrib: use LIEF in PE symbol checks (fanquake) e93ac26b8563576345c13e83c777dd39e7616b1e contrib: use LIEF in macOS symbol checks (fanquake) a632cbcee5ae982f50aba625713b7686aef29168 contrib: use f strings in symbol-check.py (fanquake) 0f5d77c8e4db691733edb455dd9e31dabe933b8d contrib: add PE PIE check to security checks (fanquake) 8e1f40dd9a5135dbdec2c25961fbd0729a42254c contrib: use LIEF for PE security checks (fanquake) a25b2e965c93fe2a46a2f8f1e7bdf5642d453511 contrib: use LIEF for macOS security checks (fanquake) 7e7eae7aa86ab95c44eed601f8c993285a256bbc contrib: use f strings in security-check.py (fanquake) 2e7a9f7ade0c7b31e762c0ddb9e0944a0d9c798e guix: install LIEF in Guix container (fanquake) 465967b5ef4b4f02e9d6783a94eca012d4ebcdab gitian: install LIEF in gitian container (fanquake) Pull request description: This PR is a proof of concept for using [LIEF](https://github.com/lief-project/LIEF) for the PE and MACHO symbol and security checks. It replaces our current approach of manually parsing the output of `objdump` & `otool`. If the consensus is that using LIEF is ok, then I also plan on replacing [pixie.py](https://github.com/bitcoin/bitcoin/blob/master/contrib/devtools/pixie.py), and using LIEF for all checks. LIEF for Linux is also currently blocked (on the next release, unless we want to build master) on one change for RISC-V that I [sent upstream](https://github.com/lief-project/LIEF/pull/562). LIEF is seemingly well maintained, and is the basis for a number of other tools. It also has some very nice documentation; i.e the [Python API for ELF](https://lief.quarkslab.com/doc/latest/api/python/elf.html). It also has many builtins we can take advantage of. i.e [`is_pie`](https://lief.quarkslab.com/doc/latest/api/python/macho.html#lief.MachO.Binary.is_pie), [`has_nx`](https://lief.quarkslab.com/doc/latest/api/python/macho.html#lief.MachO.Binary.has_nx) etc. This means we can [consolidate some of our checks](https://github.com/bitcoin/bitcoin/commit/9c5eeb54848f428109ec24dff55f189a5358e9bc). If/when end up using LIEF for lightning then we can consolidate further, and cleanup these scripts. i.e to not parse the binary inside the checks, but once at the start of the script. Guix builds: ```bash # find guix-build-$(git rev-parse --short=12 HEAD)/output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum 963a08638c46f9a3d75cd4b0c155d1ca091bbeba27167291adcd3dca03fd4c3d guix-build-f51237d94d98/output/aarch64-linux-gnu/bitcoin-f51237d94d98-aarch64-linux-gnu-debug.tar.gz a3ce927c46b103789a010c41a6ebfafe4548d90ee7d88f2a735c9183b775da5c guix-build-f51237d94d98/output/aarch64-linux-gnu/bitcoin-f51237d94d98-aarch64-linux-gnu.tar.gz 2503ac8901068805d5e7251fd5cfeb7c1f8ba3528bdfcf3aa1e0c40bfd5c1cbc guix-build-f51237d94d98/output/arm-linux-gnueabihf/bitcoin-f51237d94d98-arm-linux-gnueabihf-debug.tar.gz 5798697e58e1788df85aa9e2e4d33fef0456169fcbd2521f13b3b5806ac0d84d guix-build-f51237d94d98/output/arm-linux-gnueabihf/bitcoin-f51237d94d98-arm-linux-gnueabihf.tar.gz 4185adebc6a0abe7241a3cd409a6ab7be031c26f1c4245e30bb5f87eef0925d2 guix-build-f51237d94d98/output/dist-archive/bitcoin-f51237d94d98.tar.gz 9b4b8756c5c84295eb6b61b6b32a07a8d07723fb38aaa8f519b6133935061bda guix-build-f51237d94d98/output/powerpc64-linux-gnu/bitcoin-f51237d94d98-powerpc64-linux-gnu-debug.tar.gz cbd821aa464a9c16f7979dbec1a5e66939e777a567f55f7081499a8d528d42c5 guix-build-f51237d94d98/output/powerpc64-linux-gnu/bitcoin-f51237d94d98-powerpc64-linux-gnu.tar.gz abed530a82e97e3cf621c90a13c0881b0e39ccce2a6f42a3ff80de76e2abc5f7 guix-build-f51237d94d98/output/powerpc64le-linux-gnu/bitcoin-f51237d94d98-powerpc64le-linux-gnu-debug.tar.gz 8b6d2bdd8b58ff1f6072bf8693abe3ce773ff3a7d8d2b7218207e69945b9d31b guix-build-f51237d94d98/output/powerpc64le-linux-gnu/bitcoin-f51237d94d98-powerpc64le-linux-gnu.tar.gz d99cc705032d22ae819975992216899ed960ba25871a05c8789d00b80418511f guix-build-f51237d94d98/output/riscv64-linux-gnu/bitcoin-f51237d94d98-riscv64-linux-gnu-debug.tar.gz 5240ca4f4ef7c62088185224ac319ad9a4a9b40075df10af18d8a6355bca32fb guix-build-f51237d94d98/output/riscv64-linux-gnu/bitcoin-f51237d94d98-riscv64-linux-gnu.tar.gz adc16eaee4b51e8615ce8b3be9f6c018698237df4ad6e0886cf0d4ab6bc9e5c4 guix-build-f51237d94d98/output/x86_64-apple-darwin18/bitcoin-f51237d94d98-osx-unsigned.dmg b188af0572ee682d74cc82c7e6e464115205fc130a457cfe19d42ac9ddd267f8 guix-build-f51237d94d98/output/x86_64-apple-darwin18/bitcoin-f51237d94d98-osx-unsigned.tar.gz e764062fde144e6fb5d6dd776c10fc2daa8d775831f7e43247d17a6c6e060c97 guix-build-f51237d94d98/output/x86_64-apple-darwin18/bitcoin-f51237d94d98-osx64.tar.gz dab3d26ac94c669140f7329d14e57ef02b0fe92b8a8f9d96c32a416adea0da0f guix-build-f51237d94d98/output/x86_64-linux-gnu/bitcoin-f51237d94d98-x86_64-linux-gnu-debug.tar.gz ca59d4379fbe2b9a52deebeaf88508e0eda4215f28d319aff0781289dd159712 guix-build-f51237d94d98/output/x86_64-linux-gnu/bitcoin-f51237d94d98-x86_64-linux-gnu.tar.gz 52b7c35321a85c4f6c95bf0e687574454b71ede9bec1c9cf17f37c578c888a94 guix-build-f51237d94d98/output/x86_64-w64-mingw32/bitcoin-f51237d94d98-win-unsigned.tar.gz a543895a00f8ffb3ba50ca68396d52ad5a18dd8efe38730e0049dd70d283a092 guix-build-f51237d94d98/output/x86_64-w64-mingw32/bitcoin-f51237d94d98-win64-debug.zip aec050d03c65268a986148500f7341cceb8c5f85287e0e3cde8933ce4b4dee32 guix-build-f51237d94d98/output/x86_64-w64-mingw32/bitcoin-f51237d94d98-win64-setup-unsigned.exe 57ba33ed6ee8d3a885e342471359301473e83037d5442895beb686921a4c50e9 guix-build-f51237d94d98/output/x86_64-w64-mingw32/bitcoin-f51237d94d98-win64.zip ``` Gitian builds: ```bash # macOS: 2f066e852bdd30ac46e5ecdf7619d19d408035c318a3edf0f1893ec2e25efb69 bitcoin-41a1b3d1b130-osx-unsigned.dmg 8cf8ac4d21740f490262453c330b5f4a5c5b8139dfc1b322efefce3f3b93d1b2 bitcoin-41a1b3d1b130-osx-unsigned.tar.gz cf1b84efdd9d2588a1ce9513580fb56b38bfafe60e18f8adbeedf03521c6c2b2 bitcoin-41a1b3d1b130-osx64.tar.gz 14995244b0bb3e80e7b79975c9c70fdfb3ee3c04fda3efd5358ce1c4efa3a312 src/bitcoin-41a1b3d1b130.tar.gz 93881069d5e1dc385c08895a7b035a94eb010325afc2776c99b6aafa21096eb8 bitcoin-core-osx-22-res.yml # Windows: 4d56dd7713121684b7eaa448679c65df2fd0aa5319bf8d12fb6cfa9f0b005cf7 bitcoin-f51237d94d98-win-unsigned.tar.gz 4558f4173152b084bcba25aa1a53c605208a70fe20392141b63cefb476528c85 bitcoin-f51237d94d98-win64-debug.zip b63feaca010e86d514cfe38d716e3c8a8b8058e4f969b868aaaeb8a8a3d3dc81 bitcoin-f51237d94d98-win64-setup-unsigned.exe de7d8586cc91ba391fe911853a99d9fd15fc6f9a60f9b91a0447940173aac67a bitcoin-f51237d94d98-win64.zip 4185adebc6a0abe7241a3cd409a6ab7be031c26f1c4245e30bb5f87eef0925d2 src/bitcoin-f51237d94d98.tar.gz 45efaca35b5fad0a04dfd06e44f7c00b990aa91c7bf2faea57e020d3491a6cf0 bitcoin-core-win-22-res.yml # Linux: 055d646c5f8cf4708008374546176012ff758566a2645a3a01e1a33eab1002fe bitcoin-f51237d94d98-aarch64-linux-gnu-debug.tar.gz bfc8b0efc36b0474c88546b12d2723c04b4dc629ae311082025c7e0b8f0d1aa9 bitcoin-f51237d94d98-aarch64-linux-gnu.tar.gz 9dfaa5acfffadad8942b32996458013a155d12ed07be76601f232233627b5cb9 bitcoin-f51237d94d98-arm-linux-gnueabihf-debug.tar.gz 54eb57905ff8513b9f628707b61aa4659c362fb2f6d17e0ee240b4da3674907d bitcoin-f51237d94d98-arm-linux-gnueabihf.tar.gz ad98d876616eff578ad8cfd17dfbabe48ed14200823579687d66694bae3d2fe3 bitcoin-f51237d94d98-powerpc64-linux-gnu-debug.tar.gz fe1b421dd1cb6e04d5dc5d341459dc15fa6e15b80906e5d8e0405cf43495e0f7 bitcoin-f51237d94d98-powerpc64-linux-gnu.tar.gz 9001d95cc7d2722d9d7dd83d9da8e5adf575fddf91b615b76b9bcfece30ecf6f bitcoin-f51237d94d98-powerpc64le-linux-gnu-debug.tar.gz 9e0650ad2aba70c0fd1608a077e95f335dc1bb4a79eab9b0b56ac87427a4fd4f bitcoin-f51237d94d98-powerpc64le-linux-gnu.tar.gz fbfde0134944d3dbd32991455b0a8abdd334853ab8a4c1a1a4c060d9de071c50 bitcoin-f51237d94d98-riscv64-linux-gnu-debug.tar.gz 2fa2cfddce98c44c65305326fc623a7f065129208337503d813a08d51580cb8a bitcoin-f51237d94d98-riscv64-linux-gnu.tar.gz b2d6caeee0e3c350a43165c39876ebed8e588958007af0d06996e341c7060683 bitcoin-f51237d94d98-x86_64-linux-gnu-debug.tar.gz bfdb827e75d43d61462513c9a843620b93c9160d9d246cad13278baaa07f64ea bitcoin-f51237d94d98-x86_64-linux-gnu.tar.gz 4185adebc6a0abe7241a3cd409a6ab7be031c26f1c4245e30bb5f87eef0925d2 src/bitcoin-f51237d94d98.tar.gz 34820a093916fa35b0fd98806a50092f46b20271af7422f43e2a4223ef6f9bb7 bitcoin-core-linux-22-res.yml ``` ACKs for top commit: laanwj: re-ACK 7fc5e865b93af59364e9c8bf75ec68b4decc7e5d Tree-SHA512: 0c30838413448ecfcf55e6273f607fdb01cb1acafa1d2762afad59360fca7d8efa78ec55064f50cba56cb2c9e98741e13665cba8e9b4b8e5b62b8a53f9bf8990
2021-05-04Merge bitcoin/bitcoin#21852: ci: Add msan fuzz configMarcoFalke
fa0422c251ac7e23a01f4f504cdfcf7878657c1f ci: Add msan fuzz config (MarcoFalke) fa399a76c6f222200c144041c144da88eebfec38 ci: Use clang-12 in msan task (MarcoFalke) fab30174af7548ead6d858225e74024925d3445f ci: Set BASE_SCRATCH_DIR early, so that it can be used in test configs (MarcoFalke) Pull request description: Similar to the valgrind config, this config is not run by any ci task in this repo, but it can be used by other repos or self-hosted infrastructure. ACKs for top commit: practicalswift: cr ACK fa0422c251ac7e23a01f4f504cdfcf7878657c1f: patch looks correct Tree-SHA512: 2122ac0948978a7b952efc80d4aa3674b27d48c6166e0ce917c61ac4ee6b68d701a83e5f71ee6868c208885ee45aae409ca022ebcb23ccbe37819a8c36e34872
2021-05-04ci: Add msan fuzz configMarcoFalke
2021-05-04ci: Use clang-12 in msan taskMarcoFalke
2021-05-04ci: Set BASE_SCRATCH_DIR early, so that it can be used in test configsMarcoFalke
Can be reviewed with --color-moved=dimmed-zebra
2021-05-04test: install lief in CIfanquake
2021-05-04Merge bitcoin/bitcoin#21812: ci: Enable D_GLIBCXX_DEBUG for multiprocess taskfanquake
fa44f5119a0b412f0d46cad02f638727d140b451 ci: Clarify that previous_releases task is using DEBUG (MarcoFalke) fad0f21c3caba129106799fe6c14aff323ef99f2 ci: Use clang in multiprocess task to avoid OOM (MarcoFalke) faeabef4f386009847a0f91041d44e6f31eec618 ci: Enable D_GLIBCXX_DEBUG for multiprocess task (MarcoFalke) Pull request description: Enable `-D_GLIBCXX_DEBUG` via the depends `DEBUG` flag. Also `--enable-debug` to get debug symbols in traces. ACKs for top commit: hebasto: ACK fa44f5119a0b412f0d46cad02f638727d140b451, I have reviewed the code and it looks OK, I agree it can be merged, and CI is green. Tree-SHA512: ab2a216bb44ee462f9dd181ec9025962502bd4201a1118ff52b6a193398e7ea3ca465a45a5eb341e308758fc3ef34ea3521f8a1f85ed64478ef3c1f6c1b8b141
2021-05-01ci: Use clang in multiprocess task to avoid OOMMarcoFalke
2021-05-01build: Xcode 12.1, macOS SDK 10.15.6fanquake
2021-04-30ci: Enable D_GLIBCXX_DEBUG for multiprocess taskMarcoFalke
2021-04-23test: fix file permissions on various scriptswindsok
Updates permissions on files to comply with the new test added in the following commit
2021-04-22test: Bump shellcheck versionHennadii Stepanov
2021-04-20ci: Use clang-12 for asan taskMarcoFalke
Changes from Ubuntu Focal to Hirsute to get clang-12. Generally, it can be expected that more recent compilers have better support for sanitizers.
2021-04-13test: Remove spurious double lock tsan suppressions by bumping to clang-12MarcoFalke
2021-04-13ci: Make Cirrus CI Windows build with --enable-werrorHennadii Stepanov
2021-04-13Merge #21467: Move external signer out of wallet modulefanquake
88d4d5ff2f5c71a9a2f4c78c2b2e2fd00568cfee rpc: add help for enumeratesigners and walletdisplayaddress (Sjors Provoost) b0db187e5b30a491c9f95685430a82a1e35e921d ci: use --enable-external-signer instead of --with-boost-process (Sjors Provoost) b54b2e7b1a171203404bd41853372c73f2c64532 Move external signer out of wallet module (Sjors Provoost) Pull request description: In addition, this PR enables external signer testing on CI. This PR moves the ExternalSigner class and RPC methods out of the wallet module. The `enumeratesigners` RPC can be used without a wallet since #21417. With additional modifications external signers could be used without a wallet in general, e.g. via `signrawtransaction`. The `signerdisplayaddress` RPC is ranamed to `walletdisplayaddress` because it requires wallet context. A future `displayaddress` RPC call without wallet context could take a descriptor argument. This commit fixes a `rpc_help.py` failure when configured with `--disable-wallet`. ACKs for top commit: ryanofsky: Code review ACK 88d4d5ff2f5c71a9a2f4c78c2b2e2fd00568cfee fanquake: ACK 88d4d5ff2f5c71a9a2f4c78c2b2e2fd00568cfee Tree-SHA512: 3242a24e22313aed97eee32a520bfcb1c17495ba32a2b8e06a5e151e2611320e2da5ef35b572d84623af0a49a210d2f9377a2531250868d1a0ccf3e144352a97
generated by cgit v1.2.3 (git 2.26.2) at 2024-07-20 11:56:36 +0000