aboutsummaryrefslogtreecommitdiff
path: root/ci
AgeCommit message (Collapse)Author
2023-08-17ci: Refactor: Remove CI_USE_APT_INSTALLMarcoFalke
2023-08-16ci: Avoid error on macOS nativeMarcoFalke
This avoids "mkdir: /ci_container_base: Read-only file system"
2023-08-16ci: Fix macOS-cross SDK rsyncMarcoFalke
This should fix the macOS-cross build on Cirrus CI containers. Locally this was already working, because the SDK was cached in /ci_container_base/ in the image, which is also the folder used for a later CI run. However, on Cirrus CI, when using an image *and* a custom BASE_ROOT_DIR, the SDK will not be found in /ci_base_install/, nor in BASE_ROOT_DIR. Fix this by normalizing *all* folders to /ci_container_base/.
2023-08-15Merge bitcoin/bitcoin#28187: ci: Run "macOS native x86_64" job on GitHub ActionsAndrew Chow
9658d0dc17c270138523c41a982425e276b24271 ci: Run "macOS native x86_64" job on GitHub Actions (Hennadii Stepanov) Pull request description: From https://github.com/bitcoin/bitcoin/issues/28098: > Thus, someone would have to sponsor an amount of roughly 5kUSD/mo for those two tasks. > If the goal is to stay on a free plan, I think the only option is GitHub Actions CI. --- **IMPORTANT NOTE**. We currently ship macOS release binaries for both architectures: `x86_64` and `arm64`. If this PR gets merged, only `x86_64` architecture will be tested on CI, which implies some [drawbacks](https://github.com/bitcoin/bitcoin/pull/28187#issuecomment-1658077549). However, it has never been the case that our CI tested both architectures simultaneously. And we hope that GitHub Actions will soon host macOS `arm64` runners. Historically, we moved from `x86_64` to `arm64` in https://github.com/bitcoin/bitcoin/pull/26388 less than a year ago. --- Security concerns: - https://github.com/bitcoin/bitcoin/issues/28098#issuecomment-1651432106 - https://github.com/bitcoin/bitcoin/issues/28098#issuecomment-1651688197 `GITHUB_TOKEN` permissions (from the build log in my personal repo): ``` 2023-07-27T07:30:17.8313534Z ##[group]GITHUB_TOKEN Permissions 2023-07-27T07:30:17.8314113Z Contents: read 2023-07-27T07:30:17.8314608Z Metadata: read 2023-07-27T07:30:17.8314957Z Packages: read 2023-07-27T07:30:17.8315233Z ##[endgroup] ``` Comparison of resources: | Resource | Current, Cirrus CI | Suggested, GitHub Actions | |---|:-:|:-:| | CPU | 4 | 4 \*\* | | RAM, GB | 8 | 14 | **\*\* NOTE**: However, [docs](https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners#supported-runners-and-hardware-resources) are mentioning: > 3-core CPU (x86_64) ACKs for top commit: MarcoFalke: re-ACK 9658d0dc17c270138523c41a982425e276b24271 🏂 achow101: ACK 9658d0dc17c270138523c41a982425e276b24271 jarolrod: ACK 9658d0dc17c270138523c41a982425e276b24271 Tree-SHA512: 6123e68e6784cdf4e53c3e77b435709261db21f09091af2c22e667d3816a305fffb9d617297a5bc1bda18aaba84a6e210cec6a75c52afa7746a3780a67b69865
2023-08-09ci: Use hard-coded root path for CI containersMarcoFalke
2023-08-09ci: Only create folders when neededMarcoFalke
Now that container volumes are used, the folders are no longer mounted. They are only needed when running without a container engine (docker, podman).
2023-08-09ci: Drop BASE_SCRATCH_DIR from LIBCXX_DIRMarcoFalke
Using a hard-coded path avoids non-determinism issues and improves CI UX.
2023-08-09ci: Run "macOS native x86_64" job on GitHub ActionsHennadii Stepanov
Also, the "macOS native arm64" task has been removed from Cirrus CI.
2023-08-07ci: Use qemu-user through container engineMarcoFalke
2023-08-07Merge bitcoin/bitcoin#26296: ci: Integrate `bitcoin-tidy` clang-tidy pluginfanquake
1c976c691cc4b20f43071aabf36c7afed1571057 tidy: Integrate bicoin-tidy clang-tidy plugin (fanquake) 7de23cceb8ac13fcc709453ef0fa14fb93c460b0 refactor: fix unterminated LogPrintf()s (fanquake) 0a1029aa2946787bfe91a84de882c2dd0de076e9 lint: remove /* Continued */ markers from codebase (fanquake) 910007995d8603ffc466878856227153a638caff lint: remove lint-logs.py (fanquake) d86a83d6b8587b0971e66c6910af23dd8c042969 lint: drop DIR_IWYU global (fanquake) Pull request description: Demo of integrating the [bitcoin-tidy](https://github.com/theuni/bitcoin-tidy-plugin), [clang-tidy plugin](https://clang.llvm.org/extra/clang-tidy/) written by theuni into our tidy CI job. The plugin currently has a single check, `bitcoin-unterminated-logprintf`. This would replace our current Python driven, `git-grep`-based, `.cpp` file only, lint-logs linter. ACKs for top commit: TheCharlatan: ACK 1c976c691cc4b20f43071aabf36c7afed1571057 theuni: ACK 1c976c691cc4b20f43071aabf36c7afed1571057 MarcoFalke: re-ACK 1c976c691cc4b20f43071aabf36c7afed1571057 👠 Tree-SHA512: 725b45c70e431d48e6f276671e05c694e10b6047cae1a31906ac3ee9093bc8105fb226b36a5bac6709557526ca6007222112d66aecec05a574434edc4897e4b8
2023-08-03tidy: Integrate bicoin-tidy clang-tidy pluginfanquake
Enable `bitcoin-unterminated-logprintf`. Co-authored-by: Cory Fields <cory-nospam-@coryfields.com>
2023-08-03lint: drop DIR_IWYU globalfanquake
2023-08-01ci: Add missing linux-headers package to ASan taskMarcoFalke
Otherwise the task will throw in skip_if_no_python_bcc. Also, adjust CI_CONTAINER_CAP for all needed permissions.
2023-07-31script: update flake8 to 6.1.0Jon Atack
and touch up the spelling returned by lint-spelling.py
2023-07-30ci: Use documented `CCACHE_MAXSIZE` instead of `CCACHE_SIZE`Hennadii Stepanov
This change aims to: 1) Remove our own `CCACHE_SIZE` environment variable that violates Ccache's `CCACHE_*` namespace. 2) Introduce the `CCACHE_MAXSIZE` environment variable that is documented since v3.3, which makes its usage consistent with other ones, such as `CCACHE_DIR` and `CCACHE_NOHASHDIR`.
2023-07-28Merge bitcoin/bitcoin#28138: ci: Keep system env vars as-is (bugfix)fanquake
fabc04a4d96c4fe70e60d365aa28031d149094f3 ci: Keep system env vars as-is (MarcoFalke) fa8dcdcc8b29e58f5d285a49dde33d94b63c893b ci: Set PATH inside the CI env (MarcoFalke) fac229ab1f95ec77f18be8a783a2779dd781c684 ci: Remove P_CI_DIR and --workdir (MarcoFalke) Pull request description: This fixes a bug where the `$PATH` from the host is used inside the container. This will lead to bugs when the `$PATH` is different. For example on a host of Fedora 38, and a container of `debian:bullseye`. This can be tested with the `FILE_ENV=./ci/test/00_setup_env_arm.sh` CI env. On master: ``` Error: crun: executable file `bash` not found in $PATH: No such file or directory: OCI runtime attempted to invoke a command that was not found ``` On this pull: (everything passes) ACKs for top commit: TheCharlatan: lgtm ACK fabc04a4d96c4fe70e60d365aa28031d149094f3 Tree-SHA512: 51d2affed91624d0e5b43a3ee1e696313f934e05fde6b5a19e8ac4e6666c1e7b667a444bf3de3f77190bcd00e81efb7576196afb0f6b5e788d1a50e7ddb28d7e
2023-07-27Merge bitcoin/bitcoin#28092: ci: document that -Wreturn-type has been fixed ↵fanquake
upstream (mingw-w64) 08eb5f1b67e2af009549717eb5c66b7d7905731f ci: document that -Wreturn-type has been fixed upstream (Windows) (fanquake) Pull request description: `noreturn` attributes have been added to the mingw-w64 headers, https://github.com/mingw-w64/mingw-w64/commit/1690994f515910a31b9fb7c7bd3a52d4ba987abe, meaning that [from 11.0.0 onwards](https://www.mingw-w64.org/changelog/), you'll no-longer see `-Wreturn-type` warnings when using `assert(false)`. Add -Wno-return-type to the Windows CI, where is should have been all along, and document why it's required. This can be dropped when we are using the fixed version of the mingw-w64 headers there. Drop the -Werror -Wno-return-type special case from our build system. -Wreturn-type is on by default in Clang and GCC. The new mingw-w64 header behaviour can be checked on Ubuntu mantic, [which ships with 11.0.0](https://packages.ubuntu.com/mantic/mingw-w64), using: ```cpp #include <cassert> int f(){ assert(false); } int main() { return 0; } ``` On Mantic (with 11.0.0): ```bash x86_64-w64-mingw32-g++ test.cpp -Wreturn-type # nada ``` On Lunar ([with 10.0.0](https://packages.ubuntu.com/lunar/mingw-w64)): ```bash x86_64-w64-mingw32-g++ test.cpp -Wreturn-type test.cpp: In function 'int f()': test.cpp:3:25: warning: no return statement in function returning non-void [-Wreturn-type] 3 | int f(){ assert(false); } | ^ ``` ACKs for top commit: TheCharlatan: ACK 08eb5f1b67e2af009549717eb5c66b7d7905731f Tree-SHA512: 9cd4310a96abd87bf8ceb37949ad0259fe4adee3367c604f4c4ad521a0cf09bdcc5dd305db19a0f45ce74c85178b0d739e2fca5ad0fc841ac935523a23b28a7f
2023-07-24ci: Keep system env vars as-isMarcoFalke
2023-07-24ci: Set PATH inside the CI envMarcoFalke
This is needed for the next commit. This also requires dropping CI_RETRY from the docker build step, which is fine, because CI_RETRY should be called inside the build script, not outside. Also, fix a doc typo.
2023-07-24ci: Remove P_CI_DIR and --workdirMarcoFalke
The --workdir setting to the docker run command is not needed. And P_CI_DIR/PWD is equal to BASE_ROOT_DIR, so just use that directly.
2023-07-19lint: Add missing `set -ex` to ci/lint/06_script.shMarcoFalke
This is needed for the container-entrypoint.sh Also, remove unused `source` from ci/lint_run_all.sh, since it is the last step.
2023-07-19doc: Add doc comment to ci/test_imagefileMarcoFalke
(Similar to the doc comment in ci/lint_imagefile) Also, rename docker-entrypoint.sh to container-entrypoint.sh Also, add copyright header to touched files.
2023-07-18ci: document that -Wreturn-type has been fixed upstream (Windows)fanquake
`noreturn` attributes have been added to the mingw-w64 headers, meaning that from 11.0.0 onwards, you'll no-longer see `-Wreturn-type` warnings when using assert(false): https://github.com/mingw-w64/mingw-w64/commit/1690994f515910a31b9fb7c7bd3a52d4ba987abe. Add -Wno-return-type to the Windows CI, where is should have been all along, and document why it's required. This can be dropped when we are using the fixed version of the mingw-w64 headers there. Drop the -Werror -Wno-return-type special case from our build system. -Wreturn-type is on by default in Clang and GCC.
2023-07-16ci: Use DOCKER_BUILDKIT for lint imageMarcoFalke
Can be reviewed with: --color-moved=dimmed-zebra --ignore-all-space
2023-07-12ci: Add missing -O2 to valgrind tasksMarcoFalke
2023-07-06ci: Print full lscpu outputMarcoFalke
2023-06-30Merge bitcoin/bitcoin#28009: script, test: python typing and linter updatesfanquake
6c97757a480b6e71a0750330d69ff18ac7cc6127 script: appease spelling linter (Jon Atack) 1316119ce7ba3de4135bbf1e5ac28c9ea26f62e1 script: update ignored-words.txt (Jon Atack) 146c861da2e4236997bee3eed6110a5016a8b86b script: update linter dependencies (Jon Atack) 92408224a4cb2f454465d5ff8445c247f2c4318a test: fix PEP484 no implicit optional argument types errors (Jon Atack) f86a3014338de6a2204bbdda10795b75ef6654c0 script, test: add missing python type annotations (Jon Atack) Pull request description: With these updates, `./test/lint/lint-python.py` and `./test/lint/lint-spelling.py` should be green again for developers using relatively recent Python dependencies, in particular mypy 0.991 (released 11/2022) and later. Please see the commit messages for details. ACKs for top commit: fanquake: ACK 6c97757a480b6e71a0750330d69ff18ac7cc6127 Tree-SHA512: 8a46a4d36d5978affdcecf4f2ace20ca1b52d483e098304911a2169afe60ccb9b042fa90c04b762d94f3ce53d2cafe6f24476ae839867a770c7f31e7e7242d99
2023-06-30ci: re-enable gui tests for s390xfanquake
These work for me now. If they still don't work in other setups, maybe we can better document the issues. ```bash time FILE_ENV="./ci/test/00_setup_env_s390x.sh" ./ci/test_run_all.sh ... Running tests: coins_tests from test/coins_tests.cpp PASS: qt/test/test_bitcoin-qt Running tests: coinstatsindex_tests from test/coinstatsindex_tests.cpp ... Stop and remove CI container by ID + docker container kill 617bef8accb87530e5fbb03ff07b3b9f0aa9e3030d4da424c9612d153ab98dbf 617bef8accb87530e5fbb03ff07b3b9f0aa9e3030d4da424c9612d153ab98dbf real 51m37.809s ```
2023-06-29script: update linter dependenciesJon Atack
2023-06-29Merge bitcoin/bitcoin#27996: ci: filter all subtrees from tidy outputfanquake
62633b50461cb67dfb37d6485e604152e727559c ci: filter all subtrees from tidy output (fanquake) Pull request description: We are currently dumping output for some. i.e: ```bash diff --git a/src/minisketch/src/fields/clmul_1byte.cpp b/src/minisketch/src/fields/clmul_1byte.cpp index 8826af9..7fd6f2a 100644 --- a/src/minisketch/src/fields/clmul_1byte.cpp +++ b/src/minisketch/src/fields/clmul_1byte.cpp @@ -4,21 +4,16 @@ * file LICENSE or http://www.opensource.org/licenses/mit-license.php.* **********************************************************************/ -/* This file was substantially auto-generated by doc/gen_params.sage. */ -#include "../fielddefines.h" - +class Sketch; #if defined(ENABLE_FIELD_BYTES_INT_1) ``` ACKs for top commit: hebasto: re-ACK 62633b50461cb67dfb37d6485e604152e727559c Tree-SHA512: fd0a17af6b37fc7641547dab329c2d14ec784941c4d100db1e80d232aff39e45ad9c588982810a2cfc54b4fe820bfe0d50638b53209fec6774fd556b9b0ae180
2023-06-29Merge bitcoin/bitcoin#27932: test: Fuzz on macOSfanquake
fae7c50d201726f605938c3511dd9119efeea5ec test: Run fuzz tests on macOS (MarcoFalke) Pull request description: Any reason not to? ACKs for top commit: jamesob: Github ACK https://github.com/bitcoin/bitcoin/pull/27932/commits/fae7c50d201726f605938c3511dd9119efeea5ec dergoegge: utACK fae7c50d201726f605938c3511dd9119efeea5ec Tree-SHA512: e45122d73fafb17cea312258314b826cb0745e08daadd28465f687ec02d4c127d2f8cbe20179a9fff5712038850c02c968abb4838fa088b7555e28709317d3a3
2023-06-29ci: filter all subtrees from tidy outputfanquake
We are currently dumping output for some. i.e: ```bash diff --git a/src/minisketch/src/fields/clmul_1byte.cpp b/src/minisketch/src/fields/clmul_1byte.cpp index 8826af9..7fd6f2a 100644 --- a/src/minisketch/src/fields/clmul_1byte.cpp +++ b/src/minisketch/src/fields/clmul_1byte.cpp @@ -4,21 +4,16 @@ * file LICENSE or http://www.opensource.org/licenses/mit-license.php.* **********************************************************************/ -/* This file was substantially auto-generated by doc/gen_params.sage. */ -#include "../fielddefines.h" - +class Sketch; #if defined(ENABLE_FIELD_BYTES_INT_1) ```
2023-06-28ci: remove duplicate python3 from CI configsfanquake
2023-06-28ci: remove duplicate bsdmainutils from CI configsfanquake
2023-06-27Merge bitcoin/bitcoin#27896: Remove the syscall sandboxAndrew Chow
32e2ffc39374f61bb2435da507f285459985df9e Remove the syscall sandbox (fanquake) Pull request description: After initially being merged in #20487, it's no-longer clear that an internal syscall sandboxing mechanism is something that Bitcoin Core should have/maintain, especially when compared to better maintained/supported alterantives, i.e [firejail](https://github.com/netblue30/firejail). There is more related discussion in #24771. Note that given where it's used, the sandbox also gets dragged into the kernel. If it's removed, this should not require any sort of deprecation, as this was only ever an opt-in, experimental feature. Closes #24771. ACKs for top commit: davidgumberg: crACK https://github.com/bitcoin/bitcoin/pull/27896/commits/32e2ffc39374f61bb2435da507f285459985df9e achow101: ACK 32e2ffc39374f61bb2435da507f285459985df9e dergoegge: ACK 32e2ffc39374f61bb2435da507f285459985df9e Tree-SHA512: 8cf71c5623bb642cb515531d4a2545d806e503b9d57bfc15a996597632b06103d60d985fd7f843a3c1da6528bc38d0298d6b8bcf0be6f851795a8040d71faf16
2023-06-22test: Run fuzz tests on macOSMarcoFalke
Also, fix a few bugs: * Error: RPC command "enumeratesigners" not found in RPC_COMMANDS_SAFE_FOR_FUZZING or RPC_COMMANDS_NOT_SAFE_FOR_FUZZING. Please update test/fuzz/rpc.cpp. * in run_once: ...format(" ".join(result.args), ... TypeError: sequence item 2: expected str instance, PosixPath found
2023-06-22Merge bitcoin/bitcoin#27798: depends: modernize clang flags for Darwinfanquake
cbee1d70918b7c8e524c07f3da7049c3a1a2cbff depends: modernize clang flags (Cory Fields) 2a85857ce5cddd365353216960e2d5d76d6102b8 ci: disable false-positive warnings for now (Cory Fields) Pull request description: This is a cleaner and simpler alternative to #25098. Inspired by [this conversation](https://github.com/bitcoin/bitcoin/pull/27737#issuecomment-1562543301). The diff is large but the change itself is quite small. Fixes builds with llvm >= 11 in guix by working around the problem. As a bonus, this is much cleaner and more maintainable than what we had before. See the updated comment for more info. At a high level: rather than playing tricks and trying to work around clang's default includes, disable them and re-add what we want. ACKs for top commit: fanquake: ACK cbee1d70918b7c8e524c07f3da7049c3a1a2cbff - tested Guix and the depends cross-compile. Would like to move this along, to unblock #27676, which itself might be a blocker for #27897. Note that macOS might seem somewhat in flux for the moment, but once we finish the migration to LLVM Clang + LLD, things will be must simpler, and ultimately more maintainable. TheCharlatan: ACK cbee1d70918b7c8e524c07f3da7049c3a1a2cbff Tree-SHA512: 5a8300be528f550e15ab23d869e77df7a62201c6d40c0384795a9eecee38118a676e0b79b2b76c5e597597181443caada54a01b75a544dbcde76da1deba8e3a4
2023-06-21Merge bitcoin/bitcoin#27919: ci: Run fuzz target even if input folder is emptyfanquake
0000f552937ee787d25c8fd0af3278ea94889216 ci: Run fuzz target even if input folder is empty (MarcoFalke) Pull request description: This should catch trivial integer sanitizer bugs if the author and all reviewers forget to look for them. ACKs for top commit: brunoerg: reACK 0000f552937ee787d25c8fd0af3278ea94889216 dergoegge: reACK 0000f552937ee787d25c8fd0af3278ea94889216 Tree-SHA512: f139b9d56f0cf1aae339c2890721c77c88d1fea77b73d492c1386ec99b4f393c5b664029919ff4a22e4e8a2929f085699a148c6acc2cc3e40df8a72fd39ff474
2023-06-20ci: disable false-positive warnings for nowCory Fields
clang <=17 warns on -nostdlibinc, which causes an error on our -Werror builds. Note that this breaks the "-fPIE" check in configure because it relies on catching warnings, but that is not a problem for macOS.
2023-06-20ci: Run fuzz target even if input folder is emptyMarcoFalke
2023-06-20ci: install llvm-symbolizer in MSAN jobsfanquake
2023-06-20ci: use LLVM 16.0.6 in MSAN jobsfanquake
2023-06-16Remove the syscall sandboxfanquake
After initially being merged in #20487, it's no-longer clear that an internal syscall sandboxing mechanism is something that Bitcoin Core should have/maintain, especially when compared to better maintained/supported alterantives, i.e firejail. Note that given where it's used, the sandbox also gets dragged into the kernel. There is some related discussion in #24771. This should not require any sort of deprecation, as this was only ever an opt-in, experimental feature. Closes #24771.
2023-06-15build: suppress external warnings by defaultfanquake
2023-06-14ci: Bump macOS cross task to ubuntu:jammyMarcoFalke
2023-06-09ci: Use podman stop over podman killMarcoFalke
This should avoid a race where the kill is not done when spinning up the new container. podman stop waits 10 seconds by default.
2023-06-08ci: Invalidate Cirrus CI docker cacheMarcoFalke
2023-06-05ci: enable AArch64 target in MSAN jobsfanquake
Use Native.
2023-06-05ci: use LLVM 16.0.5 in MSAN jobsfanquake
2023-06-02Merge bitcoin/bitcoin#27737: ci: compile Clang and compiler-rt in msan jobsfanquake
5763b232e6e6a0f72d046f8aa322b39328be135b ci: return to using Ubuntu 22.04 in MSAN jobs (fanquake) d3cbcbf62693ad7394b3f8693b1c08d4271903fa ci: compile clang and compiler-rt in MSAN jobs (fanquake) 796bd1d0d147ac90f921ce3831961f97748d4e1a ci: use LLVM 16.0.4 in MSAN jobs (fanquake) 883bc9f5611648c44956a09795afd924842c1d1d ci: remove extra CC & CXX from MSAN jobs (fanquake) 2d4f4b8f29c015c26cb02b26a517450bb6056ed4 ci: standardize custom libc++ usage in MSAN jobs (fanquake) Pull request description: This reworks the MSAN CIs, to first compile Clang and compiler-rt (using GCC 12), and then, compile an MSAN instrumented libc++ using the just-built Clang 16. This fixes the `native_fuzz_with_msan` job, working around https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1005341, by not using the Debian provided Clang/LLVM. Also included are changes to streamline how we use our "custom libc++", according to upstream: https://releases.llvm.org/16.0.0/projects/libcxx/docs/UsingLibcxx.html#using-a-custom-built-libc, as well as other minor cleanups in the CI configs. An example job is currently running in the qa-assets repo: https://github.com/bitcoin-core/qa-assets/pull/129 (https://cirrus-ci.com/task/4632561431871488). ACKs for top commit: dergoegge: utACK 5763b232e6e6a0f72d046f8aa322b39328be135b Tree-SHA512: 4f2a6e0b796bb1830b8346dd1e55eaa86a79037b8b4f16a336c1e29f4fc460acca2ecba076635459370bcbb4009333cb79d27ef1521c1fb5db7599cd5bdf558c
generated by cgit v1.2.3 (git 2.26.2) at 2025-02-20 10:27:13 +0000