aboutsummaryrefslogtreecommitdiff
path: root/ci
AgeCommit message (Collapse)Author
2023-10-05ci: Avoid cache depends/buildMarcoFalke
2023-10-03ci: Print Linux kernel infoMarcoFalke
2023-09-20Merge bitcoin/bitcoin#28504: ci: Use nproc over MAKEJOBS in 01_base_installfanquake
fa3b5e5e57cd4649d577e06a3aca798b55a75fd5 ci: Use nproc over MAKEJOBS in 01_base_install (MarcoFalke) Pull request description: Currently `$MAKEJOBS` is the default value in `01_base_install.sh` when building the container image. This problem can't be fixed (see below), so just use `nproc` for now. Other solutions would be bad: * Passing in the `MAKEJOBS` as a dockerfile env would create a new image if the number of tasks are changed, seems verbose and confusing. * Leaving `master` as-is would leave CPUs unused if there are more than `4`. ACKs for top commit: fanquake: ACK - I think this is fine as-is, it's an improvement over the current state fa3b5e5e57cd4649d577e06a3aca798b55a75fd5 Tree-SHA512: 15014eb7b548945737b0fed5cd46f0cd4b72b1d54d044f60fce628f914053a2de6518878428a54822d236d08d6f257d8c464d3fb589400f4be56022d2ec8676d
2023-09-20Merge bitcoin/bitcoin#28432: build: Produce a `.zip` for macOS distributionfanquake
b5790c35f7e1d48c79b83bded36f3f72c18c9fc1 build: remove dmg dependencies (fanquake) 33ae0bd1e4756ca0f180ac4b3c32c9eb83b88cfd macdeploy: remove DMG generation from deploy script (fanquake) a128111c29ba0c31763ccbcd316268bfa9c029cd build: produce a .zip for macOS distribution (Hennadii Stepanov) c38561d6b1de954b712a92cb8a198ed42d73caea build: add -zip option to macdeployqtplus (fanquake) Pull request description: It is https://github.com/bitcoin/bitcoin/pull/27099 revived with addressed [comments](https://github.com/bitcoin/bitcoin/pull/27099#issuecomment-1708705686). From https://github.com/bitcoin/bitcoin/pull/27099#issue-1584429885: > Reviving the discussion around using a `.zip` for the distributed macOS binaries, as opposed to a `.dmg`. > > Given we only had a single report of the "no finder window" issue (#26176), I wonder if that means macOS users were able to figure it out, they gave up/didn't report, or, we just have very few macOS users. > > Related to #18128. That's how it looks on macOS: ![image](https://github.com/bitcoin/bitcoin/assets/32963518/baa637bb-256b-4b24-8645-8c2754c2ae64) ACKs for top commit: Sjors: tACK b5790c35f7e1d48c79b83bded36f3f72c18c9fc1 jarolrod: ACK b5790c35f7e1d48c79b83bded36f3f72c18c9fc1 TheCharlatan: utACK b5790c35f7e1d48c79b83bded36f3f72c18c9fc1 Tree-SHA512: 6e9cb3ab0f60f8a92bfec50577e8d096c5b23ec09ebbb334826415609140ddc96d470aea37379495c1c6bb1beec0d306b09460f62e1543bb0f4396c10a1dfbe2
2023-09-19ci: Use nproc over MAKEJOBS in 01_base_installMarcoFalke
2023-09-15build: remove dmg dependenciesfanquake
2023-09-15build: produce a .zip for macOS distributionHennadii Stepanov
Instead of a .dmg. Co-authored-by: fanquake <fanquake@gmail.com>
2023-09-15Merge bitcoin/bitcoin#28476: ci: LLVM 17 for MSAN jobsfanquake
a241d6069cf0542acdd8ec6be63724da19f10720 ci: use LLVM 17.0.0 in MSAN jobs (fanquake) Pull request description: See https://libcxx.llvm.org/Hardening.html as well as https://discourse.llvm.org/t/rfc-removing-the-legacy-debug-mode-from-libc/71026. ACKs for top commit: MarcoFalke: review ACK a241d6069cf0542acdd8ec6be63724da19f10720 Tree-SHA512: c374dabf307fe762be0da96f63695a150f6018c1468fe9414fad23f74f5818bbf7a5a699e109084e31467482a900cfebf1d5835821e4da94aa310b2c9570749c
2023-09-14Merge bitcoin/bitcoin#27850: test: Add unit & functional test coverage for ↵Andrew Chow
blockstore de8f9123afbecc3b4f59fa80af8148bc865d0588 test: cover read-only blockstore (Matthew Zipkin) 5c2185b3b624ce87320ec16412f98ab591a5860c ci: enable chattr +i capability inside containers (Matthew Zipkin) e573f2420244c583e218f51cd0d3a3cac6731003 unit test: add coverage for BlockManager (Matthew Zipkin) Pull request description: This PR adds unit and functional tests to cover the behavior described in #2039. In particular, that bitcoind will crash on startup if a reindex is requested but the `blk` files are read-only. Eventually this behavior can be updated with https://github.com/bitcoin/bitcoin/pull/27039. This PR just commits the test coverage from #27039 as suggested in https://github.com/bitcoin/bitcoin/pull/27039#issuecomment-1584915782 ACKs for top commit: jonatack: ACK de8f9123afbecc3b4f59fa80af8148bc865d0588 modulo suggestions in https://github.com/bitcoin/bitcoin/pull/27850#discussion_r1319010039, tested on macOS, but not on Linux for the Linux-related change in the last push achow101: ACK de8f9123afbecc3b4f59fa80af8148bc865d0588 MarcoFalke: lgtm ACK de8f9123afbecc3b4f59fa80af8148bc865d0588 📶 Tree-SHA512: b9bd684035dcea11c901b649fc39f397a2155a9a8459f3348e67947e387e45312fddeccb52981aef486f8a31deebb5356a7901c1bb94b78f82c24192a369af73
2023-09-14ci: Add filesystem lint checkMarcoFalke
2023-09-14ci: use LLVM 17.0.0 in MSAN jobsfanquake
See https://libcxx.llvm.org/Hardening.html as well as https://discourse.llvm.org/t/rfc-removing-the-legacy-debug-mode-from-libc/71026.
2023-09-13ci: clang-17 for fuzz and tsanMarcoFalke
2023-09-06ci: enable chattr +i capability inside containersMatthew Zipkin
2023-09-05Merge bitcoin/bitcoin#28359: ci: Asan with -ftrivial-auto-var-init=patternfanquake
fa07ac48d804beac38a98d23be2167f78cadefae ci: Asan with -ftrivial-auto-var-init=pattern (MarcoFalke) Pull request description: This makes memory bugs deterministic. `-ftrivial-auto-var-init=pattern` is incompatible with other memory sanitizers (like valgrind and msan), but that is irrelevant here, because the address sanitizer in this fuzz CI config is already incompatible with them. `-ftrivial-auto-var-init=pattern` goes well with `-fsanitize=bool` and `-fsanitize=enum`, but those are already enabled via `-fsanitize=undefined`. See https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html#available-checks ACKs for top commit: fanquake: ACK fa07ac48d804beac38a98d23be2167f78cadefae - going to get back to fixing up the cxxflags usage in CI, but not a blocker here: Tree-SHA512: 2ea6c5128a9cd262bdd1b1475c7e1be23ce2c520fad05f6c2aace6c29e203573323c0564d272e25da35ad5ff46fde488a5eae1ed14d3349e793d14a5e2533fb1
2023-08-30Merge bitcoin/bitcoin#28353: ci: Remove /ro_base bind mount, Remove LC_ALL=C ↵fanquake
from s390x task fa70cbd9693b0ccc56114647f0817c96b2f223ee ci: Remove unused TEST_RUNNER_ENV="LC_ALL=C" from s390x task (MarcoFalke) fa33354dcb6f289965e84a0cd4492c43dafd917b ci: Remove /ro_base bind mount (MarcoFalke) fa0df9d4c4cb93b0971f92f3fe6f9a1deb5b311f doc: Remove sudo from command that is already run as root (MarcoFalke) Pull request description: Remove some CI stuff no longer needed. ACKs for top commit: fanquake: ACK fa70cbd9693b0ccc56114647f0817c96b2f223ee - did not test the s390x job. Tree-SHA512: 3a6ed0cfc855a92c2f834e59494c0a19a5647510247aece5e40a1aa78074894fe7454e684a1ea1f8f0662c50ac1caf2e390398b0fcfbf81544e6488fa9b8915e
2023-08-30ci: Asan with -ftrivial-auto-var-init=patternMarcoFalke
2023-08-29Bump python minimum version to 3.9MarcoFalke
2023-08-29ci: Bump i686_multiprocess.sh to latest Ubuntu LTSMarcoFalke
There is no need to have it stuck on the previous one. This is needed for the next commit.
2023-08-28Merge bitcoin/bitcoin#28173: ci: Run Windows native task on GitHub Actionsfanquake
6a4337298035683a8748ae446eae90e5c5f41d1b ci: Run "Win64 native" job on GitHub Actions (Hennadii Stepanov) Pull request description: From https://github.com/bitcoin/bitcoin/issues/28098: > Thus, someone would have to sponsor an amount of roughly 5kUSD/mo for those two tasks. > If the goal is to stay on a free plan, I think the only option is GitHub Actions CI. Historical context: - https://github.com/bitcoin/bitcoin/pull/17697 - https://github.com/bitcoin/bitcoin/issues/17803 - https://github.com/bitcoin/bitcoin/pull/18031 Security concerns: - https://github.com/bitcoin/bitcoin/issues/28098#issuecomment-1651432106 - https://github.com/bitcoin/bitcoin/issues/28098#issuecomment-1651688197 `GITHUB_TOKEN` permissions (from the build log in my personal repo): ``` 2023-07-27T07:30:17.8313534Z ##[group]GITHUB_TOKEN Permissions 2023-07-27T07:30:17.8314113Z Contents: read 2023-07-27T07:30:17.8314608Z Metadata: read 2023-07-27T07:30:17.8314957Z Packages: read 2023-07-27T07:30:17.8315233Z ##[endgroup] ``` Comparison of resources: | Resource | Current, Cirrus CI | Suggested, GitHub Actions | |---|:-:|:-:| | CPU | 6 | 2 | | RAM, GB | 12 | 7 | The `TEST_RUNNER_TIMEOUT_FACTOR` variable is set to the current default value for all CI tasks: https://github.com/bitcoin/bitcoin/blob/64440bb733896a7a2caf902825e0406cb993e666/ci/test/00_setup_env.sh#L48 Top commit has no ACKs. Tree-SHA512: ddfdaf7a1e4793a64ac0cd20f116b29608dd06f15b062769ac70b3ea2fb82775aa96aa79c7b768efefec4338aaa5b57d267b592f62d0e8d5d94ecc11001a165d
2023-08-27ci: Remove unused TEST_RUNNER_ENV="LC_ALL=C" from s390x taskMarcoFalke
2023-08-27ci: Remove /ro_base bind mountMarcoFalke
Just set the bind mount to BASE_READ_ONLY_DIR, which allows to drop one line of code and makes the code easier to understand.
2023-08-27depends: use LLVM/Clang 15.0.6 for macOS cross-compilefanquake
There is no x86_64 binaries for 15.0.7.
2023-08-25ci: Run "Win64 native" job on GitHub ActionsHennadii Stepanov
2023-08-24Merge bitcoin/bitcoin#28330: ci: Add missing docker.io prefix for native ↵fanquake
tasks to CI_IMAGE_NAME_TAG fab7f5c01d0b4be00bdd8922d3bf8bd1a27ba8fb ci: Add missing docker.io prefix to CI_IMAGE_NAME_TAG (MarcoFalke) Pull request description: Currently, the CI system may pick the wrong (non-native) architecture due to the missing prefix. For example, assuming the CI_IMAGE_NAME_TAG is `debian:bookworm` and the user has previously pulled an s390x image: ``` $ podman run --rm 'docker.io/s390x/debian:bookworm' dpkg --print-architecture exec /usr/bin/dpkg: exec format error ``` Now, `debian:bookworm` will refer to the same image: ``` $ podman run --rm 'debian:bookworm' dpkg --print-architecture exec /usr/bin/dpkg: exec format error ``` However, `docker.io/debian:bookworm` works fine: ``` $ podman run --rm 'docker.io/debian:bookworm' dpkg --print-architecture arm64 ACKs for top commit: hebasto: ACK fab7f5c01d0b4be00bdd8922d3bf8bd1a27ba8fb. Tree-SHA512: c423c5cd454a95fa3e67081411ca08d316b8c680a5bba49196c514b91df65d9cc46a47700cc00d9579327842615f98146d0ac50abb016616a9b17d042598dab6
2023-08-23ci: Add missing docker.io prefix to CI_IMAGE_NAME_TAGMarcoFalke
2023-08-23ci: Remove no longer applicable sectionMarcoFalke
This fails with: "Error: determining starting point for build: no FROM statement found"
2023-08-23ci: Start with clean envMarcoFalke
This should help to avoid non-determinism.
2023-08-23ci: Limit scope of some env varsMarcoFalke
No need to have a larger scope than needed. Can be reviewed with --color-moved=dimmed-zebra
2023-08-18Merge bitcoin/bitcoin#28295: ci: Add missing amd64 to win64-cross taskfanquake
fa56d17a4b868f42fa45bea0d1f0c78de75e7838 ci: Add missing amd64 to win64-cross task (MarcoFalke) Pull request description: Currently the task will fail if run on non-`x86_64`. Fix this by adding the missing `amd64`, similar to https://github.com/bitcoin/bitcoin/blob/7bf078f2b7d4a0339d053144b4fb35fe020dac25/ci/test/00_setup_env_i686_multiprocess.sh#L11 ACKs for top commit: hebasto: ACK fa56d17a4b868f42fa45bea0d1f0c78de75e7838 Tree-SHA512: faab1c5b945283b7e8d080bbcc8e9379c480cf6973506149ace5990cb4d04673f83f4bc36d08d5b4e9cb17a86fdbe23ac97ef4eab0e842616b367b8138229c58
2023-08-18ci: Add missing amd64 to win64-cross taskMarcoFalke
Also, do the same for android, which also fails.
2023-08-18ci: Add missing ${CI_RETRY_EXE} before curlMarcoFalke
2023-08-17ci: Refactor: Remove CI_USE_APT_INSTALLMarcoFalke
2023-08-16ci: Avoid error on macOS nativeMarcoFalke
This avoids "mkdir: /ci_container_base: Read-only file system"
2023-08-16ci: Fix macOS-cross SDK rsyncMarcoFalke
This should fix the macOS-cross build on Cirrus CI containers. Locally this was already working, because the SDK was cached in /ci_container_base/ in the image, which is also the folder used for a later CI run. However, on Cirrus CI, when using an image *and* a custom BASE_ROOT_DIR, the SDK will not be found in /ci_base_install/, nor in BASE_ROOT_DIR. Fix this by normalizing *all* folders to /ci_container_base/.
2023-08-15Merge bitcoin/bitcoin#28187: ci: Run "macOS native x86_64" job on GitHub ActionsAndrew Chow
9658d0dc17c270138523c41a982425e276b24271 ci: Run "macOS native x86_64" job on GitHub Actions (Hennadii Stepanov) Pull request description: From https://github.com/bitcoin/bitcoin/issues/28098: > Thus, someone would have to sponsor an amount of roughly 5kUSD/mo for those two tasks. > If the goal is to stay on a free plan, I think the only option is GitHub Actions CI. --- **IMPORTANT NOTE**. We currently ship macOS release binaries for both architectures: `x86_64` and `arm64`. If this PR gets merged, only `x86_64` architecture will be tested on CI, which implies some [drawbacks](https://github.com/bitcoin/bitcoin/pull/28187#issuecomment-1658077549). However, it has never been the case that our CI tested both architectures simultaneously. And we hope that GitHub Actions will soon host macOS `arm64` runners. Historically, we moved from `x86_64` to `arm64` in https://github.com/bitcoin/bitcoin/pull/26388 less than a year ago. --- Security concerns: - https://github.com/bitcoin/bitcoin/issues/28098#issuecomment-1651432106 - https://github.com/bitcoin/bitcoin/issues/28098#issuecomment-1651688197 `GITHUB_TOKEN` permissions (from the build log in my personal repo): ``` 2023-07-27T07:30:17.8313534Z ##[group]GITHUB_TOKEN Permissions 2023-07-27T07:30:17.8314113Z Contents: read 2023-07-27T07:30:17.8314608Z Metadata: read 2023-07-27T07:30:17.8314957Z Packages: read 2023-07-27T07:30:17.8315233Z ##[endgroup] ``` Comparison of resources: | Resource | Current, Cirrus CI | Suggested, GitHub Actions | |---|:-:|:-:| | CPU | 4 | 4 \*\* | | RAM, GB | 8 | 14 | **\*\* NOTE**: However, [docs](https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners#supported-runners-and-hardware-resources) are mentioning: > 3-core CPU (x86_64) ACKs for top commit: MarcoFalke: re-ACK 9658d0dc17c270138523c41a982425e276b24271 🏂 achow101: ACK 9658d0dc17c270138523c41a982425e276b24271 jarolrod: ACK 9658d0dc17c270138523c41a982425e276b24271 Tree-SHA512: 6123e68e6784cdf4e53c3e77b435709261db21f09091af2c22e667d3816a305fffb9d617297a5bc1bda18aaba84a6e210cec6a75c52afa7746a3780a67b69865
2023-08-09ci: Use hard-coded root path for CI containersMarcoFalke
2023-08-09ci: Only create folders when neededMarcoFalke
Now that container volumes are used, the folders are no longer mounted. They are only needed when running without a container engine (docker, podman).
2023-08-09ci: Drop BASE_SCRATCH_DIR from LIBCXX_DIRMarcoFalke
Using a hard-coded path avoids non-determinism issues and improves CI UX.
2023-08-09ci: Run "macOS native x86_64" job on GitHub ActionsHennadii Stepanov
Also, the "macOS native arm64" task has been removed from Cirrus CI.
2023-08-07ci: Use qemu-user through container engineMarcoFalke
2023-08-07Merge bitcoin/bitcoin#26296: ci: Integrate `bitcoin-tidy` clang-tidy pluginfanquake
1c976c691cc4b20f43071aabf36c7afed1571057 tidy: Integrate bicoin-tidy clang-tidy plugin (fanquake) 7de23cceb8ac13fcc709453ef0fa14fb93c460b0 refactor: fix unterminated LogPrintf()s (fanquake) 0a1029aa2946787bfe91a84de882c2dd0de076e9 lint: remove /* Continued */ markers from codebase (fanquake) 910007995d8603ffc466878856227153a638caff lint: remove lint-logs.py (fanquake) d86a83d6b8587b0971e66c6910af23dd8c042969 lint: drop DIR_IWYU global (fanquake) Pull request description: Demo of integrating the [bitcoin-tidy](https://github.com/theuni/bitcoin-tidy-plugin), [clang-tidy plugin](https://clang.llvm.org/extra/clang-tidy/) written by theuni into our tidy CI job. The plugin currently has a single check, `bitcoin-unterminated-logprintf`. This would replace our current Python driven, `git-grep`-based, `.cpp` file only, lint-logs linter. ACKs for top commit: TheCharlatan: ACK 1c976c691cc4b20f43071aabf36c7afed1571057 theuni: ACK 1c976c691cc4b20f43071aabf36c7afed1571057 MarcoFalke: re-ACK 1c976c691cc4b20f43071aabf36c7afed1571057 👠 Tree-SHA512: 725b45c70e431d48e6f276671e05c694e10b6047cae1a31906ac3ee9093bc8105fb226b36a5bac6709557526ca6007222112d66aecec05a574434edc4897e4b8
2023-08-03tidy: Integrate bicoin-tidy clang-tidy pluginfanquake
Enable `bitcoin-unterminated-logprintf`. Co-authored-by: Cory Fields <cory-nospam-@coryfields.com>
2023-08-03lint: drop DIR_IWYU globalfanquake
2023-08-01ci: Add missing linux-headers package to ASan taskMarcoFalke
Otherwise the task will throw in skip_if_no_python_bcc. Also, adjust CI_CONTAINER_CAP for all needed permissions.
2023-07-31script: update flake8 to 6.1.0Jon Atack
and touch up the spelling returned by lint-spelling.py
2023-07-30ci: Use documented `CCACHE_MAXSIZE` instead of `CCACHE_SIZE`Hennadii Stepanov
This change aims to: 1) Remove our own `CCACHE_SIZE` environment variable that violates Ccache's `CCACHE_*` namespace. 2) Introduce the `CCACHE_MAXSIZE` environment variable that is documented since v3.3, which makes its usage consistent with other ones, such as `CCACHE_DIR` and `CCACHE_NOHASHDIR`.
2023-07-28Merge bitcoin/bitcoin#28138: ci: Keep system env vars as-is (bugfix)fanquake
fabc04a4d96c4fe70e60d365aa28031d149094f3 ci: Keep system env vars as-is (MarcoFalke) fa8dcdcc8b29e58f5d285a49dde33d94b63c893b ci: Set PATH inside the CI env (MarcoFalke) fac229ab1f95ec77f18be8a783a2779dd781c684 ci: Remove P_CI_DIR and --workdir (MarcoFalke) Pull request description: This fixes a bug where the `$PATH` from the host is used inside the container. This will lead to bugs when the `$PATH` is different. For example on a host of Fedora 38, and a container of `debian:bullseye`. This can be tested with the `FILE_ENV=./ci/test/00_setup_env_arm.sh` CI env. On master: ``` Error: crun: executable file `bash` not found in $PATH: No such file or directory: OCI runtime attempted to invoke a command that was not found ``` On this pull: (everything passes) ACKs for top commit: TheCharlatan: lgtm ACK fabc04a4d96c4fe70e60d365aa28031d149094f3 Tree-SHA512: 51d2affed91624d0e5b43a3ee1e696313f934e05fde6b5a19e8ac4e6666c1e7b667a444bf3de3f77190bcd00e81efb7576196afb0f6b5e788d1a50e7ddb28d7e
2023-07-27Merge bitcoin/bitcoin#28092: ci: document that -Wreturn-type has been fixed ↵fanquake
upstream (mingw-w64) 08eb5f1b67e2af009549717eb5c66b7d7905731f ci: document that -Wreturn-type has been fixed upstream (Windows) (fanquake) Pull request description: `noreturn` attributes have been added to the mingw-w64 headers, https://github.com/mingw-w64/mingw-w64/commit/1690994f515910a31b9fb7c7bd3a52d4ba987abe, meaning that [from 11.0.0 onwards](https://www.mingw-w64.org/changelog/), you'll no-longer see `-Wreturn-type` warnings when using `assert(false)`. Add -Wno-return-type to the Windows CI, where is should have been all along, and document why it's required. This can be dropped when we are using the fixed version of the mingw-w64 headers there. Drop the -Werror -Wno-return-type special case from our build system. -Wreturn-type is on by default in Clang and GCC. The new mingw-w64 header behaviour can be checked on Ubuntu mantic, [which ships with 11.0.0](https://packages.ubuntu.com/mantic/mingw-w64), using: ```cpp #include <cassert> int f(){ assert(false); } int main() { return 0; } ``` On Mantic (with 11.0.0): ```bash x86_64-w64-mingw32-g++ test.cpp -Wreturn-type # nada ``` On Lunar ([with 10.0.0](https://packages.ubuntu.com/lunar/mingw-w64)): ```bash x86_64-w64-mingw32-g++ test.cpp -Wreturn-type test.cpp: In function 'int f()': test.cpp:3:25: warning: no return statement in function returning non-void [-Wreturn-type] 3 | int f(){ assert(false); } | ^ ``` ACKs for top commit: TheCharlatan: ACK 08eb5f1b67e2af009549717eb5c66b7d7905731f Tree-SHA512: 9cd4310a96abd87bf8ceb37949ad0259fe4adee3367c604f4c4ad521a0cf09bdcc5dd305db19a0f45ce74c85178b0d739e2fca5ad0fc841ac935523a23b28a7f
2023-07-24ci: Keep system env vars as-isMarcoFalke
2023-07-24ci: Set PATH inside the CI envMarcoFalke
This is needed for the next commit. This also requires dropping CI_RETRY from the docker build step, which is fine, because CI_RETRY should be called inside the build script, not outside. Also, fix a doc typo.
generated by cgit v1.2.3 (git 2.26.2) at 2024-10-13 01:17:23 +0000