Age | Commit message (Collapse) | Author |
|
|
|
6889a807661cb570e1464ffdedb9ddb1ec970078 ci: Add missing CI_RETRY_EXE before git clone (MarcoFalke)
b705bade44973e61655d5f847f49d97fb5bb8393 ci: Export `IN_GETOPT_BIN` on macOS (Hennadii Stepanov)
1c2132ddd97dbf8fc908b0d1113bbb70e1a6a64e Revert "ci: Upgrading pip version in macos environment" (Hennadii Stepanov)
Pull request description:
This PR is a resurrection of https://github.com/bitcoin/bitcoin/pull/28623:
> This should fix [bitcoin/bitcoin/actions/runs/6457002476/job/17527598426#step:7:240](https://github.com/bitcoin/bitcoin/actions/runs/6457002476/job/17527598426#step:7:240):
>
> ```
> + git clone --depth=1 https://github.com/bitcoin-core/qa-assets /Users/runner/work/bitcoin/bitcoin/ci/scratch/qa-assets
> Cloning into '/Users/runner/work/bitcoin/bitcoin/ci/scratch/qa-assets'...
> error: RPC failed; curl 18 HTTP/2 stream 5 was reset
> error: 54975 bytes of body are still expected
> fetch-pack: unexpected disconnect while reading sideband packet
> fatal: early EOF
> fatal: fetch-pack: invalid index-pack output
> Error: Process completed with exit code 128.
> ```
ACKs for top commit:
maflcko:
lgtm ACK 6889a807661cb570e1464ffdedb9ddb1ec970078
Tree-SHA512: cd50102061a5c57fcf53ecbf552e7eae09995395e91651d00037e3f101c4906c0a994b2f6b1ccf984f2fc902ca59f0db66206ace56a2b6e2810dfa4167f3e115
|
|
Since all bitcoin-ci-test images are now labeled, we can always
prune all dangling images, regardless of whether we are in
RESTART_CI_DOCKER_BEFORE_RUN.
To be safe, still prune all images if RESTART_CI_DOCKER_BEFORE_RUN
in case the filtering doesn't work, or if images were created on
an earlier version that did not assign labels.
|
|
|
|
This variable is required for the `retry` script.
|
|
This reverts commit 057750c09d0a8331c33966d2cc2285ef82f08af8.
It is not needed anymore in the GHA CI.
This change will make the code much simpler in the following commit.
|
|
This allows us or the user to perform batch operations on all
images produced by the ci, e.g. to prune all dangling images,
without affecting non-ci images.
|
|
fa2c894cbb41a64371717139fb3c3ddfb9bb8b19 ci: move-only CI_CONTAINER_ID to 02_run_container.sh (MarcoFalke)
fa695b4df069425414fd26b2ddc08d72a6b506f6 ci: Work around podman stop bug (MarcoFalke)
fa09a031c1eb8abcb9a04cacdf5629f95ffc77f8 ci: Add set -ex to 02_run_container.sh (MarcoFalke)
fac9abbf475a1de6f9f39ddede9a6a59bbd1cff4 ci: Rename 04_install to 02_run_container (MarcoFalke)
Pull request description:
Sometimes, it seems that `podman stop` does not work. Presumably, it falls back to `podman kill`, which is async.
Try to work around this intermittent issue by using the `rm --force` over `stop`.
Example failing log https://cirrus-ci.com/task/4549784611061760?logs=ci#L238:
```
Restart docker before run to stop and clear all containers started with --rm
++ podman container stop --all
e4eca0766f87864d89fc230aa884a238c214cfbcd44cf76a4dbdb2a30c982009
++ echo 'Prune all dangling images'
Prune all dangling images
++ docker image prune --force
Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.
+++ docker run --cap-add LINUX_IMMUTABLE --rm --interactive --detach --tty --mount type=bind,src=/tmp/cirrus-build-1970593815,dst=/tmp/cirrus-build-1970593815,readonly --mount type=volume,src=ci_macos_cross_ccache,dst=/tmp/ccache_dir --mount type=volume,src=ci_macos_cross_depends,dst=/ci_container_base/depends --mount type=volume,src=ci_macos_cross_previous_releases,dst=/ci_container_base/prev_releases --env-file /tmp/env --name ci_macos_cross ci_macos_cross
Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.
time="2023-09-27T20:55:39Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly"
Error: creating container storage: the container name "ci_macos_cross" is already in use by e4eca0766f87864d89fc230aa884a238c214cfbcd44cf76a4dbdb2a30c982009. You have to remove that container to be able to reuse that name: that name is already in use
ACKs for top commit:
hebasto:
ACK fa2c894cbb41a64371717139fb3c3ddfb9bb8b19, I have reviewed the code and tested it locally.
Tree-SHA512: 31fca340c6bedaadf4dd51fa745d9b3969042cebc0c7c904ef18af3f2f986039ec4354ccdff1422fbf77cf223e4423857368dce53cfa67ef15c76b78d007eace
|
|
|
|
78d3062b68988f5094c61a845be756788933c752 ci: Install Android API 31 platform as Qt expects (Hennadii Stepanov)
Pull request description:
When building the `qt` package, it expects that the default (in Qt's view) Android API platform is installed.
During the recent Qt version [update](https://github.com/bitcoin/bitcoin/pull/28561), it has been changed:
```diff
--- a/mkspecs/features/android/sdk.prf
+++ b/mkspecs/features/android/sdk.prf
@@ -1,6 +1,6 @@
API_VERSION_TO_USE = $$(ANDROID_API_VERSION)
isEmpty(API_VERSION_TO_USE): API_VERSION_TO_USE = $$API_VERSION
-isEmpty(API_VERSION_TO_USE): API_VERSION_TO_USE = android-28
+isEmpty(API_VERSION_TO_USE): API_VERSION_TO_USE = android-31
ANDROID_JAR_FILE = $$ANDROID_SDK_ROOT/platforms/$$API_VERSION_TO_USE/android.jar
!exists($$ANDROID_JAR_FILE) {
```
This PR fixes the CI for the Android task and addresses https://github.com/bitcoin/bitcoin/pull/28561#issuecomment-1749180177.
Qt [docs](https://doc.qt.io/qt-5/android.html) still claim that Android API Level 21 and up are supported, however, I did not test every possible configuration.
NOTE: https://github.com/bitcoin/bitcoin/pull/28611 is still valid.
ACKs for top commit:
maflcko:
lgtm ACK 78d3062b68988f5094c61a845be756788933c752
jarolrod:
tACK 78d3062b68988f5094c61a845be756788933c752
Tree-SHA512: 781fba6d80aae7e6500854de14af0d30169c258e395b9e482a5430a7b4a2211a6181f8c9ee58543c896b431abf09e3e7c5573b9672ed128658f11f98a2006e7e
|
|
|
|
|
|
|
|
This limits the scope of the CI_CONTAINER_ID symbol.
Can be reviewed with --color-moved=dimmed-zebra
|
|
Force remove any containers, pontentially leaving dangling processes,
which should be fine.
|
|
The same is done by the 06 script.
|
|
This reflects what the script does (docker run ...).
|
|
|
|
|
|
fa3b5e5e57cd4649d577e06a3aca798b55a75fd5 ci: Use nproc over MAKEJOBS in 01_base_install (MarcoFalke)
Pull request description:
Currently `$MAKEJOBS` is the default value in `01_base_install.sh` when building the container image.
This problem can't be fixed (see below), so just use `nproc` for now.
Other solutions would be bad:
* Passing in the `MAKEJOBS` as a dockerfile env would create a new image if the number of tasks are changed, seems verbose and confusing.
* Leaving `master` as-is would leave CPUs unused if there are more than `4`.
ACKs for top commit:
fanquake:
ACK - I think this is fine as-is, it's an improvement over the current state fa3b5e5e57cd4649d577e06a3aca798b55a75fd5
Tree-SHA512: 15014eb7b548945737b0fed5cd46f0cd4b72b1d54d044f60fce628f914053a2de6518878428a54822d236d08d6f257d8c464d3fb589400f4be56022d2ec8676d
|
|
b5790c35f7e1d48c79b83bded36f3f72c18c9fc1 build: remove dmg dependencies (fanquake)
33ae0bd1e4756ca0f180ac4b3c32c9eb83b88cfd macdeploy: remove DMG generation from deploy script (fanquake)
a128111c29ba0c31763ccbcd316268bfa9c029cd build: produce a .zip for macOS distribution (Hennadii Stepanov)
c38561d6b1de954b712a92cb8a198ed42d73caea build: add -zip option to macdeployqtplus (fanquake)
Pull request description:
It is https://github.com/bitcoin/bitcoin/pull/27099 revived with addressed [comments](https://github.com/bitcoin/bitcoin/pull/27099#issuecomment-1708705686).
From https://github.com/bitcoin/bitcoin/pull/27099#issue-1584429885:
> Reviving the discussion around using a `.zip` for the distributed macOS binaries, as opposed to a `.dmg`.
>
> Given we only had a single report of the "no finder window" issue (#26176), I wonder if that means macOS users were able to figure it out, they gave up/didn't report, or, we just have very few macOS users.
>
> Related to #18128.
That's how it looks on macOS:
![image](https://github.com/bitcoin/bitcoin/assets/32963518/baa637bb-256b-4b24-8645-8c2754c2ae64)
ACKs for top commit:
Sjors:
tACK b5790c35f7e1d48c79b83bded36f3f72c18c9fc1
jarolrod:
ACK b5790c35f7e1d48c79b83bded36f3f72c18c9fc1
TheCharlatan:
utACK b5790c35f7e1d48c79b83bded36f3f72c18c9fc1
Tree-SHA512: 6e9cb3ab0f60f8a92bfec50577e8d096c5b23ec09ebbb334826415609140ddc96d470aea37379495c1c6bb1beec0d306b09460f62e1543bb0f4396c10a1dfbe2
|
|
|
|
|
|
Instead of a .dmg.
Co-authored-by: fanquake <fanquake@gmail.com>
|
|
a241d6069cf0542acdd8ec6be63724da19f10720 ci: use LLVM 17.0.0 in MSAN jobs (fanquake)
Pull request description:
See https://libcxx.llvm.org/Hardening.html as well as https://discourse.llvm.org/t/rfc-removing-the-legacy-debug-mode-from-libc/71026.
ACKs for top commit:
MarcoFalke:
review ACK a241d6069cf0542acdd8ec6be63724da19f10720
Tree-SHA512: c374dabf307fe762be0da96f63695a150f6018c1468fe9414fad23f74f5818bbf7a5a699e109084e31467482a900cfebf1d5835821e4da94aa310b2c9570749c
|
|
blockstore
de8f9123afbecc3b4f59fa80af8148bc865d0588 test: cover read-only blockstore (Matthew Zipkin)
5c2185b3b624ce87320ec16412f98ab591a5860c ci: enable chattr +i capability inside containers (Matthew Zipkin)
e573f2420244c583e218f51cd0d3a3cac6731003 unit test: add coverage for BlockManager (Matthew Zipkin)
Pull request description:
This PR adds unit and functional tests to cover the behavior described in #2039. In particular, that bitcoind will crash on startup if a reindex is requested but the `blk` files are read-only. Eventually this behavior can be updated with https://github.com/bitcoin/bitcoin/pull/27039. This PR just commits the test coverage from #27039 as suggested in https://github.com/bitcoin/bitcoin/pull/27039#issuecomment-1584915782
ACKs for top commit:
jonatack:
ACK de8f9123afbecc3b4f59fa80af8148bc865d0588 modulo suggestions in https://github.com/bitcoin/bitcoin/pull/27850#discussion_r1319010039, tested on macOS, but not on Linux for the Linux-related change in the last push
achow101:
ACK de8f9123afbecc3b4f59fa80af8148bc865d0588
MarcoFalke:
lgtm ACK de8f9123afbecc3b4f59fa80af8148bc865d0588 📶
Tree-SHA512: b9bd684035dcea11c901b649fc39f397a2155a9a8459f3348e67947e387e45312fddeccb52981aef486f8a31deebb5356a7901c1bb94b78f82c24192a369af73
|
|
See https://libcxx.llvm.org/Hardening.html as well as
https://discourse.llvm.org/t/rfc-removing-the-legacy-debug-mode-from-libc/71026.
|
|
|
|
|
|
fa07ac48d804beac38a98d23be2167f78cadefae ci: Asan with -ftrivial-auto-var-init=pattern (MarcoFalke)
Pull request description:
This makes memory bugs deterministic. `-ftrivial-auto-var-init=pattern` is incompatible with other memory sanitizers (like valgrind and msan), but that is irrelevant here, because the address sanitizer in this fuzz CI config is already incompatible with them.
`-ftrivial-auto-var-init=pattern` goes well with `-fsanitize=bool` and `-fsanitize=enum`, but those are already enabled via `-fsanitize=undefined`. See https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html#available-checks
ACKs for top commit:
fanquake:
ACK fa07ac48d804beac38a98d23be2167f78cadefae - going to get back to fixing up the cxxflags usage in CI, but not a blocker here:
Tree-SHA512: 2ea6c5128a9cd262bdd1b1475c7e1be23ce2c520fad05f6c2aace6c29e203573323c0564d272e25da35ad5ff46fde488a5eae1ed14d3349e793d14a5e2533fb1
|
|
from s390x task
fa70cbd9693b0ccc56114647f0817c96b2f223ee ci: Remove unused TEST_RUNNER_ENV="LC_ALL=C" from s390x task (MarcoFalke)
fa33354dcb6f289965e84a0cd4492c43dafd917b ci: Remove /ro_base bind mount (MarcoFalke)
fa0df9d4c4cb93b0971f92f3fe6f9a1deb5b311f doc: Remove sudo from command that is already run as root (MarcoFalke)
Pull request description:
Remove some CI stuff no longer needed.
ACKs for top commit:
fanquake:
ACK fa70cbd9693b0ccc56114647f0817c96b2f223ee - did not test the s390x job.
Tree-SHA512: 3a6ed0cfc855a92c2f834e59494c0a19a5647510247aece5e40a1aa78074894fe7454e684a1ea1f8f0662c50ac1caf2e390398b0fcfbf81544e6488fa9b8915e
|
|
|
|
6a4337298035683a8748ae446eae90e5c5f41d1b ci: Run "Win64 native" job on GitHub Actions (Hennadii Stepanov)
Pull request description:
From https://github.com/bitcoin/bitcoin/issues/28098:
> Thus, someone would have to sponsor an amount of roughly 5kUSD/mo for those two tasks.
> If the goal is to stay on a free plan, I think the only option is GitHub Actions CI.
Historical context:
- https://github.com/bitcoin/bitcoin/pull/17697
- https://github.com/bitcoin/bitcoin/issues/17803
- https://github.com/bitcoin/bitcoin/pull/18031
Security concerns:
- https://github.com/bitcoin/bitcoin/issues/28098#issuecomment-1651432106
- https://github.com/bitcoin/bitcoin/issues/28098#issuecomment-1651688197
`GITHUB_TOKEN` permissions (from the build log in my personal repo):
```
2023-07-27T07:30:17.8313534Z ##[group]GITHUB_TOKEN Permissions
2023-07-27T07:30:17.8314113Z Contents: read
2023-07-27T07:30:17.8314608Z Metadata: read
2023-07-27T07:30:17.8314957Z Packages: read
2023-07-27T07:30:17.8315233Z ##[endgroup]
```
Comparison of resources:
| Resource | Current, Cirrus CI | Suggested, GitHub Actions |
|---|:-:|:-:|
| CPU | 6 | 2 |
| RAM, GB | 12 | 7 |
The `TEST_RUNNER_TIMEOUT_FACTOR` variable is set to the current default value for all CI tasks: https://github.com/bitcoin/bitcoin/blob/64440bb733896a7a2caf902825e0406cb993e666/ci/test/00_setup_env.sh#L48
Top commit has no ACKs.
Tree-SHA512: ddfdaf7a1e4793a64ac0cd20f116b29608dd06f15b062769ac70b3ea2fb82775aa96aa79c7b768efefec4338aaa5b57d267b592f62d0e8d5d94ecc11001a165d
|
|
|
|
Just set the bind mount to BASE_READ_ONLY_DIR, which allows to drop one
line of code and makes the code easier to understand.
|
|
There is no x86_64 binaries for 15.0.7.
|
|
|
|
tasks to CI_IMAGE_NAME_TAG
fab7f5c01d0b4be00bdd8922d3bf8bd1a27ba8fb ci: Add missing docker.io prefix to CI_IMAGE_NAME_TAG (MarcoFalke)
Pull request description:
Currently, the CI system may pick the wrong (non-native) architecture due to the missing prefix.
For example, assuming the CI_IMAGE_NAME_TAG is `debian:bookworm` and the user has previously pulled an s390x image:
```
$ podman run --rm 'docker.io/s390x/debian:bookworm' dpkg --print-architecture
exec /usr/bin/dpkg: exec format error
```
Now, `debian:bookworm` will refer to the same image:
```
$ podman run --rm 'debian:bookworm' dpkg --print-architecture
exec /usr/bin/dpkg: exec format error
```
However, `docker.io/debian:bookworm` works fine:
```
$ podman run --rm 'docker.io/debian:bookworm' dpkg --print-architecture
arm64
ACKs for top commit:
hebasto:
ACK fab7f5c01d0b4be00bdd8922d3bf8bd1a27ba8fb.
Tree-SHA512: c423c5cd454a95fa3e67081411ca08d316b8c680a5bba49196c514b91df65d9cc46a47700cc00d9579327842615f98146d0ac50abb016616a9b17d042598dab6
|
|
|
|
This fails with:
"Error: determining starting point for build: no FROM statement found"
|
|
This should help to avoid non-determinism.
|
|
No need to have a larger scope than needed.
Can be reviewed with --color-moved=dimmed-zebra
|
|
fa56d17a4b868f42fa45bea0d1f0c78de75e7838 ci: Add missing amd64 to win64-cross task (MarcoFalke)
Pull request description:
Currently the task will fail if run on non-`x86_64`.
Fix this by adding the missing `amd64`, similar to
https://github.com/bitcoin/bitcoin/blob/7bf078f2b7d4a0339d053144b4fb35fe020dac25/ci/test/00_setup_env_i686_multiprocess.sh#L11
ACKs for top commit:
hebasto:
ACK fa56d17a4b868f42fa45bea0d1f0c78de75e7838
Tree-SHA512: faab1c5b945283b7e8d080bbcc8e9379c480cf6973506149ace5990cb4d04673f83f4bc36d08d5b4e9cb17a86fdbe23ac97ef4eab0e842616b367b8138229c58
|
|
Also, do the same for android, which also fails.
|
|
|
|
|
|
This avoids "mkdir: /ci_container_base: Read-only file system"
|
|
This should fix the macOS-cross build on Cirrus CI containers.
Locally this was already working, because the SDK was cached in
/ci_container_base/ in the image, which is also the folder used for a
later CI run.
However, on Cirrus CI, when using an image *and* a custom BASE_ROOT_DIR,
the SDK will not be found in /ci_base_install/, nor in BASE_ROOT_DIR.
Fix this by normalizing *all* folders to /ci_container_base/.
|
|
9658d0dc17c270138523c41a982425e276b24271 ci: Run "macOS native x86_64" job on GitHub Actions (Hennadii Stepanov)
Pull request description:
From https://github.com/bitcoin/bitcoin/issues/28098:
> Thus, someone would have to sponsor an amount of roughly 5kUSD/mo for those two tasks.
> If the goal is to stay on a free plan, I think the only option is GitHub Actions CI.
---
**IMPORTANT NOTE**. We currently ship macOS release binaries for both architectures: `x86_64` and `arm64`. If this PR gets merged, only `x86_64` architecture will be tested on CI, which implies some [drawbacks](https://github.com/bitcoin/bitcoin/pull/28187#issuecomment-1658077549).
However, it has never been the case that our CI tested both architectures simultaneously. And we hope that GitHub Actions will soon host macOS `arm64` runners.
Historically, we moved from `x86_64` to `arm64` in https://github.com/bitcoin/bitcoin/pull/26388 less than a year ago.
---
Security concerns:
- https://github.com/bitcoin/bitcoin/issues/28098#issuecomment-1651432106
- https://github.com/bitcoin/bitcoin/issues/28098#issuecomment-1651688197
`GITHUB_TOKEN` permissions (from the build log in my personal repo):
```
2023-07-27T07:30:17.8313534Z ##[group]GITHUB_TOKEN Permissions
2023-07-27T07:30:17.8314113Z Contents: read
2023-07-27T07:30:17.8314608Z Metadata: read
2023-07-27T07:30:17.8314957Z Packages: read
2023-07-27T07:30:17.8315233Z ##[endgroup]
```
Comparison of resources:
| Resource | Current, Cirrus CI | Suggested, GitHub Actions |
|---|:-:|:-:|
| CPU | 4 | 4 \*\* |
| RAM, GB | 8 | 14 |
**\*\* NOTE**: However, [docs](https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners#supported-runners-and-hardware-resources) are mentioning:
> 3-core CPU (x86_64)
ACKs for top commit:
MarcoFalke:
re-ACK 9658d0dc17c270138523c41a982425e276b24271 🏂
achow101:
ACK 9658d0dc17c270138523c41a982425e276b24271
jarolrod:
ACK 9658d0dc17c270138523c41a982425e276b24271
Tree-SHA512: 6123e68e6784cdf4e53c3e77b435709261db21f09091af2c22e667d3816a305fffb9d617297a5bc1bda18aaba84a6e210cec6a75c52afa7746a3780a67b69865
|
|
|