Age | Commit message (Collapse) | Author |
|
|
|
fa3b5e5e57cd4649d577e06a3aca798b55a75fd5 ci: Use nproc over MAKEJOBS in 01_base_install (MarcoFalke)
Pull request description:
Currently `$MAKEJOBS` is the default value in `01_base_install.sh` when building the container image.
This problem can't be fixed (see below), so just use `nproc` for now.
Other solutions would be bad:
* Passing in the `MAKEJOBS` as a dockerfile env would create a new image if the number of tasks are changed, seems verbose and confusing.
* Leaving `master` as-is would leave CPUs unused if there are more than `4`.
ACKs for top commit:
fanquake:
ACK - I think this is fine as-is, it's an improvement over the current state fa3b5e5e57cd4649d577e06a3aca798b55a75fd5
Tree-SHA512: 15014eb7b548945737b0fed5cd46f0cd4b72b1d54d044f60fce628f914053a2de6518878428a54822d236d08d6f257d8c464d3fb589400f4be56022d2ec8676d
|
|
b5790c35f7e1d48c79b83bded36f3f72c18c9fc1 build: remove dmg dependencies (fanquake)
33ae0bd1e4756ca0f180ac4b3c32c9eb83b88cfd macdeploy: remove DMG generation from deploy script (fanquake)
a128111c29ba0c31763ccbcd316268bfa9c029cd build: produce a .zip for macOS distribution (Hennadii Stepanov)
c38561d6b1de954b712a92cb8a198ed42d73caea build: add -zip option to macdeployqtplus (fanquake)
Pull request description:
It is https://github.com/bitcoin/bitcoin/pull/27099 revived with addressed [comments](https://github.com/bitcoin/bitcoin/pull/27099#issuecomment-1708705686).
From https://github.com/bitcoin/bitcoin/pull/27099#issue-1584429885:
> Reviving the discussion around using a `.zip` for the distributed macOS binaries, as opposed to a `.dmg`.
>
> Given we only had a single report of the "no finder window" issue (#26176), I wonder if that means macOS users were able to figure it out, they gave up/didn't report, or, we just have very few macOS users.
>
> Related to #18128.
That's how it looks on macOS:
![image](https://github.com/bitcoin/bitcoin/assets/32963518/baa637bb-256b-4b24-8645-8c2754c2ae64)
ACKs for top commit:
Sjors:
tACK b5790c35f7e1d48c79b83bded36f3f72c18c9fc1
jarolrod:
ACK b5790c35f7e1d48c79b83bded36f3f72c18c9fc1
TheCharlatan:
utACK b5790c35f7e1d48c79b83bded36f3f72c18c9fc1
Tree-SHA512: 6e9cb3ab0f60f8a92bfec50577e8d096c5b23ec09ebbb334826415609140ddc96d470aea37379495c1c6bb1beec0d306b09460f62e1543bb0f4396c10a1dfbe2
|
|
|
|
|
|
Instead of a .dmg.
Co-authored-by: fanquake <fanquake@gmail.com>
|
|
a241d6069cf0542acdd8ec6be63724da19f10720 ci: use LLVM 17.0.0 in MSAN jobs (fanquake)
Pull request description:
See https://libcxx.llvm.org/Hardening.html as well as https://discourse.llvm.org/t/rfc-removing-the-legacy-debug-mode-from-libc/71026.
ACKs for top commit:
MarcoFalke:
review ACK a241d6069cf0542acdd8ec6be63724da19f10720
Tree-SHA512: c374dabf307fe762be0da96f63695a150f6018c1468fe9414fad23f74f5818bbf7a5a699e109084e31467482a900cfebf1d5835821e4da94aa310b2c9570749c
|
|
blockstore
de8f9123afbecc3b4f59fa80af8148bc865d0588 test: cover read-only blockstore (Matthew Zipkin)
5c2185b3b624ce87320ec16412f98ab591a5860c ci: enable chattr +i capability inside containers (Matthew Zipkin)
e573f2420244c583e218f51cd0d3a3cac6731003 unit test: add coverage for BlockManager (Matthew Zipkin)
Pull request description:
This PR adds unit and functional tests to cover the behavior described in #2039. In particular, that bitcoind will crash on startup if a reindex is requested but the `blk` files are read-only. Eventually this behavior can be updated with https://github.com/bitcoin/bitcoin/pull/27039. This PR just commits the test coverage from #27039 as suggested in https://github.com/bitcoin/bitcoin/pull/27039#issuecomment-1584915782
ACKs for top commit:
jonatack:
ACK de8f9123afbecc3b4f59fa80af8148bc865d0588 modulo suggestions in https://github.com/bitcoin/bitcoin/pull/27850#discussion_r1319010039, tested on macOS, but not on Linux for the Linux-related change in the last push
achow101:
ACK de8f9123afbecc3b4f59fa80af8148bc865d0588
MarcoFalke:
lgtm ACK de8f9123afbecc3b4f59fa80af8148bc865d0588 📶
Tree-SHA512: b9bd684035dcea11c901b649fc39f397a2155a9a8459f3348e67947e387e45312fddeccb52981aef486f8a31deebb5356a7901c1bb94b78f82c24192a369af73
|
|
See https://libcxx.llvm.org/Hardening.html as well as
https://discourse.llvm.org/t/rfc-removing-the-legacy-debug-mode-from-libc/71026.
|
|
|
|
|
|
fa07ac48d804beac38a98d23be2167f78cadefae ci: Asan with -ftrivial-auto-var-init=pattern (MarcoFalke)
Pull request description:
This makes memory bugs deterministic. `-ftrivial-auto-var-init=pattern` is incompatible with other memory sanitizers (like valgrind and msan), but that is irrelevant here, because the address sanitizer in this fuzz CI config is already incompatible with them.
`-ftrivial-auto-var-init=pattern` goes well with `-fsanitize=bool` and `-fsanitize=enum`, but those are already enabled via `-fsanitize=undefined`. See https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html#available-checks
ACKs for top commit:
fanquake:
ACK fa07ac48d804beac38a98d23be2167f78cadefae - going to get back to fixing up the cxxflags usage in CI, but not a blocker here:
Tree-SHA512: 2ea6c5128a9cd262bdd1b1475c7e1be23ce2c520fad05f6c2aace6c29e203573323c0564d272e25da35ad5ff46fde488a5eae1ed14d3349e793d14a5e2533fb1
|
|
from s390x task
fa70cbd9693b0ccc56114647f0817c96b2f223ee ci: Remove unused TEST_RUNNER_ENV="LC_ALL=C" from s390x task (MarcoFalke)
fa33354dcb6f289965e84a0cd4492c43dafd917b ci: Remove /ro_base bind mount (MarcoFalke)
fa0df9d4c4cb93b0971f92f3fe6f9a1deb5b311f doc: Remove sudo from command that is already run as root (MarcoFalke)
Pull request description:
Remove some CI stuff no longer needed.
ACKs for top commit:
fanquake:
ACK fa70cbd9693b0ccc56114647f0817c96b2f223ee - did not test the s390x job.
Tree-SHA512: 3a6ed0cfc855a92c2f834e59494c0a19a5647510247aece5e40a1aa78074894fe7454e684a1ea1f8f0662c50ac1caf2e390398b0fcfbf81544e6488fa9b8915e
|
|
|
|
6a4337298035683a8748ae446eae90e5c5f41d1b ci: Run "Win64 native" job on GitHub Actions (Hennadii Stepanov)
Pull request description:
From https://github.com/bitcoin/bitcoin/issues/28098:
> Thus, someone would have to sponsor an amount of roughly 5kUSD/mo for those two tasks.
> If the goal is to stay on a free plan, I think the only option is GitHub Actions CI.
Historical context:
- https://github.com/bitcoin/bitcoin/pull/17697
- https://github.com/bitcoin/bitcoin/issues/17803
- https://github.com/bitcoin/bitcoin/pull/18031
Security concerns:
- https://github.com/bitcoin/bitcoin/issues/28098#issuecomment-1651432106
- https://github.com/bitcoin/bitcoin/issues/28098#issuecomment-1651688197
`GITHUB_TOKEN` permissions (from the build log in my personal repo):
```
2023-07-27T07:30:17.8313534Z ##[group]GITHUB_TOKEN Permissions
2023-07-27T07:30:17.8314113Z Contents: read
2023-07-27T07:30:17.8314608Z Metadata: read
2023-07-27T07:30:17.8314957Z Packages: read
2023-07-27T07:30:17.8315233Z ##[endgroup]
```
Comparison of resources:
| Resource | Current, Cirrus CI | Suggested, GitHub Actions |
|---|:-:|:-:|
| CPU | 6 | 2 |
| RAM, GB | 12 | 7 |
The `TEST_RUNNER_TIMEOUT_FACTOR` variable is set to the current default value for all CI tasks: https://github.com/bitcoin/bitcoin/blob/64440bb733896a7a2caf902825e0406cb993e666/ci/test/00_setup_env.sh#L48
Top commit has no ACKs.
Tree-SHA512: ddfdaf7a1e4793a64ac0cd20f116b29608dd06f15b062769ac70b3ea2fb82775aa96aa79c7b768efefec4338aaa5b57d267b592f62d0e8d5d94ecc11001a165d
|
|
|
|
Just set the bind mount to BASE_READ_ONLY_DIR, which allows to drop one
line of code and makes the code easier to understand.
|
|
There is no x86_64 binaries for 15.0.7.
|
|
|
|
tasks to CI_IMAGE_NAME_TAG
fab7f5c01d0b4be00bdd8922d3bf8bd1a27ba8fb ci: Add missing docker.io prefix to CI_IMAGE_NAME_TAG (MarcoFalke)
Pull request description:
Currently, the CI system may pick the wrong (non-native) architecture due to the missing prefix.
For example, assuming the CI_IMAGE_NAME_TAG is `debian:bookworm` and the user has previously pulled an s390x image:
```
$ podman run --rm 'docker.io/s390x/debian:bookworm' dpkg --print-architecture
exec /usr/bin/dpkg: exec format error
```
Now, `debian:bookworm` will refer to the same image:
```
$ podman run --rm 'debian:bookworm' dpkg --print-architecture
exec /usr/bin/dpkg: exec format error
```
However, `docker.io/debian:bookworm` works fine:
```
$ podman run --rm 'docker.io/debian:bookworm' dpkg --print-architecture
arm64
ACKs for top commit:
hebasto:
ACK fab7f5c01d0b4be00bdd8922d3bf8bd1a27ba8fb.
Tree-SHA512: c423c5cd454a95fa3e67081411ca08d316b8c680a5bba49196c514b91df65d9cc46a47700cc00d9579327842615f98146d0ac50abb016616a9b17d042598dab6
|
|
|
|
No need to have a larger scope than needed.
Can be reviewed with --color-moved=dimmed-zebra
|
|
fa56d17a4b868f42fa45bea0d1f0c78de75e7838 ci: Add missing amd64 to win64-cross task (MarcoFalke)
Pull request description:
Currently the task will fail if run on non-`x86_64`.
Fix this by adding the missing `amd64`, similar to
https://github.com/bitcoin/bitcoin/blob/7bf078f2b7d4a0339d053144b4fb35fe020dac25/ci/test/00_setup_env_i686_multiprocess.sh#L11
ACKs for top commit:
hebasto:
ACK fa56d17a4b868f42fa45bea0d1f0c78de75e7838
Tree-SHA512: faab1c5b945283b7e8d080bbcc8e9379c480cf6973506149ace5990cb4d04673f83f4bc36d08d5b4e9cb17a86fdbe23ac97ef4eab0e842616b367b8138229c58
|
|
Also, do the same for android, which also fails.
|
|
|
|
|
|
This avoids "mkdir: /ci_container_base: Read-only file system"
|
|
This should fix the macOS-cross build on Cirrus CI containers.
Locally this was already working, because the SDK was cached in
/ci_container_base/ in the image, which is also the folder used for a
later CI run.
However, on Cirrus CI, when using an image *and* a custom BASE_ROOT_DIR,
the SDK will not be found in /ci_base_install/, nor in BASE_ROOT_DIR.
Fix this by normalizing *all* folders to /ci_container_base/.
|
|
9658d0dc17c270138523c41a982425e276b24271 ci: Run "macOS native x86_64" job on GitHub Actions (Hennadii Stepanov)
Pull request description:
From https://github.com/bitcoin/bitcoin/issues/28098:
> Thus, someone would have to sponsor an amount of roughly 5kUSD/mo for those two tasks.
> If the goal is to stay on a free plan, I think the only option is GitHub Actions CI.
---
**IMPORTANT NOTE**. We currently ship macOS release binaries for both architectures: `x86_64` and `arm64`. If this PR gets merged, only `x86_64` architecture will be tested on CI, which implies some [drawbacks](https://github.com/bitcoin/bitcoin/pull/28187#issuecomment-1658077549).
However, it has never been the case that our CI tested both architectures simultaneously. And we hope that GitHub Actions will soon host macOS `arm64` runners.
Historically, we moved from `x86_64` to `arm64` in https://github.com/bitcoin/bitcoin/pull/26388 less than a year ago.
---
Security concerns:
- https://github.com/bitcoin/bitcoin/issues/28098#issuecomment-1651432106
- https://github.com/bitcoin/bitcoin/issues/28098#issuecomment-1651688197
`GITHUB_TOKEN` permissions (from the build log in my personal repo):
```
2023-07-27T07:30:17.8313534Z ##[group]GITHUB_TOKEN Permissions
2023-07-27T07:30:17.8314113Z Contents: read
2023-07-27T07:30:17.8314608Z Metadata: read
2023-07-27T07:30:17.8314957Z Packages: read
2023-07-27T07:30:17.8315233Z ##[endgroup]
```
Comparison of resources:
| Resource | Current, Cirrus CI | Suggested, GitHub Actions |
|---|:-:|:-:|
| CPU | 4 | 4 \*\* |
| RAM, GB | 8 | 14 |
**\*\* NOTE**: However, [docs](https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners#supported-runners-and-hardware-resources) are mentioning:
> 3-core CPU (x86_64)
ACKs for top commit:
MarcoFalke:
re-ACK 9658d0dc17c270138523c41a982425e276b24271 🏂
achow101:
ACK 9658d0dc17c270138523c41a982425e276b24271
jarolrod:
ACK 9658d0dc17c270138523c41a982425e276b24271
Tree-SHA512: 6123e68e6784cdf4e53c3e77b435709261db21f09091af2c22e667d3816a305fffb9d617297a5bc1bda18aaba84a6e210cec6a75c52afa7746a3780a67b69865
|
|
|
|
Now that container volumes are used, the folders are no longer mounted.
They are only needed when running without a container engine (docker,
podman).
|
|
Using a hard-coded path avoids non-determinism issues and improves CI
UX.
|
|
Also, the "macOS native arm64" task has been removed from Cirrus CI.
|
|
|
|
1c976c691cc4b20f43071aabf36c7afed1571057 tidy: Integrate bicoin-tidy clang-tidy plugin (fanquake)
7de23cceb8ac13fcc709453ef0fa14fb93c460b0 refactor: fix unterminated LogPrintf()s (fanquake)
0a1029aa2946787bfe91a84de882c2dd0de076e9 lint: remove /* Continued */ markers from codebase (fanquake)
910007995d8603ffc466878856227153a638caff lint: remove lint-logs.py (fanquake)
d86a83d6b8587b0971e66c6910af23dd8c042969 lint: drop DIR_IWYU global (fanquake)
Pull request description:
Demo of integrating the [bitcoin-tidy](https://github.com/theuni/bitcoin-tidy-plugin), [clang-tidy plugin](https://clang.llvm.org/extra/clang-tidy/) written by theuni into our tidy CI job.
The plugin currently has a single check, `bitcoin-unterminated-logprintf`. This would replace our current Python driven, `git-grep`-based, `.cpp` file only, lint-logs linter.
ACKs for top commit:
TheCharlatan:
ACK 1c976c691cc4b20f43071aabf36c7afed1571057
theuni:
ACK 1c976c691cc4b20f43071aabf36c7afed1571057
MarcoFalke:
re-ACK 1c976c691cc4b20f43071aabf36c7afed1571057 👠
Tree-SHA512: 725b45c70e431d48e6f276671e05c694e10b6047cae1a31906ac3ee9093bc8105fb226b36a5bac6709557526ca6007222112d66aecec05a574434edc4897e4b8
|
|
Enable `bitcoin-unterminated-logprintf`.
Co-authored-by: Cory Fields <cory-nospam-@coryfields.com>
|
|
|
|
Otherwise the task will throw in skip_if_no_python_bcc.
Also, adjust CI_CONTAINER_CAP for all needed permissions.
|
|
This change aims to:
1) Remove our own `CCACHE_SIZE` environment variable that violates
Ccache's `CCACHE_*` namespace.
2) Introduce the `CCACHE_MAXSIZE` environment variable that is
documented since v3.3, which makes its usage consistent with other ones,
such as `CCACHE_DIR` and `CCACHE_NOHASHDIR`.
|
|
fabc04a4d96c4fe70e60d365aa28031d149094f3 ci: Keep system env vars as-is (MarcoFalke)
fa8dcdcc8b29e58f5d285a49dde33d94b63c893b ci: Set PATH inside the CI env (MarcoFalke)
fac229ab1f95ec77f18be8a783a2779dd781c684 ci: Remove P_CI_DIR and --workdir (MarcoFalke)
Pull request description:
This fixes a bug where the `$PATH` from the host is used inside the container. This will lead to bugs when the `$PATH` is different. For example on a host of Fedora 38, and a container of `debian:bullseye`.
This can be tested with the `FILE_ENV=./ci/test/00_setup_env_arm.sh` CI env. On master:
```
Error: crun: executable file `bash` not found in $PATH: No such file or directory: OCI runtime attempted to invoke a command that was not found
```
On this pull:
(everything passes)
ACKs for top commit:
TheCharlatan:
lgtm ACK fabc04a4d96c4fe70e60d365aa28031d149094f3
Tree-SHA512: 51d2affed91624d0e5b43a3ee1e696313f934e05fde6b5a19e8ac4e6666c1e7b667a444bf3de3f77190bcd00e81efb7576196afb0f6b5e788d1a50e7ddb28d7e
|
|
|
|
This is needed for the next commit.
This also requires dropping CI_RETRY from the docker build step, which
is fine, because CI_RETRY should be called inside the build script, not
outside.
Also, fix a doc typo.
|
|
The --workdir setting to the docker run command is not needed. And
P_CI_DIR/PWD is equal to BASE_ROOT_DIR, so just use that directly.
|
|
`noreturn` attributes have been added to the mingw-w64 headers, meaning
that from 11.0.0 onwards, you'll no-longer see `-Wreturn-type` warnings
when using assert(false):
https://github.com/mingw-w64/mingw-w64/commit/1690994f515910a31b9fb7c7bd3a52d4ba987abe.
Add -Wno-return-type to the Windows CI, where is should have been all
along, and document why it's required. This can be dropped when we are
using the fixed version of the mingw-w64 headers there.
Drop the -Werror -Wno-return-type special case from our build system.
-Wreturn-type is on by default in Clang and GCC.
|
|
|
|
|
|
These work for me now. If they still don't work in other setups,
maybe we can better document the issues.
```bash
time FILE_ENV="./ci/test/00_setup_env_s390x.sh" ./ci/test_run_all.sh
...
Running tests: coins_tests from test/coins_tests.cpp
PASS: qt/test/test_bitcoin-qt
Running tests: coinstatsindex_tests from test/coinstatsindex_tests.cpp
...
Stop and remove CI container by ID
+ docker container kill 617bef8accb87530e5fbb03ff07b3b9f0aa9e3030d4da424c9612d153ab98dbf
617bef8accb87530e5fbb03ff07b3b9f0aa9e3030d4da424c9612d153ab98dbf
real 51m37.809s
```
|
|
62633b50461cb67dfb37d6485e604152e727559c ci: filter all subtrees from tidy output (fanquake)
Pull request description:
We are currently dumping output for some. i.e:
```bash
diff --git a/src/minisketch/src/fields/clmul_1byte.cpp b/src/minisketch/src/fields/clmul_1byte.cpp
index 8826af9..7fd6f2a 100644
--- a/src/minisketch/src/fields/clmul_1byte.cpp
+++ b/src/minisketch/src/fields/clmul_1byte.cpp
@@ -4,21 +4,16 @@
* file LICENSE or http://www.opensource.org/licenses/mit-license.php.*
**********************************************************************/
-/* This file was substantially auto-generated by doc/gen_params.sage. */
-#include "../fielddefines.h"
-
+class Sketch;
#if defined(ENABLE_FIELD_BYTES_INT_1)
```
ACKs for top commit:
hebasto:
re-ACK 62633b50461cb67dfb37d6485e604152e727559c
Tree-SHA512: fd0a17af6b37fc7641547dab329c2d14ec784941c4d100db1e80d232aff39e45ad9c588982810a2cfc54b4fe820bfe0d50638b53209fec6774fd556b9b0ae180
|
|
fae7c50d201726f605938c3511dd9119efeea5ec test: Run fuzz tests on macOS (MarcoFalke)
Pull request description:
Any reason not to?
ACKs for top commit:
jamesob:
Github ACK https://github.com/bitcoin/bitcoin/pull/27932/commits/fae7c50d201726f605938c3511dd9119efeea5ec
dergoegge:
utACK fae7c50d201726f605938c3511dd9119efeea5ec
Tree-SHA512: e45122d73fafb17cea312258314b826cb0745e08daadd28465f687ec02d4c127d2f8cbe20179a9fff5712038850c02c968abb4838fa088b7555e28709317d3a3
|
|
We are currently dumping output for some. i.e:
```bash
diff --git a/src/minisketch/src/fields/clmul_1byte.cpp b/src/minisketch/src/fields/clmul_1byte.cpp
index 8826af9..7fd6f2a 100644
--- a/src/minisketch/src/fields/clmul_1byte.cpp
+++ b/src/minisketch/src/fields/clmul_1byte.cpp
@@ -4,21 +4,16 @@
* file LICENSE or http://www.opensource.org/licenses/mit-license.php.*
**********************************************************************/
-/* This file was substantially auto-generated by doc/gen_params.sage. */
-#include "../fielddefines.h"
-
+class Sketch;
#if defined(ENABLE_FIELD_BYTES_INT_1)
```
|