aboutsummaryrefslogtreecommitdiff
path: root/ci
AgeCommit message (Collapse)Author
2023-01-28ci: Cache package manager install stepMarcoFalke
2023-01-25refactor: Remove stray cs_main redundant declarationMarcoFalke
2023-01-24ci: Fix APPEND_APT_SOURCES_LIST trying to modify the host systemMarcoFalke
2023-01-19Merge bitcoin/bitcoin#26906: test: add an easy way to run linters locallyMarcoFalke
b68e5a7feff3e93027e75da0cd9a590fef99aac1 lint: specify the right commit range when running locally (James O'Beirne) dff7ed5732c01b3ba47ef9e9ddb687d073d7739d test: add an easy way to run linters locally (James O'Beirne) Pull request description: Adds a Dockerfile configuration ~~(originally written mostly by fanquake)~~ that allows straightforward running of linters with compatible versions locally. This removes a ton of annoyance when trying to appease CI, because many of the linter versions are quite old and difficult to maintain locally. I realize that people may not be thrilled to add more ancillary tooling to the repo, but I think this makes a lot of sense given the linter versions listed in this container configuration are dictated by this repo (within the CI configuration), so having these things live in two separate places is a recipe for version mismatches. Eventually we can likely just use this container on CI directly to avoid any chance of inconsistencies between local dev experience and CI. ACKs for top commit: aureleoules: ACK b68e5a7feff3e93027e75da0cd9a590fef99aac1 stickies-v: ACK b68e5a7fe john-moffett: ACK b68e5a7feff3e93027e75da0cd9a590fef99aac1 Tree-SHA512: 7ef7a5dae023d81fdb6296d5d92dfa074ee321c7993e607c9f014d0f21c91558611aa00fc3ce1edc7b5e68371aea0d27fa1931291a79bb867a6c783bb536775f
2023-01-18lint: specify the right commit range when running locallyJames O'Beirne
When running lints on Cirrus, a special envvar is set ($CIRRUS_PR); emulate this when running linters locally by setting $LOCAL_BRANCH to any value.
2023-01-18test: add an easy way to run linters locallyJames O'Beirne
Adds a Dockerfile configuration that allows straightforward running of linters with compatible versions locally. This removes a ton of annoyance when trying to appease CI, because many of the linter versions are quite old and difficult to maintain locally. I realize that people may not be thrilled to more ancillary tooling to the repo, but I think this makes a lot of sense given the linter versions listed in this container configuration are dictated by this repo (within the CI configuration), so having these things live in two separate places is a recipe for version mismatches. Eventually we can likely just use this container on CI directly to avoid any chance of inconsistencies between local dev experience and CI.
2023-01-18Bump minimum python version to 3.7MarcoFalke
2023-01-18Merge bitcoin/bitcoin#26908: ci: Add missing lint dependency (gpg)fanquake
fa87d71872f1a6f85f5ac699290c45a6e3c8933a ci: Add missing lint dependency (MarcoFalke) Pull request description: Also, document each dependency. Adding `gpg` avoids errors when running a release or dev branch in the CI: https://github.com/bitcoin/bitcoin/blob/01ec5308bf616740c804247043046b23b483df5c/ci/lint/06_script.sh#L30-L42 ``` bash: line 1: gpg: command not found ``` https://cirrus-ci.com/task/4582854860996608?logs=lint#L185 ACKs for top commit: hebasto: ACK fa87d71872f1a6f85f5ac699290c45a6e3c8933a Tree-SHA512: 869a3d2feab764b2c8d47d481359680a1d2c54a33b13ca26c5f8ce56cf2f368d4c74637dcbc53fdbf323f10940965c1c0e592e2fb4ce725d5cd467e77e62b6e5
2023-01-18ci: Bump --combinedlogslen to debug intermittent issuesMarcoFalke
2023-01-17ci: Add missing lint dependencyMarcoFalke
2023-01-17Merge bitcoin/bitcoin#26716: ci: Build python from source in "lint" taskMarcoFalke
123043e99cf3aab9eef7e381b133477b518ac4d0 ci: Bump lint task image to Ubuntu Jammy (Hennadii Stepanov) 9b86114058b3931c5b4ddc931a8a2cfdc8b085b6 ci: Use pyenv's `python-build` to install Python in lint task (Hennadii Stepanov) Pull request description: This PR: - is an alternative to bitcoin/bitcoin#26581 and bitcoin/bitcoin#26637 - closes bitcoin/bitcoin#26548 Key advantages of this PR over others: - it uses pyenv's `python-build` [standalone](https://github.com/pyenv/pyenv/tree/master/plugins/python-build#using-python-build-standalone) - requires no additional computational resources Note for testing. The lint task must success regardless of whether the `python_cache` is populated or invalidated. ACKs for top commit: MarcoFalke: ACK 123043e99cf3aab9eef7e381b133477b518ac4d0 fanquake: ACK 123043e99cf3aab9eef7e381b133477b518ac4d0 Tree-SHA512: ba0fcdd4f2939a59692b173dcd1f5704444cfcfbb8111538c6f8160056d0536bba250e4f9b0f8c66f8b454e52034bf36ffe6afae76cdc0f7cc5b58b576d790ba
2023-01-16ci: Bump lint task image to Ubuntu JammyHennadii Stepanov
2023-01-16ci: Use pyenv's `python-build` to install Python in lint taskHennadii Stepanov
2023-01-11ci: Stop and remove CI containerMarcoFalke
2023-01-11scripted-diff: ci: Rework docker namingMarcoFalke
DOCKER in names is confusingly used as synonym for "image", "container", and "ci". Fix the confusion by picking the term that fits the context. -BEGIN VERIFY SCRIPT- ren() { sed -i "s:$1:$2:g" $( git grep -l "$1" ) ; } ren DOCKER_PACKAGES CI_BASE_PACKAGES # This better reflects that they are the common base for all CI # containers. ren DOCKER_ID CI_CONTAINER_ID # This is according to the documentation of "--detach , -d: Run # container in background and print container ID". ren DOCKER_NAME_TAG CI_IMAGE_NAME_TAG # This avoids confusing with CONTAINER_NAME and clarifies that it is an # image. ren DOCKER_ADMIN CI_CONTAINER_CAP # This clarifies that it is a capability added to the container. ren DOCKER_CI_CMD_PREFIX CI_EXEC_CMD_PREFIX # This brings it in line with the CI_EXEC naming. -END VERIFY SCRIPT-
2023-01-09ci: Remove unused busybox workaroundMarcoFalke
The find workaround is no longer needed after commit d3d547c545021d8339db666d36e48f10ea478f9c
2023-01-09ci: Create named symbol for BINS_SCRATCH_DIRMarcoFalke
Also, create the dir a bit earlier, right after it was put in the PATH.
2023-01-07ci: Fix ci_native_fuzz_msan CONTAINER_NAMEMarcoFalke
2023-01-06build: Re-enable external signer on WindowsHennadii Stepanov
2023-01-04Merge bitcoin/bitcoin#26791: ci: Properly set COMMIT_RANGE in lint taskMarcoFalke
fa5cbf2290dab08296cabbd973e9adefc883f3fc ci: Properly set COMMIT_RANGE in lint task (MarcoFalke) Pull request description: Currently the variable holds (apart from the commits in the pull request) all commits to master since the pull was opened. This is problematic, because already merged commits are linted in unrelated pulls, leading to: * Wasted resources. For example, currently the lint task may take 9 minutes, when it should take 1. See https://cirrus-ci.com/task/6032782770569216?logs=lint#L1449 * False failures. For example, when a "wrong" commit is in master it can lead to some pulls failing unrelatedly, and others not. Now that the CI has the `/merge` commit (since commit fad7281d7842f337932cf44e703fdd631230ddd6), `COMMIT_RANGE` can simply be set to `HEAD~..HEAD` to only hold the changes in the pull. ACKs for top commit: fanquake: ACK fa5cbf2290dab08296cabbd973e9adefc883f3fc Tree-SHA512: e85fca4ca9d2615ddd2544403485e06885769a3f70bca297e23eefda2a1d28f47c5271f6adfa6ce0e5e972335c78098b76e0db4b109f59d0986bf508cef7528f
2023-01-03script: update python linter dependenciesJon Atack
2023-01-02ci: Properly set COMMIT_RANGE in lint taskMarcoFalke
2023-01-01Merge bitcoin/bitcoin#26775: ci: Revert tsan task changesMarcoFalke
fabb6af8508a9c0d05d2e9f0006b0b1d3f38119a ci: Remove duplicate CC and CXX from tsan task (MarcoFalke) fa5d9a0e246e94eb0d0e9a75f63ff1b09ec93c03 Revert "ci: Use clang-15 in tsan task" (MarcoFalke) faa835e7e564536fb0589ec94d3f85042a1f9cd8 Revert "test: Drop no longer needed `race:epoll_ctl` TSan suppression" (MarcoFalke) Pull request description: Looks like there are still bugs in clang-15, so we need to roll back all the way to the previously used version (clang-13). ACKs for top commit: hebasto: ACK fabb6af8508a9c0d05d2e9f0006b0b1d3f38119a, I have reviewed the code and it looks OK, I agree it can be merged. Tree-SHA512: d62203049847ab9095ee3fc89e18bdd721d1d9d5a7ef7a9f524c80e6be58d1d9f6aa2f14533df1ea77eb59597fba6fa9b987b17eb03b2c3f7cb577ab59cd59c0
2022-12-30ci: Remove duplicate CC and CXX from tsan taskMarcoFalke
2022-12-30Revert "ci: Use clang-15 in tsan task"MarcoFalke
This reverts commit faa00ca78e27a40cbafe701348e6b3674e3ddeed.
2022-12-29Merge bitcoin/bitcoin#26770: ci: Move `--enable-c++20` from "tidy" task back ↵MarcoFalke
to "ASan..." one afc60524305669b54ffa8857e3c053c6db6375f8 ci: Move `--enable-c++20` from "tidy" task back to "ASan..." one (Hennadii Stepanov) Pull request description: This PR reverts cc7335edc87c6ef34429b4df94f53973db520aac from https://github.com/bitcoin/bitcoin/pull/25528 partially. C++20 has introduced some new headers, and it is premature to consider them when using the IWYU tool. Required for https://github.com/bitcoin/bitcoin/pull/26763 and https://github.com/bitcoin/bitcoin/pull/26766. Related discussions: - https://github.com/bitcoin/bitcoin/pull/25528#discussion_r1058906785 - https://github.com/bitcoin/bitcoin/pull/26763#discussion_r1058860943 ACKs for top commit: MarcoFalke: review only ACK afc60524305669b54ffa8857e3c053c6db6375f8 Tree-SHA512: 9c1d3317d0f7a94d46d1e442da1a91669cd9ed1e62a41579edc96e1f5447551b536d32eeb222caf277e85228482753be545e0a874208cb24f9a8491fce3b6d9f
2022-12-29ci: Move `--enable-c++20` from "tidy" task back to "ASan..." oneHennadii Stepanov
This change reverts cc7335edc87c6ef34429b4df94f53973db520aac partially. C++20 has introduced some new headers, and it is premature to consider them when using the IWYU tool.
2022-12-29ci: Use clang-15 in tsan taskMarcoFalke
2022-12-24scripted-diff: Bump copyright headersHennadii Stepanov
-BEGIN VERIFY SCRIPT- ./contrib/devtools/copyright_header.py update ./ -END VERIFY SCRIPT- Commits of previous years: - 2021: f47dda2c58b5d8d623e0e7ff4e74bc352dfa83d7 - 2020: fa0074e2d82928016a43ca408717154a1c70a4db - 2019: aaaaad6ac95b402fe18d019d67897ced6b316ee0
2022-12-20ci: remove --prefix from msan jobfanquake
2022-12-10refactor: modernize the implementation of uint256.*pasta
- Constructors of uint256 to utilize Span instead of requiring a std::vector - converts m_data into a std::array - Prefers using `WIDTH` instead of `sizeof(m_data)` - make all the things constexpr - replace C style functions with c++ equivalents - memset -> std::fill - memcpy -> std::copy Note: In practice, implementations of std::copy avoid multiple assignments and use bulk copy functions such as std::memmove if the value type is TriviallyCopyable and the iterator types satisfy LegacyContiguousIterator. (https://en.cppreference.com/w/cpp/algorithm/copy) - memcmp -> std::memcmp
2022-12-10ci: Use `CONFIG_SITE` variable and `--prefix` option properlyHennadii Stepanov
This change fixes scripts when they are being run locally with a pre- existed `$DEPENDS_DIR/$HOST` directory.
2022-12-02Merge bitcoin/bitcoin#26592: ci: only run USDT interface tests on CirrusCIMarcoFalke
2811f40f3099a22091870d6aa2efecd741d186ce ci: only run USDT interface tests on CirrusCI (0xb10c) Pull request description: As mentioned in #26571, the task running the USDT interface tests fail when run in docker. cc7335edc87c6ef34429b4df94f53973db520aac in #25528 added that the tests are run in a **VM** in Cirrus CI. Running them locally in docker containers might not work: - We use [bcc] as tracing toolkit which requires the kernel headers to compile the BPF bytecode. As docker containers use the hosts kernel and don't run their own, there is a potential for mismatches between kernel headers available in the container and the host kernel. This results in a failure loading the BPF byte code. - Privilges are required to load the BPF byte code into the kernel. Normally, the docker containers aren't run with these. - We currently use an untrusted third-party PPA to install the bpfcc-tools package on Ubuntu 22.04. Using this on a local dev system could be a security risk. To not hinder the ASan + LSan + UBSan part of the CI task, the USDT tests are disabled on non-CirrusCI runs. [bcc]: https://github.com/iovisor/bcc Top commit has no ACKs. Tree-SHA512: 7c159b59630b36d5ed927b501fa0ad6f54ff3d98d0902f975cc8122b4147a7f828175807d40a470a85f0f3b6eeb79307d3465a287dbd2f3d75b803769ad0b6e7
2022-11-28ci: only run USDT interface tests on CirrusCI0xb10c
As mentioned in #26571, the task running the USDT interface tests fail when run in docker. cc7335edc87c6ef34429b4df94f53973db520aac in #25528 added that the tests are run in a **VM** in Cirrus CI. Running them locally in docker containers might not work: - We use [bcc] as tracing toolkit which requires the kernel headers to compile the BPF bytecode. As docker containers use the hosts kernel and don't run their own, there is a potential for mismatches between kernel headers available in the container and the host kernel. This results in a failure loading the BPF byte code. - Privilges are required to load the BPF byte code into the kernel. Normally, the docker containers aren't run with these. - We currently use an untrusted third-party PPA to install the bpfcc-tools package on Ubuntu 22.04. Using this on a local dev system could be a security risk. To not hinder the ASan + LSan + UBSan part of the CI task, the USDT tests are disabled on non-CirrusCI runs. [bcc]: https://github.com/iovisor/bcc
2022-11-28Merge bitcoin/bitcoin#26588: ci: Skip COMMIT_RANGE if no CIRRUS_PRMarcoFalke
fad1c55301b9f2d091d3b0d8a75ff522ce8dae5a lint: Skip COMMIT_RANGE if no CIRRUS_PR (MarcoFalke) Pull request description: It doesn't make sense to run this for non-PRs, because: * There are known whitespace "violations" in previous commits, so the lint may fail * Once the changes are merged, it is too late to fix them up (force pushes are illegal) * It isn't possible to determine which commits to run on if there is no reference branch (target branch of the pull request) Moreover, the test fails on non-master: * https://github.com/bitcoin/bitcoin/runs/8664441400 Fix all issues by skipping it. ACKs for top commit: hebasto: ACK fad1c55301b9f2d091d3b0d8a75ff522ce8dae5a, also tested in my personal Cirrus account. Tree-SHA512: be15f00e2b2a9069583833545883e0e5968a33d2455dad59e6fb47c1102b4dd16ef932e9ba945e29e9d941e6c17bd531a02c66b0491097801be6bda476875537
2022-11-28lint: Skip COMMIT_RANGE if no CIRRUS_PRMarcoFalke
2022-11-28Merge bitcoin/bitcoin#26574: ci: use ci_exec_root for clang installMarcoFalke
54dd8f51ce8815a93a6aceb36dd72bd449d6b327 ci: use ci_exec_root for clang install (josibake) Pull request description: fixes a bug introduced in #25900 ; see https://github.com/bitcoin/bitcoin/pull/25900#issuecomment-1327311069 the general idea of #25900 was to use a non-root user as much as possible to avoid modifying the user's local filesystem. however, it appears the root user is needed to correctly install clang. ACKs for top commit: hebasto: ACK 54dd8f51ce8815a93a6aceb36dd72bd449d6b327, tested on Ubuntu 22.04. Tree-SHA512: beb01d4b6127fbba3c8d18e85cf7ec7d1b2ec93ea05c475ab51bcaa04ef1b0591d886f1a7e0732c5ae86806013f022c0b44027380d2b0cfb1bfdc843e40f99b4
2022-11-25ci: use ci_exec_root for clang installjosibake
2022-11-23fuzz: Move-only net utilsMarcoFalke
2022-11-22Merge bitcoin/bitcoin#25900: ci: run docker wrapper with a non-root userMarcoFalke
849f20a6d3e437631a07469d3c4af5faa0aa06ed ci: create and use non-root user for docker image (josibake) Pull request description: Previously, everything in the ci docker image ran as the root user. This would lead to certain directories (`ci/scratch`, `depends`) being owned by `root` after running the ci locally which would lead to annoying behavior such as subsequent guix builds failing due to `depends/` being owned by root. This PR adds a non-root user in the container and chowns the mounted working directory. All the `docker exec` commands now run as the non-root user, except for the few that still need to run as root (mainly, installing packages). To test this I checked out a fresh copy of the repo, applied my changes, ran the CI, and verified all the local file permissions were unchanged after the CI was finished running. ACKs for top commit: hebasto: ACK 849f20a6d3e437631a07469d3c4af5faa0aa06ed, tested on Ubuntu 22.04 by running commands as follows: Tree-SHA512: 734dca0f36157fce5fab243b4ff657fc17ba980e8e4e4644305f41002ff21bd5cef02c306ea1e0b5c841d4c07c095e8e4be16722e6a38c890717c60a3f5ec62a
2022-11-22Merge bitcoin/bitcoin#26292: util: move threadinterrupt into util/fanquake
b89530483d39f6a6a777df590b87ba2fad8c8b60 util: move threadinterrupt into util (fanquake) Pull request description: Alongside thread and threadnames. It's part of libbitcoin_util. ACKs for top commit: ryanofsky: Code review ACK b89530483d39f6a6a777df590b87ba2fad8c8b60. No changes since last review other than rebase theuni: ACK b89530483d39f6a6a777df590b87ba2fad8c8b60. Tree-SHA512: 0421f4d1881ec295272446804b27d16bf63e6b62b272f8bb52bfecde9ae6605e8109ed16294690d3e3ce4b15cc5e7c4046f99442df73adb10bdf069d3fb165aa
2022-11-21ci: create and use non-root user for docker imagejosibake
Running all commands as the root user in the docker image will change local file permissions in the ci and depends directory. Add a non-root user to the container and use this user whenever possible when running docker exec commands.
2022-11-16util: Move error message formatting of NonFatalCheckError to cppMacroFake
This allows to strip down the header file
2022-11-01util: move threadinterrupt into utilfanquake
2022-10-31refactor: move url.h/cpp from lib util to lib commonfanquake
2022-10-27Merge bitcoin/bitcoin#26388: ci: Use `macos-ventura-xcode:14.1` image for ↵MacroFake
"macOS native" task da168934741b776bce07d5503ca2344d300723b3 ci: Use `macos-ventura-xcode:14.1` image for "macOS native" task (Hennadii Stepanov) 702836530ffa351e863b1b1300fd2e559a14ef23 ci: Make `getopt` path architecture agnostic (Hennadii Stepanov) Pull request description: The "macOS native" CI task always uses the recent OS image. This PR updates it up to the recent macOS release. Cirrus Labs [stopped](https://github.com/bitcoin/bitcoin/pull/25160#issuecomment-1162829773) updating macOS images for `x86_64`, therefore, an `arm64` image been used. Also `make test-security-check` has been dropped as it ["isn't even expected to pass"](https://github.com/bitcoin/bitcoin/issues/26386#issuecomment-1290318628) on `arm64` in CI. ACKs for top commit: Sjors: utACK da16893 Tree-SHA512: 36785d33b7f11b3cdbc53bcfbf97d88bf821fad248c825982dd9f8e3413809a4ef11190eaf950e60fdf479b62ff66920c35d9ea42d534723f015742eec7e19b6
2022-10-25ci: Use `macos-ventura-xcode:14.1` image for "macOS native" taskHennadii Stepanov
2022-10-25ci: Make `getopt` path architecture agnosticHennadii Stepanov
2022-10-18iwyu: Add missing includesMacroFake
2022-10-10Merge bitcoin/bitcoin#26254: iwyu: Add zmq source filesMacroFake
13afcc0cd4c2975852924d2d9be5e96096147716 iwyu: Add zmq source files (Hennadii Stepanov) Pull request description: ACKs for top commit: fanquake: ACK 13afcc0cd4c2975852924d2d9be5e96096147716 Tree-SHA512: 7af95e991fc2782aeba2edfef0a2f75f9c361058295586adb062087aa31c47cfcce2425aee9dd5153e18e018cf1f9272c9617c671b7262db55f241526c3fcb15
generated by cgit v1.2.3 (git 2.26.2) at 2024-10-12 23:31:30 +0000