aboutsummaryrefslogtreecommitdiff
path: root/contrib
AgeCommit message (Collapse)Author
2021-07-09net: change I2P seeds' ports to 0Vasil Dimov
2021-07-07guix: Test security-check sanity before performing themCarl Dong
2021-07-07scripts: adjust test-symbol-check for guix release environmentfanquake
Now that our release binaries are build in a glibc 2.24 and 2.27 environment, we can't use a symbol from glibc 2.28 to test our checks. Replace renameat2() with nextup(), which was introduced in 2.24. Note that this also means re-disabling the test for RISC-V, however RISC-V is built in a glibc 2.27 environment, and our minimum required glibc for that binary is 2.27.
2021-07-07scripts: more robustly test macOS symbol checksfanquake
2021-07-07build: Use and test PE binutils with --reloc-sectionCarl Dong
Also fix test-security-check.py to account for new PE PIE failure indication.
2021-07-07guix: Patch binutils to add security-related disable flagsCarl Dong
We use these flags in our test-security-check make target, but they are only available because debian patches them in. We can patch them in for our Guix builds so that we can check the sanity of our security/symbol checking suite before running them.
2021-07-07devtools: Improve *-check.py tool detectionCarl Dong
This is important to make sure that we're not testing tools different from the one we're building with. Introduce determine_wellknown_cmd, which encapsulates how we should handle well-known tools specification (IFS splitting, env override, etc.).
2021-07-06script: remove gitian reference from symbol-check.pyfanquake
2021-07-06guix: add additional documentation to patchesfanquake
2021-07-06guix: no-longer pass --enable-glibc-back-compat to Guixfanquake
Now that our Guix builds are performed on glibc 2.24 and 2.27 (RISCV), we no-longer need to pass the --enable-glibc-back-compat option. Replace it with --disable-threadlocal, to prevent the usage of symbols from glibc 2.18. None of the binaries produced required symbols later than 2.17, and 2.27 (RISCV).
2021-07-02guix: Also sort SHA256SUMS.partCarl Dong
2021-07-01guix: Build depends/qt with our platform definitionCarl Dong
Our 'bitcoin-linux-g++' definition better integrates with our depends system than the stock linux-g++-64 definition. This fixes a bug whereby Guix builds on x86_64 for x86_64 did not produce a QMinimalIntegrationPlugin and led to bitcoin-qt not being built.
2021-07-01guix: Rebase toolchain on glibc 2.24 (2.27 for riscv64)Carl Dong
Support for riscv64 in glibc landed in 2.27 so it's unavoidable that we use 2.27. Running a Bitcoin build with toolchains based on 2.24 for platforms other than riscv64 seem to produce binaries which do not have 2.17 symbols. So use 2.24 since it's more recent and maintained by Debian Stretch.
2021-06-24Merge bitcoin/bitcoin#22257: test: refactor: various (de)serialization ↡MarcoFalke
helpers cleanups/improvements bdb8b9a347e68f80a2e8d44ce5590a2e8214b6bb test: doc: improve doc for `from_hex` helper (mention `to_hex` alternative) (Sebastian Falbesoner) 191405420815d49ab50184513717a303fc2744d6 scripted-diff: test: rename `FromHex` to `from_hex` (Sebastian Falbesoner) a79396fe5f8f81c78cf84117a87074c6ff6c9d95 test: remove `ToHex` helper, use .serialize().hex() instead (Sebastian Falbesoner) 2ce7b47958c4a10ba20dc86c011d71cda4b070a5 test: introduce `tx_from_hex` helper for tx deserialization (Sebastian Falbesoner) Pull request description: There are still many functional tests that perform conversions from a hex-string to a message object (deserialization) manually. This PR identifies all those instances and replaces them with a newly introduced helper `tx_from_hex`. Instances were found via * `git grep "deserialize.*BytesIO"` and some of them manually, when it were not one-liners. Further, the helper `ToHex` was removed and simply replaced by `.serialize().hex()`, since now both variants are in use (sometimes even within the same test) and using the helper doesn't really have an advantage in readability. (see discussion https://github.com/bitcoin/bitcoin/pull/22257#discussion_r652404782) ACKs for top commit: MarcoFalke: review re-ACK bdb8b9a347e68f80a2e8d44ce5590a2e8214b6bb 😁 Tree-SHA512: e25d7dc85918de1d6755a5cea65471b07a743204c20ad1c2f71ff07ef48cc1b9ad3fe5f515c1efaba2b2e3d89384e7980380c5d81895f9826e2046808cd3266e
2021-06-21scripted-diff: test: rename `FromHex` to `from_hex`Sebastian Falbesoner
-BEGIN VERIFY SCRIPT- sed -i 's/\<FromHex\>/from_hex/g' $(git grep -l FromHex) -END VERIFY SCRIPT- Co-authored-by: MarcoFalke <falke.marco@gmail.com>
2021-06-21test: remove `ToHex` helper, use .serialize().hex() insteadSebastian Falbesoner
2021-06-21test: introduce `tx_from_hex` helper for tx deserializationSebastian Falbesoner
`FromHex` is mostly used for transactions, so we introduce a shortcut `tx_from_hex` for `FromHex(CTransaction, hex_str)`.
2021-06-21Merge bitcoin/bitcoin#22244: devtools: Correctly extract symbol versions in ↡W. J. van der Laan
symbol-check e8cd3700eeb27437f5ea435869c9d61214285fdd devtools: Integrate ARCH_MIN_GLIBC_VER table into MAX_VERSIONS in symbol-check.py (W. J. van der Laan) a33381acf5ae2b43616fffaf26b1c8962e8ef0bb devtools: Add xkb version to symbol-check (W. J. van der Laan) 19e598bab0a1cb5ad93321eb9fa25d1a58d5e276 devtools: Fix verneed section parsing in pixie (W. J. van der Laan) Pull request description: I misunderstood the ELF specification for version symbols (verneed): The `vn_aux` pointer is relative to the main verneed record, not the start of the section. This caused many symbols to not be versioned properly in the return value of `elf.dyn_symbols`. This was discovered in #21454. Fix it by correcting the offset computation. - xkb versions symbols (using the prefix `V`), as this library is used by bitcoin-qt, add it to the valid versions in `symbol-check.py` This unfortunately brings to light some symbols that have been introduced since and weren't caught (from a gitian compile of master): ``` bitcoin-cli: symbol getrandom from unsupported version GLIBC_2.25 bitcoin-cli: failed IMPORTED_SYMBOLS bitcoind: symbol getrandom from unsupported version GLIBC_2.25 bitcoind: symbol log from unsupported version GLIBC_2.29 bitcoind: symbol fcntl64 from unsupported version GLIBC_2.28 bitcoind: symbol pow from unsupported version GLIBC_2.29 bitcoind: symbol exp from unsupported version GLIBC_2.29 bitcoind: failed IMPORTED_SYMBOLS bitcoin-qt: symbol exp from unsupported version GLIBC_2.29 bitcoin-qt: symbol fcntl64 from unsupported version GLIBC_2.28 bitcoin-qt: symbol log from unsupported version GLIBC_2.29 bitcoin-qt: symbol pow from unsupported version GLIBC_2.29 bitcoin-qt: symbol statx from unsupported version GLIBC_2.28 bitcoin-qt: symbol getrandom from unsupported version GLIBC_2.25 bitcoin-qt: symbol renameat2 from unsupported version GLIBC_2.28 bitcoin-qt: symbol getentropy from unsupported version GLIBC_2.25 bitcoin-qt: failed IMPORTED_SYMBOLS bitcoin-wallet: symbol exp from unsupported version GLIBC_2.29 bitcoin-wallet: symbol log from unsupported version GLIBC_2.29 bitcoin-wallet: symbol fcntl64 from unsupported version GLIBC_2.28 bitcoin-wallet: failed IMPORTED_SYMBOLS test_bitcoin: symbol getrandom from unsupported version GLIBC_2.25 test_bitcoin: symbol log from unsupported version GLIBC_2.29 test_bitcoin: symbol fcntl64 from unsupported version GLIBC_2.28 test_bitcoin: symbol pow from unsupported version GLIBC_2.29 test_bitcoin: symbol exp from unsupported version GLIBC_2.29 test_bitcoin: failed IMPORTED_SYMBOLS ``` ACKs for top commit: hebasto: ACK e8cd3700eeb27437f5ea435869c9d61214285fdd Tree-SHA512: 8c15e3478eb642f01a1ddaadef03f80583f088f9fa8e3bf171ce16b0ec05ffb4675ec147d7ffc6a4360637ed47fca517c6ca2bac7bb30d794c03783cfb964b79
2021-06-18Merge bitcoin/bitcoin#20923: signet miner followupsW. J. van der Laan
b3c712cb2871e0034898b37454fb6a4e1fc78f84 contrib/signet/miner: remove debug code (Anthony Towns) 297e35159f99d4caa995de29d419aefa4b87630c bitcoin-util: use AddCommand / GetCommand (Anthony Towns) b6d493fd4d602a67d16d2566466e18091e272fa0 contrib/signet/README.md: Update miner description (Anthony Towns) e66543827cd4ae9b76ff4c5c6e414c1942c7d727 contrib/signet/miner: Automatic timestamp for first block (Anthony Towns) a383ce5b4add6cdf4e9974527b609a1147a0d972 contrib/signet/miner: --grind-cmd is required for calibrate (Anthony Towns) 1a45cd2e51214f81442a5f9bf16612100adaf5b8 contrib/signet: Fix typos (Anthony Towns) Pull request description: Followups from #19937 ACKs for top commit: laanwj: Code review ACK b3c712cb2871e0034898b37454fb6a4e1fc78f84 Tree-SHA512: a1003f9ee3697438114b60872b50f4300c8b52f0d58551566eb61c421d787525807ae75be205dcab2c24358cd568f53260120880109a9d728773405ff987596f
2021-06-18devtools: Integrate ARCH_MIN_GLIBC_VER table into MAX_VERSIONS in ↡W. J. van der Laan
symbol-check.py The (ancient) versions specified here were deceptive. Entries older than MAX_VERSIONS['GLIBC'], which is 2.17, are ignored here. So reorganize the code to avoid confusion for other people reading this code.
2021-06-18Merge bitcoin/bitcoin#21871: scripts: add checks for minimum required OS ↡fanquake
versions aa80b5759dfa613780a99801641519dd78bb3eca scripts: check macOS SDK version is set (fanquake) c972345bacd0cb01371b3f00941e81dce16278e1 scripts: check minimum required Windows version is set (fanquake) 29615aef52d7f1a29a87a29dfe4d39bf0e9867f3 scripts: check minimum required macOS vesion is set (fanquake) 8732f7b6c92f9dcf37f3ab618e9daab0c52fc781 scripts: LIEF 0.11.5 (fanquake) Pull request description: macOS: We use a compile flag ([-mmacosx-version-min=10.14](https://github.com/bitcoin/bitcoin/blob/master/depends/hosts/darwin.mk#L96)) to set the minimum required version of macOS needed to run our binaries. This adds a sanity check that the version is being set as expected. Clangs Darwin driver should infer the SDK version used during compilation, and forward that through to the linker. Add a check that this has been done, and the expected SDK version is set. Should help prevent issues like #21771 in future. Windows: We use linker flags ([-Wl,--major/minor-subsystem-version](https://github.com/bitcoin/bitcoin/blob/master/configure.ac#L683)) to set the minimum required version of Windows needed to run our binaries. This adds a sanity check that the version is being set as expected. Gitian builds: ```bash # macOS: 8b6fcd61d75001c37b2af3fceb5ae09f5d2fe85e97d361f684214bd91c27954a bitcoin-f015e1c2cac9-osx-unsigned.dmg 3c1e412bc7f5a7a5d0f78e2cd84b7096831414e1304c1307211aa3e135d89bbf bitcoin-f015e1c2cac9-osx-unsigned.tar.gz 50b7b2804e8481f63c69c78e3e8a71c0d811bf2db8895dd6d3edae9c46a738ae bitcoin-f015e1c2cac9-osx64.tar.gz fe6b5c0a550096b76b6727efee30e85b60163a41c83f21868c849fdd9876b675 src/bitcoin-f015e1c2cac9.tar.gz 8a20f21b20673dfc8c23e22b20ae0839bcaf65bf0e02f62381cdf5e7922936f0 bitcoin-core-osx-22-res.yml # Windows: b01fcdc2a5673387050d6c6c4f96f1d350976a121155fde3f76c2af309111f9d bitcoin-f015e1c2cac9-win-unsigned.tar.gz b95bdcbef638804030671d2332d58011f8c4ed4c1db87d6ffd211515c32c9d02 bitcoin-f015e1c2cac9-win64-debug.zip 350bf180252d24a3d40f05e22398fec7bb00e06d812204eb5a421100a8e10638 bitcoin-f015e1c2cac9-win64-setup-unsigned.exe 2730ddabe246d99913c9a779e97edcadb2d55309933d46f1dffd0d23ecf9aae5 bitcoin-f015e1c2cac9-win64.zip fe6b5c0a550096b76b6727efee30e85b60163a41c83f21868c849fdd9876b675 src/bitcoin-f015e1c2cac9.tar.gz aa60d7a753e8cb2d4323cfbbf4d964ad3645e74c918cccd66862888f8646d80f bitcoin-core-win-22-res.yml ``` ACKs for top commit: hebasto: ACK aa80b5759dfa613780a99801641519dd78bb3eca, tested by breaking tests: Tree-SHA512: 10150219910e8131715fbfe20edaa15778387616ef3bfe1a5152c7acd3958fe8f88c74961c3d3641074eb72824680c22764bb1dc01a19e92e946c2d4962a8d2c
2021-06-17Merge bitcoin/bitcoin#22182: guix: Overhaul how guix-{attest,verify} works ↡fanquake
and hierarchy e2c40a4ed5272d72fea997bd936fba28bb753226 guix-attest: Error out if SHA256SUMS is unexpected (Carl Dong) 4cc35daed557f38b080360a89036b2e97a6f78c2 Rewrite guix-{attest,verify} for new hier (Carl Dong) 28a9c9b83924f585b397f0f3b8e9e73780ac0ad6 Make SHA256SUMS fragment right after build (Carl Dong) Pull request description: Based on: #22075 Code reviewers: I recommend reading the new `guix-{attest,verify}` files instead of trying to read the diff The following changes resolve many usability improvements which were pointed out to me: 1. Some maintainers like to extract their "uncodesigned tarball" inside the `output/` directory, resulting in the older `guix-attest` mistakenly attesting to the extracted contents 2. Maintainers whose GPG keys reside on an external smartcard often need to physically interact with the smartcard as a way to approve the signing operation, having one signature per platform means a lot of fidgeting 3. Maintainers wishing to sign on a separate machine now has the option of transferring only a subtree of `output/`, namely `output/*/SHA256SUMS.part`, in order to perform a signature (you may need to specify an `$OUTDIR_BASE` env var) 4. An `all.SHA256SUMS` file should be usable as the base `SHA256SUMS` in bitcoin core torrents and on the release server. For those who sign on an separate machine than the one you do builds on, the following steps will work: 1. `env GUIX_SIGS_REPO=/home/achow101/guix.sigs SIGNER=achow101 NO_SIGN=1 ./contrib/guix/guix-attest` 2. Copy `/home/achow101/guix.sigs/<tag>/achow101` (which does not yet have signatures) to signing machine 3. Sign the `SHA256SUMS` files: ```bash for i in "<path-to-achow101>/*.SHA256SUMS"; do gpg --detach-sign --local-user "<your-key-here>" --armor --output "$i"{.asc,} done ``` 5. Upload `<path-to-achow101>` (now with signatures) to `guix.sigs` ----- After this change, output directories will now include a `SHA256SUMS.part` fragment, created immediately after a successful build: ``` output └── x86_64-w64-mingw32 β”œβ”€β”€ bitcoin-4e069f7589da-win64-debug.zip β”œβ”€β”€ bitcoin-4e069f7589da-win64-setup-unsigned.exe β”œβ”€β”€ bitcoin-4e069f7589da-win64.zip β”œβ”€β”€ bitcoin-4e069f7589da-win-unsigned.tar.gz └── SHA256SUMS.part ``` These `SHA256SUMS.part` fragments look something like: ``` 3ebd7262b1a0a5bb757fef1f70e7e14033c70f98c059bc4dbfee5d1992b25825 dist-archive/bitcoin-4e069f7589da.tar.gz def2e7d3de5ab3e3f955344e75151df4f33713f9101f5295bd13c9375bdf633b x86_64-w64-mingw32/bitcoin-4e069f7589da-win64-debug.zip 643049fe3ee4a4e83a1739607e67b11b7c9b1a66208a6f35a9ff634ba795500e x86_64-w64-mingw32/bitcoin-4e069f7589da-win64-setup-unsigned.exe a247a1ccec0ccc2e138c648284bd01f6a761f2d8d6d07d91b5b4a6670ec3f288 x86_64-w64-mingw32/bitcoin-4e069f7589da-win-unsigned.tar.gz fab76a836dcc592e39c04fd2396696633fb6eb56e39ecbf6c909bd173ed4280c x86_64-w64-mingw32/bitcoin-4e069f7589da-win64.zip ``` Meaning that they are valid `SHA256SUMS` files when `sha256sum --check`'d at the `guix-build-*/output` directory level When `guix-attest` is invoked, these `SHA256SUMS.part` files are combined and sorted (by `-k2`, `LC_ALL=C`) to create: 1. `noncodesigned.SHA256SUMS` for a manifest of all non-codesigned outputs, and 3. `all.SHA256SUMS` for a manifest of all outputs including non-codesigned outputs Then both files are signed, resulting in the following `guix.sigs` hierarchy: ``` 4e069f7589da/ └── dongcarl β”œβ”€β”€ all.SHA256SUMS β”œβ”€β”€ all.SHA256SUMS.asc β”œβ”€β”€ noncodesigned.SHA256SUMS └── noncodesigned.SHA256SUMS.asc ``` ACKs for top commit: achow101: ACK e2c40a4ed5272d72fea997bd936fba28bb753226 hebasto: ACK e2c40a4ed5272d72fea997bd936fba28bb753226, tested on Linux Mint 20.1 (x86_64) with and w/o `NO_SIGN=1`. Changes in `contrib/guix/libexec/codesign.sh` and `contrib/guix/guix-verify` are reviewed only. Tree-SHA512: 618aacefb0eb6595735a9ab6a98ea6598fce65f9ccf33fa1e7ef93bf140c0f6cfc16e34870c6aa3e4777dd3f004b92a82a994141879870141742df948ec59c1f
2021-06-14guix-attest: Error out if SHA256SUMS is unexpectedCarl Dong
2021-06-14devtools: Add xkb version to symbol-checkW. J. van der Laan
xkb versions symbols (using the prefix `V`), as this library is used by bitcoin-qt, add it to the valid versions in `symbol-check.py`.
2021-06-14devtools: Fix verneed section parsing in pixieW. J. van der Laan
I misunderstood the ELF specification for version symbols (verneed): The `vn_aux` pointer is relative to the main verneed record, not the start of the section. This caused many symbols to not be versioned properly in the return value of `elf.dyn_symbols`. This was discovered in #21454. Fix it by correcting the offset computation.
2021-06-11Merge bitcoin/bitcoin#22190: Use latest signapple commitfanquake
683d197970a533690ca1bd4d06d021900e87cb8b Use latest signapple commit (Andrew Chow) Pull request description: Update gitian and guix to use the same latest signapple commit. Also changed guix to use the actual repo. The changes from the fork were incorporated upstream. ACKs for top commit: fanquake: ACK 683d197970a533690ca1bd4d06d021900e87cb8b - sanity checked that the updated package is built: Tree-SHA512: a4981f8bbe33e6c5654632bc9b9f6f2f1e675741a19ac7296205e370f1e64a747101ecb632e0cc82a0134e4c2e9ce47b3f7b4d8c8f75f0f06dd069c078303759
2021-06-10scripts: check macOS SDK version is setfanquake
Clangs Darwin driver should infer the SDK version used during compilation, and forward that through to the linker. Add a check that this has been done, and the expected SDK version is set. Should help prevent issues like #21771 in future.
2021-06-10scripts: check minimum required Windows version is setfanquake
We use linker flags (-Wl,--major/minor-subsystem-version) to set the minimum required version of Windows needed to run our binaries. This adds a sanity check that the version is being set as expected.
2021-06-10scripts: check minimum required macOS vesion is setfanquake
We use a compile flag (-mmacosx-version-min) to set the minimum required version of macOS needed to run our binaries. This adds a sanity check that the version is being set as expected.
2021-06-10scripts: LIEF 0.11.5fanquake
2021-06-09Rewrite guix-{attest,verify} for new hierCarl Dong
2021-06-09Make SHA256SUMS fragment right after buildCarl Dong
2021-06-09Merge bitcoin/bitcoin#22075: guix: Misc leftover usability improvementsfanquake
108a6be92adc1e80839d90b552e72b8142140f6c guix: Check for disk space availability before building (Carl Dong) d7dec89091ee4a456ff64ad7ce675ae6813668f1 guix: Remove dest if OUTDIR mv fails (Carl Dong) Pull request description: There seems to be some corner cases that can be hit when guix scripts unexpectedly fail in the middle of operation, see: https://gnusha.org/bitcoin-builds/2021-05-24.log - Perform an early disk space check for `guix-build` - Overwrite existing output directory after a successful build (the existing one might be malformed), and cleanup output directory if the `mv` somehow fails ACKs for top commit: laanwj: Tested ACK 108a6be92adc1e80839d90b552e72b8142140f6c achow101: ACK 108a6be92adc1e80839d90b552e72b8142140f6c Tree-SHA512: cf6438317da40bf55714cd2d8cce859b3d435cc66cabefe8d4a53552d7880966acfe84ffe8fadf1c80e368ae6b037992258a6d409df85ffc6ce8bf780e98e2e5
2021-06-08Use latest signapple commitAndrew Chow
Update gitian and guix to use the same latest signapple commit
2021-06-03Merge bitcoin/bitcoin#22050: p2p: remove tor v2 supportW. J. van der Laan
5d82a57db4f67506a4e80d186ba76f3a8665e147 contrib: remove torv2 seed nodes (Jon Atack) 5f7e086dac78c9070f8292a1757d7e77e110f772 contrib: update generate-seeds.py to ignore torv2 addresses (Jon Atack) 8be56f0f8ecc54744d572e5678a3089665587b98 p2p, refactor: extract OnionToString() from CNetAddr::ToStringIp() (Jon Atack) 5f9d3c09b4c9cd026cdc7c3a81f91632280917b7 p2p: remove torv2 from CNetAddr::ToStringIP() (Jon Atack) 3d390421440f1cae9a9f2b089561c183ecd1b073 p2p: remove torv2 in SetIP() and ADDR_TORV2_SIZE constant (Jon Atack) cff5ec477a388ae9aa9fd9ef6a7dad1f678e7d23 p2p: remove pre-addrv2 onions from SerializeV1Array() (Jon Atack) 4192a74413907717d6173e393724b931f2225dd9 p2p: ignore torv2-in-ipv6 addresses in SetLegacyIPv6() (Jon Atack) 1d631e956fffbbc7891ed40be4fd39aeff036c52 p2p: remove BIP155Network::TORV2 from GetBIP155Network() (Jon Atack) 7d1769bc450a98c093a066d6daed84337040dbfb p2p: remove torv2 from SetNetFromBIP155Network() (Jon Atack) eba9a94b9f56be2fda623e77f19b960425ea1eb5 fuzz: rename CNetAddr/CService deserialize targets (Jon Atack) c56a1c9b182815018b8bd3d8e6b8c2cb27859607 p2p: drop onions from IsAddrV1Compatible(), no longer relay torv2 (Jon Atack) f8e94002fcfdc7890d38c23488b1f3a662e97bc4 p2p: remove torv2/ADDR_TORV2_SIZE from SetTor() (Jon Atack) 0f1c58ae87d6a3fe81816500e7b8275420d151d0 test: update feature_proxy to torv3 (Jon Atack) Pull request description: ![image](https://user-images.githubusercontent.com/2415484/120018909-4d425a00-bfd7-11eb-83c9-95a3dac97926.jpeg) This patch removes support in Bitcoin Core for Tor v2 onions, which are already removed from the release of Tor 0.4.6. - no longer serialize/deserialize and relay Tor v2 addresses - ignore incoming Tor v2 addresses - remove Tor v2 addresses from the addrman and peers.dat on node launch - update generate-seeds.py to ignore Tor v2 addresses - remove Tor v2 hard-coded seeds Tested with tor-0.4.6.1-alpha (no v2 support) and 0.4.5.7 (v2 support). With the latest Tor (no v2 support), this removes all the warnings like those reported with current master in https://github.com/bitcoin/bitcoin/issues/21351 ``` <bitcoind debug log> Socks5() connect to […].onion:8333 failed: general failure <tor log> Invalid hostname [scrubbed]; rejecting ``` and the addrman no longer has Tor v2 addresses on launching bitcoind. ```rake $ ./src/bitcoin-cli -addrinfo { "addresses_known": { "ipv4": 44483, "ipv6": 8467, "torv2": 0, "torv3": 2296, "i2p": 6, "total": 55252 } } ``` After recompiling back to current master and restarting with either of the two Tor versions (0.4.5.7 or 0.4.6.1), -addrinfo initially returns 0 Tor v2 addresses and then begins finding them again. Ran nodes on this patch over the past week on mainnet/testnet/signet/regtest after building with DEBUG_ADDRMAN. Verified that this patch bootstraps an onlynet=onion node from the Tor v3 hardcoded fixed seeds on mainnet and testnet and connects to blocks and v3 onion peers: `rm ~/.bitcoin/testnet3/peers.dat ; ./src/bitcoind -testnet -dnsseed=0 -onlynet=onion` ![Screenshot from 2021-05-28 00-26-17](https://user-images.githubusercontent.com/2415484/119905021-ea02ea00-bf3a-11eb-875f-27ef57640c49.png) Tested using `addnode`, `getaddednodeinfo`,`addpeeraddress`, `disconnectnode` and `-addrinfo` that a currently valid, connectable Tor v2 peer can no longer be added: ![Screenshot from 2021-05-30 11-32-05](https://user-images.githubusercontent.com/2415484/120099282-29435d80-c12a-11eb-81b6-5084244d7d2a.png) Thanks to Vasil Dimov, Carl Dong, and Wladimir J. van der Laan for their work on BIP155 and Tor v3 that got us here. ACKs for top commit: laanwj: Code review ACK 5d82a57db4f67506a4e80d186ba76f3a8665e147 Tree-SHA512: 590ff3d2f6ef682608596facb4b01f44fef69716d2ab3552ae1655aa225f4bf104f9ee08d6769abb9982a8031de93340df553279ce1f5023771f9f2b651178bb
2021-06-03Merge bitcoin/bitcoin#21654: build, qt: Make Qt rcc output always deterministicfanquake
a58868d201cb6d263aa552815f7f86562c1ca9a5 build: Makes rcc output always deterministic (Hennadii Stepanov) Pull request description: The Qt Resource Compiler ([rcc](https://doc.qt.io/qt-5/rcc.html)) has a command-line option `--format-version` which has the [default value](https://code.qt.io/cgit/qt/qtbase.git/tree/src/tools/rcc/main.cpp?h=5.12.10#n172) 2. The only difference from `--format-version 1` is adding a [last modified timestamp](https://code.qt.io/cgit/qt/qtbase.git/tree/src/tools/rcc/rcc.cpp?h=5.12.10#n207) to the output file ([credits](https://github.com/bitcoin/bitcoin/pull/21654#issuecomment-819198228) to **fanquake**). That, in turn, forces us to use `QT_RCC_SOURCE_DATE_OVERRIDE=1` to get deterministic builds (#13732). This change makes rcc output always deterministic by using `--format-version 1` option that makes usage of the `QT_RCC_SOURCE_DATE_OVERRIDE` needless. --- Also it improves interaction with ccache: On master (f6c44e999b7d1d9a0de5d678ac8f1679aa271f65): ``` $ make && make clean && ccache --zero-stats && make && ccache --show-stats ... cache directory /home/hebasto/.ccache primary config /home/hebasto/.ccache/ccache.conf secondary config (readonly) /etc/ccache.conf stats updated Sun Apr 11 15:45:43 2021 stats zeroed Sun Apr 11 15:45:05 2021 cache hit (direct) 638 cache hit (preprocessed) 0 cache miss 1 cache hit rate 99.84 % called for link 10 cleanups performed 0 files in cache 20023 cache size 13.2 GB max cache size 15.0 GB ``` The missed file is always `qt/libbitcoinqt_a-qrc_bitcoin_locale.o`. With this PR: ``` $ make && make clean && ccache --zero-stats && make && ccache --show-stats ... cache directory /home/hebasto/.ccache primary config /home/hebasto/.ccache/ccache.conf secondary config (readonly) /etc/ccache.conf stats updated Sun Apr 11 15:28:46 2021 stats zeroed Sun Apr 11 15:28:21 2021 cache hit (direct) 639 cache hit (preprocessed) 0 cache miss 0 cache hit rate 100.00 % called for link 10 cleanups performed 0 files in cache 20012 cache size 13.2 GB max cache size 15.0 GB ``` ACKs for top commit: fanquake: ACK a58868d201cb6d263aa552815f7f86562c1ca9a5 Tree-SHA512: 52f4a3267f41883d13025c0de79b6da22e92d60c729e01b986935c6812bbfe7fadc40b742bd715bfdf09df94af6838d4fbbe8208c6123f366108e38c8e1121c5
2021-06-03contrib: remove torv2 seed nodesJon Atack
2021-06-03contrib: update generate-seeds.py to ignore torv2 addressesJon Atack
2021-05-31doc: Fix external links (IRC, ...)MarcoFalke
2021-05-27Merge bitcoin/bitcoin#22017: Update Windows code signing certificateW. J. van der Laan
167fb1fc72e309587a8ef1d7844cb51a5483f54f Update Windows code signing certificate (Andrew Chow) Pull request description: Updates the Windows code signing certificate to a new one issued by Digicert. This certificate has been issued to Bitcoin Core Code Signing LLC registered in Delaware, US. Note that this is different from the previous Bitcoin Core Code Signing Association registered in Zurich, Switzerland as it was unable to meet the validation requirements in time. ACKs for top commit: Sjors: utACK 167fb1f laanwj: ACK 167fb1fc72e309587a8ef1d7844cb51a5483f54f Tree-SHA512: 8d5308c710ef94330417955b9bc82c5894d282798cebece82b84b425e3354e566aa6a68693ec359391ea40ddd7e2032d35ce28d104683d75ec3010ddf00be209
2021-05-27Merge bitcoin/bitcoin#22060: contrib: add torv3 seed nodes for testnet, drop ↡W. J. van der Laan
v2 ones 6fe05168587ee818e6ae4483a5984a5893131e44 contrib: add torv3 seed nodes for testnet, drop v2 ones (Jon Atack) Pull request description: Replace the ancient (2015) Tor V2 hardcoded seeds with new Tor V3 ones. This needs to be done before 0.22 to make sure onion-only testnet nodes can still connect to the network. Continues #21560. Ways to test: - Re-generate ` src/chainparamsseeds.h` with `cd contrib/seeds && python3 generate-seeds.py . > ../../src/chainparamsseeds.h`, check if git tree stays the same. - Create a new testnet node with `bitcoind -testnet -onlynet=onion -proxy=127.0.0.1:9050` (or delete `~/.bitcoin/testnet3/peers.dat`), check if it is able to connect to the network and get blocks. - Check if the addresses are connectable for ex.: ```python3 #!/usr/bin/env python3 import subprocess with open('contrib/seeds/nodes_test.txt') as f: for line in (line for line in (line.rstrip().split('#', 1)[0] for line in f) if line): subprocess.call(["nc", "-v", "-x", "127.0.0.1:9050", "-z"] + line.split(':')) ``` Thanks to jonatack for providing the list. ACKs for top commit: jonatack: ACK 6fe05168587ee818e6ae4483a5984a5893131e44 Tree-SHA512: 61bfdb44dfab9d02b75e5cb06c089a3b1a1fe7134875e1d09166c4116e961d809aa25422fe03f068876e9423b571ecc4a0c7a7eeacba4aac3b2768717f3ee6d6
2021-05-25guix: Check for disk space availability before buildingCarl Dong
2021-05-25guix: Remove dest if OUTDIR mv failsCarl Dong
2021-05-25contrib: add torv3 seed nodes for testnet, drop v2 onesJon Atack
2021-05-24Merge bitcoin/bitcoin#21239: guix: Add codesignature attachment support for ↡W. J. van der Laan
osx+win ee883201cf134952284632e9e9ae72bf1c8c792f guix: repro: Sort find output in libtool for gcc-8 (Carl Dong) ee0a67c32a8861eab650bf8894af06807578eba0 codesigning: Use SHA256 as digest for osslsigncode (Windows) (Carl Dong) 38eb91eb0616ed6dbe34c23e11588d130fef07f8 guix: Add codesigning functionality (Carl Dong) bac2690e6f683fcedb883fe1d32f3c33c628a141 guix: Package codesigning tools (Carl Dong) 0a2176d47767972e4cd5ed302c1dbeedece1708b guix: Reindent existing manifest.scm (Carl Dong) c090a3e9238ba2df07875b4708e908d8dca4ed9b Makefile.am: use APP_DIST_DIR instead of hard-coding dist (Carl Dong) Pull request description: This is the last PR before we reach feature-parity with the Gitian process! Note: I tried using the `Makefile` inside the distsrc to make the dmg instead of manually listing out the commands, but `make` seems to want to re-make a lot of other files which broke the dmg. The workflow looks something like this: 1. `env [ FOO=bar... ] ./contrib/guix/guix-build` (add additional env vars as necessary) 2. Codesigners only: 1. Copy `guix-build-<short-id>/output/x86_64-apple-darwin18/bitcoin-<short-id>-osx-unsigned.tar.gz` and `guix-build-<short-id>/output/x86_64-w64-mingw32/bitcoin-<short-id>-win-unsigned.tar.gz` to signing computer 2. Codesign with `./detached-sig-create.sh` inside the tarball 3. Upload contents of `signature-{osx,win}.tar.gz` to https://github.com/bitcoin-core/bitcoin-detached-sigs (as a new tag) 3. Checkout new tag for `bitcoin-core/bitcoin-detached-sigs` with the detached signatures 4. `env [ FOO=bar... ] DETACHED_SIGS_REPO=<path/to/bitcoin-detached-sigs> ./contrib/guix/guix-codesign` (modify env vars as necessary) 5. Make sure `guix.sigs` is cloned and updated 6. `env GUIX_SIGS_REPO=<path/to/guix.sigs> SIGNER=0x96AB007F1A7ED999=dongcarl ./contrib/guix/guix-attest` (modify env vars as necessary) 7. Commit your new signatures and SHA256SUMS in `guix.sigs` 8. Optionally, after there are multiple signatures in `guix.sigs`: `env GUIX_SIGS_REPO=<path/to/guix.sigs> ./contrib/guix/guix-verify` ACKs for top commit: laanwj: Tested ACK ee883201cf134952284632e9e9ae72bf1c8c792f achow101: ACK ee883201cf134952284632e9e9ae72bf1c8c792f Tree-SHA512: e812a07a5f19f900600c70cb9c717769ef544a6c0c12760b5558b76b6b37df863257f3dbf38b0757e6e06e334470267e94c9f2bdbc27409d6837b1a0bfc6acbc
2021-05-21Update Windows code signing certificateAndrew Chow
2021-05-19guix: repro: Sort find output in libtool for gcc-8Carl Dong
Otherwise the resulting .a static libraries (e.g. libstdc++.a) will not be reproducible and end up making the Bitcoin binaries non-reproducible as well. See: https://reproducible-builds.org/docs/archives/#gnu-libtool
2021-05-14Merge bitcoin/bitcoin#21889: macho: check for control flow instrumentationW. J. van der Laan
42b589d18fed5e2b3cb6ac9937e3333619967a6c scripts: test for MACHO control flow instrumentation (fanquake) 469a5bc4fa74d70556cce454efbc38fb7945acd8 build: build Boost with -fcf-protection when targeting Darwin (fanquake) Pull request description: Addresses the macOS portion of #21888. Build Boost with `-fcf-protection` when targeting Darwin. This should be ok, because our cross-compiler (Clang 10) supports the option, and I'd expect all versions of Apple Clang being used to compile Core would also support it. Building Boost with this option is required so that the `main` provided to `test_bitcoin` has instrumentation. Note that the presence of instrumentation does not mean it will be used, as that is determined at runtime by the CPU. From the Intel control flow enforcement documentation: > The ENDBR32 and ENDBR64 instructions will have the same effect as the NOP instruction on Intel 64 processors that do not support CET. On processors supporting CET, these instructions do not change register or flag state. This allows CET instrumented programs to execute on processors that do not support CET. Even when CET is supported and enabled, these NOP–like instructions do not affect the execution state of the program, do not cause any additional register pressure, and are minimally intrusive from power and performance perspectives. Follow up from #21135. Guix builds: ```bash 663df8471400f06d4da739e39a886aa17f56a36d66e0ff7cc290686294ef39c9 guix-build-42b589d18fed/output/dist-archive/bitcoin-42b589d18fed.tar.gz 45e841661e1659a634468b6f8c9fb0a7956c31ba296f1fd0c02cd880736d6127 guix-build-42b589d18fed/output/x86_64-apple-darwin18/bitcoin-42b589d18fed-osx-unsigned.dmg 0ea85c99fef35429a5048fa14850bce6b900eaa887aeea419b019852f8d2be78 guix-build-42b589d18fed/output/x86_64-apple-darwin18/bitcoin-42b589d18fed-osx-unsigned.tar.gz 85857a5a4a5d4d3a172d6c361c12c4a94f6505fc12b527ea63b75bfe54ee1001 guix-build-42b589d18fed/output/x86_64-apple-darwin18/bitcoin-42b589d18fed-osx64.tar.gz ``` Gitian builds: ```bash # macOS: bdfd677a6b88273a741b433e1e7f554af50cc76b3342d44ab0c441e2b40efc96 bitcoin-42b589d18fed-osx-unsigned.dmg f3b2d09f3bea7a5cc489b02e8e53dd76a9922338500fae79cad0506655af56f9 bitcoin-42b589d18fed-osx-unsigned.tar.gz 29d5ad5e46bc9fb0056922a8b47c026e5e9f71e6cf447203b74644587d6fb6f7 bitcoin-42b589d18fed-osx64.tar.gz 663df8471400f06d4da739e39a886aa17f56a36d66e0ff7cc290686294ef39c9 src/bitcoin-42b589d18fed.tar.gz 366f8d7a2fc1f3e22cb1018043099126a71ce65380cc27b1c3280cce42d06c98 bitcoin-core-osx-22-res.yml ``` ACKs for top commit: laanwj: Code review ACK 42b589d18fed5e2b3cb6ac9937e3333619967a6c Tree-SHA512: 12cb8d462d64d845b9fe48c5c6978892adff8bf5b5572bb29f35df1f6176e47b32a68bcb6e4883c7d9454e76e8868851005a7325916852a2d0d32659ac7dae3f
2021-05-13codesigning: Use SHA256 as digest for osslsigncode (Windows)Carl Dong
2021-05-13guix: Add codesigning functionalityCarl Dong
generated by cgit v1.2.3 (git 2.26.2) at 2024-11-05 10:43:39 +0000