aboutsummaryrefslogtreecommitdiff
path: root/contrib
AgeCommit message (Collapse)Author
2020-04-28Merge #18556: build: Drop make dist in gitian buildsfanquake
2aa48edec0101f8a77a2189244fc62722ff7a123 refactor: Drop unused ${WRAP_DIR}/${HOST} directory (Hennadii Stepanov) 1362be044724bb49d785ca2e296a3b43343c1690 build: Drop make dist in gitian builds (Hennadii Stepanov) Pull request description: After the merge of #18331, the packaged source tarball is created by `git archive`, but the binaries are built from another one which is made by `make dist`. With this PR the only source tarball, created by `git archive`, is used both for binaries building and for packaging to users. Close #16588. Close #18547. As a good side-effect, #18349 becomes redundant. **Change in behavior** The following variables https://github.com/bitcoin/bitcoin/blob/1b151e3ffce7c1a2ee46bf280cc1d96775d1f91e/configure.ac#L2-L6 are no longer used for naming of directories and tarballs. Instead of them the gitian descriptors use a git tag (if available) or a commit hash. --- Also a small refactor commit picked from #18404. ACKs for top commit: dongcarl: ACK 2aa48edec0101f8a77a2189244fc62722ff7a123 MarcoFalke: ACK 2aa48edec0101f8a77a2189244fc62722ff7a123 fanquake: ACK 2aa48edec0101f8a77a2189244fc62722ff7a123 - I've had a quick look over this, and don't want to block merging if this actually gets as closer to finally having this all sorted out. Obviously we've still got #18741, and after speaking to Carl this morning, there will likely be even more changes after that (not Guix specific). Tree-SHA512: d3b16f87e48d1790a3264940c28acd5d881bfd10f3ce94fb0c8a6af76d8039289d01e0cd4972adac49ae24362857251f6c1e5e09e3e9fbf636c10708b4015a7c
2020-04-28Merge #18629: scripts: add PE .reloc section check to security-check.pyfanquake
3e38023af724a76972d39cbccfb0bba4c54a0323 scripts: add PE .reloc section check to security-check.py (fanquake) Pull request description: The `ld` in binutils has historically had a few issues with PE binaries, there's a good summary in this [thread](https://sourceware.org/bugzilla/show_bug.cgi?id=19011). One issue in particular was `ld` stripping the `.reloc` section out of PE binaries, even though it's required for functioning ASLR. This was [reported by a Tor developer in 2014](https://sourceware.org/bugzilla/show_bug.cgi?id=17321) and they have been patching their [own binutils](https://gitweb.torproject.org/builders/tor-browser-build.git/tree/projects/binutils) ever since. However their patch only made it into binutils at the [start of this year](https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=commit;h=dc9bd8c92af67947db44b3cb428c050259b15cd0). It adds an `--enable-reloc-section` flag, which is turned on by default if you are using `--dynamic-base`. In the mean time this issue has also been worked around by other projects, such as FFmpeg, see [this commit](https://github.com/TheRyuu/FFmpeg/commit/91b668acd6decec0a6f8d20bf56e2644f96adcb9). I have checked our recent supported Windows release binaries, and they do contain a `.reloc` section. From what I understand, we are using all the right compile/linker flags, including `-pie` & `-fPIE`, and have never run into the crashing/entrypoint issues that other projects might have seen. One other thing worth noting here, it how Debian/Ubuntu patch the binutils that they distribute, because that's what we end up using in our gitian builds. In the binutils-mingw-w64 in Bionic (18.04), which we currently use in gitian, PE hardening options/security flags are enabled by default. See the [changelog](https://changelogs.ubuntu.com/changelogs/pool/universe/b/binutils-mingw-w64/binutils-mingw-w64_8ubuntu1/changelog) and the [relevant commit](https://salsa.debian.org/mingw-w64-team/binutils-mingw-w64/-/commit/452b3013b8280cbe35eaeb166a43621b88d5f8b7). However in Focal (20.04), this has now been reversed. PE hardening options are no-longer the default. See the [changelog](https://changelogs.ubuntu.com/changelogs/pool/universe/b/binutils-mingw-w64/binutils-mingw-w64_8.8/changelog) and [relevant commit](https://salsa.debian.org/mingw-w64-team/binutils-mingw-w64/-/commit/7bd8b2fbc242a8c2fc2217f29fd61f94d3babf6f), which cites same .reloc issue mentioned here. Given that we explicitly specify/opt-in to everything that we want to use, the defaults aren't necessarily an issue for us. However I think it highlights the importance of continuing to be explicit about what we want, and not falling-back or relying on upstream. This was also prompted by the possibility of us doing link time garbage collection, see #18579 & #18605. It seemed some sanity checks would be worthwhile in-case the linker goes haywire while garbage collecting. I think Guix is going to bring great benefits when dealing with these kinds of issues. Carl you might have something to say in that regard. ACKs for top commit: dongcarl: ACK 3e38023af724a76972d39cbccfb0bba4c54a0323 Tree-SHA512: af14d63bdb334bde548dd7de3e0946556b7e2598d817b56eb4e75b3f56c705c26aa85dd9783134c4b6a7aeb7cb4de567eed996e94d533d31511f57ed332287da
2020-04-24Merge #18589: Fix naming of macOS SDK and clarify versionfanquake
eb37275a6f972c81caef010b4ee9c5dc88edc759 Fix naming of macOS SDK and clarify version (Andrew Chow) Pull request description: Fixes the `MacOSX10.14.sdk.tar.gz` creation command to have `MacOSX.sdk` be correctly named as `MacOSX10.14.sdk` and for the resulting file to be placed in the current directory. Gitian requires that `tar.gz` contains a folder named `MacOSX10.14.sdk` and the command did not do this originally. Having the file be placed in the current directory is a convenience so builders don't have to go find it. Also clarifies which version of Xcode to download and where it can be downloaded. ACKs for top commit: fanquake: ACK eb37275a6f972c81caef010b4ee9c5dc88edc759 - tested the macOS and Linux SDK extraction. Also noticed something seemingly broken with Apple `tar`, but will open an issue to follow up. Sjors: ACK eb37275 for the macOS instruction Tree-SHA512: d691e14711cf195999291dd6fb7ffe552c86f8b30d2b1a77e88b4db6050dd817ba128b047cf36d29b0bb0d4183e709b7c03aa27f31b64e562ea8cd948434ca55
2020-04-23Fix naming of macOS SDK and clarify versionAndrew Chow
2020-04-23scripts: add PE .reloc section check to security-check.pyfanquake
2020-04-21scripts: add MACHO LAZY_BINDINGS test to test-security-check.pyfanquake
I didn't add the relevant test in #18295.
2020-04-21scripts: add MACHO Canary check to security-check.pyfanquake
2020-04-17Merge #18673: scripted-diff: Sort test includesMarcoFalke
fa4632c41714dfaa699bacc6a947d72668a4deef test: Move boost/stdlib includes last (MarcoFalke) fa488f131fd4f5bab0d01376c5a5013306f1abcd scripted-diff: Bump copyright headers (MarcoFalke) fac5c373006a9e4bcbb56843bb85f1aca4d87599 scripted-diff: Sort test includes (MarcoFalke) Pull request description: When writing tests, often includes need to be added or removed. Currently the list of includes is not sorted, so developers that write tests and have `clang-format` installed will either have an unrelated change (sorting) included in their commit or they will have to manually undo the sort. This pull preempts both issues by just sorting all includes in one commit. Please be aware that this is **NOT** a change to policy to enforce clang-format or any other developer guideline or process. Developers are free to use whatever tool they want, see also #18651. Edit: Also includes a commit to bump the copyright headers, so that the touched files don't need to be touched again for that. ACKs for top commit: practicalswift: ACK fa4632c41714dfaa699bacc6a947d72668a4deef jonatack: ACK fa4632c41714dfaa, light review and sanity checks with gcc build and clang fuzz build Tree-SHA512: 130a8d073a379ba556b1e64104d37c46b671425c0aef0ed725fd60156a95e8dc83fb6f0b5330b2f8152cf5daaf3983b4aca5e75812598f2626c39fd12b88b180
2020-04-17refactor: Drop unused ${WRAP_DIR}/${HOST} directoryHennadii Stepanov
This commit removes the directory that is no longer used since #16667.
2020-04-17build: Drop make dist in gitian buildsHennadii Stepanov
2020-04-16Merge #18598: gitian: Add missing automake package to gitian-win-signer.ymlWladimir J. van der Laan
e44aeefaaed8d698d1b9004b66f85384397b1a75 gitian: Add missing automake package to gitian-win-signer.yml (Andrew Chow) Pull request description: automake is needed to build osslsigncode otherwise autogen.sh fails with the docker virtualization method. ACKs for top commit: hebasto: ACK e44aeefaaed8d698d1b9004b66f85384397b1a75, for `osslsigncode-1.7.1` we did not run `autogen.sh` in the past. fanquake: ACK e44aeefaaed8d698d1b9004b66f85384397b1a75 jonatack: ACK e44aeef Tree-SHA512: a0e615c1b099ee1c469ce41f886f2ece6746234a5a800743a4e8be671e4114fd30e1c35bc0ddcb75778409564129d0fde7ac4e3d70b0f7691f97f729f34c8e0c
2020-04-16scripted-diff: Bump copyright headersMarcoFalke
-BEGIN VERIFY SCRIPT- ./contrib/devtools/copyright_header.py update ./ -END VERIFY SCRIPT-
2020-04-15Merge #18619: gitian: add jonatack gpg key fingerprintfanquake
905e2e85baca388ee1a34c6e2f559b7dd815c253 gitian: add jonatack gpg key fingerprint (Jon Atack) Pull request description: per request https://github.com/bitcoin-core/gitian.sigs/pull/1221#issuecomment-612778063 ACKs for top commit: laanwj: ACK 905e2e85baca388ee1a34c6e2f559b7dd815c253 fanquake: ACK 905e2e85baca388ee1a34c6e2f559b7dd815c253 Tree-SHA512: bddd734f13c53859280db2fa94b47cbb2a8b3f17ed6a6fdda2bf04f7e201e310ae24930e8f4be2f8b65a949659a9d3369704ed70031da9653a66a513fe597b67
2020-04-15Merge #18624: Added my fingerprint Stephan Oeste (Emzy)fanquake
c47adf8df435831c26ca25813fb7272176bd4eb7 Added my fingerprint Stephan Oeste (Emzy) (Stephan Oeste) Pull request description: By request from laanwj added my PGP fingerprint. See: https://github.com/bitcoin-core/gitian.sigs/pull/1220#issuecomment-612778442 ACKs for top commit: Sjors: ACK c47adf8. Fingerprint matches Twitter profile: https://twitter.com/emzy (haven't verified it in any other way) fanquake: ACK c47adf8df435831c26ca25813fb7272176bd4eb7 Tree-SHA512: 3e39ae88f507a12f11fb2d5c779eba79ee2daeddecd0dc3f1fddfa29ce963d0e9af3fa5a10357157812597c10205a6beae31cc70af9471a782da23d8753b7cbd
2020-04-13Added my fingerprint Stephan Oeste (Emzy)Stephan Oeste
By request from added my PGP fingerprint. See: https://github.com/bitcoin-core/gitian.sigs/pull/1220#issuecomment-612778442
2020-04-13gitian: add jonatack gpg key fingerprintJon Atack
2020-04-12build: add linker optimization flags to guixfanquake
Any -O argument will enable optimizations in GNU ld. We can use -O2 here, as this matches our compile flags. Note that this would also enable additional optimizations if using the lld or gold linkers, when compared to -O0.
2020-04-12build: add linker optimization flags to gitian descriptorsfanquake
Any -O argument will enable optimizations in GNU ld. We can use -O2 here, as this matches our compile flags. Note that this would also enable additional optimizations if using the lld or gold linkers, when compared to -O0.
2020-04-12Merge #17595: guix: Enable building for `x86_64-w64-mingw32` targetfanquake
a35e3235891d35daa167116cc70340140e883f06 guix: Appease travis. (Carl Dong) 0b66d22da5f53640e22f05adf880782c613e6d0f guix: Use gcc-9 for mingw-w64 instead of 8 (Carl Dong) ba0b99bdd613ba7f17c6247ece3001e1b44759a3 guix: Don't set MINGW_HAS_SECURE_API CFLAG in depends (Carl Dong) 93439a71eda49fb69f1e82966a23a946733aa6fa guix: Bump to upstream commit with mingw-w64 changes (Carl Dong) 35a96792dda9e78165b1598aeac7b2ab759e7be5 guix: Check mingw symbols, improve SSP fix docs (Carl Dong) 449d8fe25bbe25daacfc67aa89ca32b0a3254c5a guix: Expand on INT trap message (Carl Dong) 3f1f03c67a8e9edf487f08d272adb18b0a3942c8 guix: Spelling fixes (Carl Dong) ff821dd2a1c600488d11e7d9a20e9179ecc9144b guix: Reinstate make-ssp-fixed-gcc (Carl Dong) 360a9e0ad50a36ec79a1a160dbed3966689fd41c guix: Bump time-machine for mingw-w64 patches (Carl Dong) 93e41b7e3b54c17fd1b4c61ee95fc0dc2827e954 guix: Use gcc-8 for mingw-w64 instead of 7 (Carl Dong) ef4f7e4c45c60a69406134122f091c77c6ef740f guix: Set the well-known timezone env var (Carl Dong) acf4b3b3b5accf60a19441a0298ef27001b78e72 guix: Make x86_64-w64-mingw32 builds reproducible (Carl Dong) c4cce00eac691625b78b92f7dba0b7f57def19e5 guix: Remove dead links from README. (Carl Dong) df953a4c9a6143f45864757b706c88b6fa70545a guix: Appease shellcheck. (Carl Dong) 91897c95e191d293eb27d8af15cbeafc5b8f3895 guix: Improve guix-build.sh documentation (Carl Dong) 570d769c6c59b9f6d1a2b95b2ed60432cb33b3ba guix: Build support for Windows (Carl Dong) Pull request description: ~~Based on: https://github.com/bitcoin/bitcoin/pull/16519~~ Based on: #17933 (Time Machines are... shall we say... superior :grin:) This PR allows us to perform Guix builds for the `x86_64-w64-mingw32` target. We do this _without_ splitting up the build script like we do in Gitian by using this newfangled alien technology called `case` statements. (This is WIP and might be changed to `if` statements soon) ACKs for top commit: fanquake: ACK a35e3235891d35daa167116cc70340140e883f06 2/3 Tree-SHA512: c471951c23eb2cda919a71285d8b8f2580cb20f09d5db17b53e13dbd8813e01b3e7a83ea848e4913fd0f2bc12c6c133c5f76b54e65c0d89fed4dfd2e0be19875
2020-04-11gitian: Add missing automake package to gitian-win-signer.ymlAndrew Chow
automake is needed to build osslsigncode otherwise autogen.sh fails.
2020-04-10build: Bump gitian descriptors to 0.21Wladimir J. van der Laan
Per the release process.
2020-04-10Merge #18295: scripts: add MACHO lazy bindings check to security-check.pyfanquake
5ca90f8b598978437340bb8467f527b9edfb2bbf scripts: add MACHO lazy bindings check to security-check.py (fanquake) Pull request description: This is a slightly belated follow up to #17686 and some discussion with Cory. It's not entirely clear if we should make this change due to the way the macOS dynamic loader appears to work. However I'm opening this for some discussion. Also related to #17768. #### Issue: [`LD64`](https://opensource.apple.com/source/ld64/) doesn't set the [MH_BINDATLOAD](https://opensource.apple.com/source/xnu/xnu-6153.11.26/EXTERNAL_HEADERS/mach-o/loader.h.auto.html) bit in the header of MACHO executables, when building with `-bind_at_load`. This is in contradiction to the [documentation](https://opensource.apple.com/source/ld64/ld64-450.3/doc/man/man1/ld.1.auto.html): ```bash -bind_at_load Sets a bit in the mach header of the resulting binary which tells dyld to bind all symbols when the binary is loaded, rather than lazily. ``` The [`ld` in Apples cctools](https://opensource.apple.com/source/cctools/cctools-927.0.2/ld/layout.c.auto.html) does set the bit, however the [cctools-port](https://github.com/tpoechtrager/cctools-port/) that we use for release builds, bundles `LD64`. However; even if the linker hasn't set that bit, the dynamic loader ([`dyld`](https://opensource.apple.com/source/dyld/)) doesn't seem to ever check for it, and from what I understand, it looks at a different part of the header when determining whether to lazily load symbols. Note that our release binaries are currently working as expected, and no lazy loading occurs. #### Example: Using a small program, we can observe the behaviour of the dynamic loader. Conducted using: ```bash clang++ --version Apple clang version 11.0.0 (clang-1100.0.33.17) Target: x86_64-apple-darwin18.7.0 ld -v @(#)PROGRAM:ld PROJECT:ld64-530 BUILD 18:57:17 Dec 13 2019 LTO support using: LLVM version 11.0.0, (clang-1100.0.33.17) (static support for 23, runtime is 23) TAPI support using: Apple TAPI version 11.0.0 (tapi-1100.0.11) ``` ```cpp #include <iostream> int main() { std::cout << "Hello World!\n"; return 0; } ``` Compile and check the MACHO header: ```bash clang++ test.cpp -o test otool -vh test ... Mach header magic cputype cpusubtype caps filetype ncmds sizeofcmds flags MH_MAGIC_64 X86_64 ALL LIB64 EXECUTE 16 1424 NOUNDEFS DYLDLINK TWOLEVEL WEAK_DEFINES BINDS_TO_WEAK PIE # Run and dump dynamic loader bindings: DYLD_PRINT_BINDINGS=1 DYLD_PRINT_TO_FILE=no_bind.txt ./test Hello World! ``` Recompile with `-bind_at_load`. Note still no `BINDATLOAD` flag: ```bash clang++ test.cpp -o test -Wl,-bind_at_load otool -vh test Mach header magic cputype cpusubtype caps filetype ncmds sizeofcmds flags MH_MAGIC_64 X86_64 ALL LIB64 EXECUTE 16 1424 NOUNDEFS DYLDLINK TWOLEVEL WEAK_DEFINES BINDS_TO_WEAK PIE ... DYLD_PRINT_BINDINGS=1 DYLD_PRINT_TO_FILE=bind.txt ./test Hello World! ``` If we diff the outputs, you can see that `dyld` doesn't perform any lazy bindings when the binary is compiled with `-bind_at_load`, even if the `BINDATLOAD` flag is not set: ```diff @@ -1,11 +1,27 @@ +dyld: bind: test:0x103EDF030 = libc++.1.dylib:__ZNKSt3__16locale9use_facetERNS0_2idE, *0x103EDF030 = 0x7FFF70C9FA58 +dyld: bind: test:0x103EDF038 = libc++.1.dylib:__ZNKSt3__18ios_base6getlocEv, *0x103EDF038 = 0x7FFF70CA12C2 +dyld: bind: test:0x103EDF068 = libc++.1.dylib:__ZNSt3__113basic_ostreamIcNS_11char_traitsIcEEE6sentryC1ERS3_, *0x103EDF068 = 0x7FFF70CA12B6 +dyld: bind: test:0x103EDF070 = libc++.1.dylib:__ZNSt3__113basic_ostreamIcNS_11char_traitsIcEEE6sentryD1Ev, *0x103EDF070 = 0x7FFF70CA1528 +dyld: bind: test:0x103EDF080 = libc++.1.dylib:__ZNSt3__16localeD1Ev, *0x103EDF080 = 0x7FFF70C9FAE6 <trim> -dyld: lazy bind: test:0x10D4AC0C8 = libsystem_platform.dylib:_strlen, *0x10D4AC0C8 = 0x7FFF73C5C6E0 -dyld: lazy bind: test:0x10D4AC068 = libc++.1.dylib:__ZNSt3__113basic_ostreamIcNS_11char_traitsIcEEE6sentryC1ERS3_, *0x10D4AC068 = 0x7FFF70CA12B6 -dyld: lazy bind: test:0x10D4AC038 = libc++.1.dylib:__ZNKSt3__18ios_base6getlocEv, *0x10D4AC038 = 0x7FFF70CA12C2 -dyld: lazy bind: test:0x10D4AC030 = libc++.1.dylib:__ZNKSt3__16locale9use_facetERNS0_2idE, *0x10D4AC030 = 0x7FFF70C9FA58 -dyld: lazy bind: test:0x10D4AC080 = libc++.1.dylib:__ZNSt3__16localeD1Ev, *0x10D4AC080 = 0x7FFF70C9FAE6 -dyld: lazy bind: test:0x10D4AC070 = libc++.1.dylib:__ZNSt3__113basic_ostreamIcNS_11char_traitsIcEEE6sentryD1Ev, *0x10D4AC070 = 0x7FFF70CA1528 ``` Note: `dyld` also has a `DYLD_BIND_AT_LAUNCH=1` environment variable, that when set, will force any lazy bindings to be non-lazy: ```bash dyld: forced lazy bind: test:0x10BEC8068 = libc++.1.dylib:__ZNSt3__113basic_ostream ``` #### Thoughts: After looking at the dyld source, I can't find any checks for `MH_BINDATLOAD`. You can see the flags it does check for, such as MH_PIE or MH_BIND_TO_WEAK [here](https://opensource.apple.com/source/dyld/dyld-732.8/src/ImageLoaderMachO.cpp.auto.html). It seems that the lazy binding of any symbols depends on whether or not [lazy_bind_size](https://opensource.apple.com/source/xnu/xnu-6153.11.26/EXTERNAL_HEADERS/mach-o/loader.h.auto.html) from the `LC_DYLD_INFO_ONLY` load command is > 0. Which was mentioned in [#17686](https://github.com/bitcoin/bitcoin/pull/17686#issue-350216254). #### Changes: This PR is one of [Corys commits](https://github.com/theuni/bitcoin/commit/7b6ba26178d2754568a1308d3d44e038e9ebf450), that I've rebased and modified to make build. I've also included an addition to the `security-check.py` script to check for the flag. However, given the above, I'm not entirely sure this patch is the correct approach. If the linker no-longer inserts it, and the dynamic loader doesn't look for it, there might be little benefit to setting it. Or, maybe this is an oversight from Apple and needs some upstream discussion. Looking for some thoughts / Concept ACK/NACK. One alternate approach we could take is to drop the patch and modify security-check.py to look for `lazy_bind_size` == 0 in the `LC_DYLD_INFO_ONLY` load command, using `otool -l`. ACKs for top commit: theuni: ACK 5ca90f8b598978437340bb8467f527b9edfb2bbf Tree-SHA512: 444022ea9d19ed74dd06dc2ab3857a9c23fbc2f6475364e8552d761b712d684b3a7114d144f20de42328d1a99403b48667ba96885121392affb2e05b834b6e1c
2020-04-07guix: Appease travis.Carl Dong
2020-04-07guix: Use gcc-9 for mingw-w64 instead of 8Carl Dong
The libtool unsorted 'find' determinism issue seemed to have been solved in gcc-9's git: d41cd173e23ebea7c758644d6ad6e0fde1c2e3a6 or SVN: r262451 Furthermore, it seems that Ubuntu Focal 20.04 LTS is going to ship with gcc 9 and mingw-w64 7, which will match what we have now. ----- A note on this: Careful observers will see that previously I stated that all released versions of gcc were bootstrapped with a libtool 2.2.7a, meaning that they all had the unsorted 'find' determinism issue first resolved in libtool 2.2.7b. However, I was mistaken, gcc's ltmain.sh CLAIMS it was generated by libtool 2.2.7a, but it was in fact edited manually. It seems that gcc maintains their own versions of ltmain.sh and libtool.m4, and only sometimes backports patches from upstream. Quite confusing.
2020-04-07guix: Don't set MINGW_HAS_SECURE_API CFLAG in dependsCarl Dong
This is no longer needed after 3bef7c22 in the mingw-w64 git repository, which is first included in mingw-w64 v7.0.0. As of the previous bump to our Guix time machine, we now use mingw-w64 v7.0.0.
2020-04-07guix: Bump to upstream commit with mingw-w64 changesCarl Dong
Most of the mingw-w64 toolchain changes have now been upstreamed, we can point to a commit that exists upstream. NOTE: I'm not changing the URL yet until we see that Guix upstream will accept all my patches for macOS. ----- The Guix tree that's referred to by this commit contains the following changes relevant to our mingw-w64 build: b066c25026 Adds a PACKAGES-WITH-*PATCHES procedure which we can use in the future to apply patches to packages if those patches are not considered appropriate to upstream Guix 4719b71572 Adds mingw-w64 (the libc itself) reproducibility patches, taken from debian. 79825bee07 + 401d28e433 + c1c50cb5b0 Add mingw-w64 specific binutils patches, taken from debian. Specifically, the "Make DLL import libraries reproducible" patch made libbitcoinconsensus.dll.a build reproducibly. The followup commits were hotfixes for my mistakes. 0f864175dc Bumps mingw-w64 to v7.0.0. This is the first release that enables secure APIs by default (which we need), and gains _FORTIFY_SOURCE support. This will also be what Ubuntu Focal 20.04 LTS releases with. cdf00cf75d Bumps NSIS to v3.05. This is the first release that includes a fix for a reproducibility bug found by some of the electrum developers. See details here: https://sourceforge.net/p/nsis/bugs/1230/
2020-04-06Merge #18506: net: Hardcoded seeds update for 0.20Wladimir J. van der Laan
0eeb0468e7debb1dbe38242769207d22ed52c1df net: Hardcoded seeds update for 0.20 (Wladimir J. van der Laan) Pull request description: Update hardcoded seeds from http://bitcoin.sipa.be/seeds.txt.gz, according to release process. Output from makeseeds.py: ``` IPv4 IPv6 Onion Pass 1364173 244127 2454 Initial 1364173 244127 2454 Skip entries with invalid address 1129552 213117 2345 After removing duplicates 1129548 213117 2345 Skip entries from suspicious hosts 338216 191944 2249 Enforce minimal number of blocks 336851 188993 2189 Require service bit 1 6998 1520 150 Require minimum uptime 5682 1290 89 Require a known and recent user agent 5622 1279 89 Filter out hosts with multiple bitcoin ports 512 146 89 Look up ASNs and limit results per ASN and per net ``` Top commit has no ACKs. Tree-SHA512: ce1c2cda18dd5bd22586a5283a0877f3bd890437cc29dc1d85452ba4a4d28032f591c8b37f3329e8e649556cf83750b6949a068fad76d1773853d93014609da0
2020-04-04scripts: add MACHO lazy bindings check to security-check.pyfanquake
2020-04-03net: Hardcoded seeds update for 0.20Wladimir J. van der Laan
Update hardcoded seeds from seeds_emzy.txt seeds_lukejr.txt seeds_sipa.txt seeds_sjors.txt, according to release process. Output from makeseeds.py: ``` IPv4 IPv6 Onion Pass 1364173 244127 2454 Initial 1364173 244127 2454 Skip entries with invalid address 1129552 213117 2345 After removing duplicates 1129548 213117 2345 Skip entries from suspicious hosts 338216 191944 2249 Enforce minimal number of blocks 336851 188993 2189 Require service bit 1 6998 1520 150 Require minimum uptime 5682 1290 89 Require a known and recent user agent 5622 1279 89 Filter out hosts with multiple bitcoin ports 512 146 89 Look up ASNs and limit results per ASN and per net ```
2020-04-03Merge #18426: scripts: previous_release: improve behaviour on failed downloadfanquake
332f373a9dece71717f75eb06e6a1fc957f2952b [scripts] previous_release: improve failed download error message (Sebastian Falbesoner) Pull request description: Currently, if the earlier release build/fetch script `previous_release.sh` is invoked with the option `-b` (intending to fetch a binary package from `https://bitcoin.org`) and the download fails, the user sees the following confusing output: ``` $ contrib/devtools/previous_release.sh -r -b v0.9.5 [...] gzip: stdin: not in gzip format tar: Child returned status 1 tar: Error is not recoverable: exiting now ``` This implies that the download worked, but the archive is corrupted, when in reality the HTML document containing the delivery fail reason (most likely 404 Not Found) is saved and tried to get unpacked. In contrast to wget, curl is a bit stubborn and needs explicit instructions to react to server errors via the flag `-f` (outputs error message and returns error code, ideal for scripts): https://curl.haxx.se/docs/manpage.html#-f On the PR branch, the output on failed download looks now the following: ``` $ contrib/devtools/previous_release.sh -r -b v0.9.5 [...] curl: (22) The requested URL returned error: 404 Not Found Download failed. ``` ACKs for top commit: fanquake: ACK 332f373a9dece71717f75eb06e6a1fc957f2952b Tree-SHA512: 046c931ad9e78aeb2d13faa4866d46122ed325aa142483547c2b04032d03223ed2411783b00106fcab0cd91b2f78691531ac526ed7bb3ed7547b6e2adbfb2e93
2020-04-02guix: Check mingw symbols, improve SSP fix docsCarl Dong
2020-04-02guix: Expand on INT trap messageCarl Dong
2020-04-02guix: Spelling fixesCarl Dong
2020-04-02guix: Reinstate make-ssp-fixed-gccCarl Dong
Unfortunately, gcc is still not smart enough to detect whether or not mingw-w64 provides ssp, so let's put it back just for mingw-w64.
2020-04-02guix: Bump time-machine for mingw-w64 patchesCarl Dong
This bump will includes a couple of commits which improve the reproducibility of the mingw-w64 toolchain. Most of which came from debian. They will be upstreamed as upstream Guix release timeline allows.
2020-04-02guix: Use gcc-8 for mingw-w64 instead of 7Carl Dong
We're using mingw-w64 6.0.0, which is paired with gcc-8 in most distros.
2020-04-02guix: Set the well-known timezone env varCarl Dong
2020-04-02guix: Make x86_64-w64-mingw32 builds reproducibleCarl Dong
- Add "--no-insert-timestamp" LDFLAG for x86_64-w64-mingw32 builds "The option --no-insert-timestamp can be used to insert a zero value for the timestamp, this ensuring that binaries produced from identical sources will compare identically." - ld(1) - Set "SetDateSave off" in NSIS script From https://nsis.sourceforge.io/Docs/Chapter4.html#flags "This command sets the file date/time saving flag which is used by the File command to determine whether or not to save the last write date and time of the file, so that it can be restored on installation. Valid flags are 'on' and 'off'. 'on' is the default." - Add commented out NSIS options for reproducibility debugging in NSIS script - Make ZIPs deterministic by reseting file modification times to SOURCE_DATE_EPOCH using touch(1) (Reference: https://reproducible-builds.org/docs/archives/)
2020-04-02guix: Remove dead links from README.Carl Dong
2020-04-02guix: Appease shellcheck.Carl Dong
2020-04-02guix: Improve guix-build.sh documentationCarl Dong
2020-04-02guix: Build support for WindowsCarl Dong
2020-03-26scripts: rename test_64bit_PE to test_PEfanquake
2020-03-26scripts: add MACHO NX check to security-check.pyfanquake
2020-03-26scripts: add MACHO tests to test-security-check.pyfanquake
2020-03-25Merge #18425: releases: Update with new Windows code signing certificateWladimir J. van der Laan
3e0df92bf216e1dce05ca9bf14049f2e42783c30 Update with new Windows code signing certificate (Andrew Chow) Pull request description: The current Windows code signing certificate is about expire (on March 26th 2020). As I have volunteered to take over the Windows code signing duties, I've purchased a new Windows code signing certificate with the same CA and under the same organization (Bitcoin Core Code Signing Association). A signature by the old certificate over the new certificate has been provided to me. This signature can be verified using ``` openssl cms -verify -inform pem -purpose any -content path/to/new/win-codesign.cert -CAfile path/to/old/win-codesign.cert -certfile path/to/old/win-codesign.cert ``` The verification should succeed and the new certificate will be printed out. This can be compared to the contents of `win-codesign.cert`. ``` -----BEGIN PKCS7----- MIIC3AYJKoZIhvcNAQcCoIICzTCCAskCAQExDzANBglghkgBZQMEAgEFADALBgkq hkiG9w0BBwExggKkMIICoAIBATCBkTB8MQswCQYDVQQGEwJHQjEbMBkGA1UECBMS R3JlYXRlciBNYW5jaGVzdGVyMRAwDgYDVQQHEwdTYWxmb3JkMRgwFgYDVQQKEw9T ZWN0aWdvIExpbWl0ZWQxJDAiBgNVBAMTG1NlY3RpZ28gUlNBIENvZGUgU2lnbmlu ZyBDQQIRALWcUnSOxv9FQW3xdaMDO6swDQYJYIZIAWUDBAIBBQCggeQwGAYJKoZI hvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMjAwMzI0MjA0ODM3 WjAvBgkqhkiG9w0BCQQxIgQgtLkmnuSQyczDlJSnJeqbi61p3iJ/rpFABrY8JWBO o74weQYJKoZIhvcNAQkPMWwwajALBglghkgBZQMEASowCwYJYIZIAWUDBAEWMAsG CWCGSAFlAwQBAjAKBggqhkiG9w0DBzAOBggqhkiG9w0DAgICAIAwDQYIKoZIhvcN AwICAUAwBwYFKw4DAgcwDQYIKoZIhvcNAwICASgwDQYJKoZIhvcNAQEBBQAEggEA XaCl3Q8HwI9VpLCb9OY9eQh0QOPyl1KWEc3TP3UvwZwR4/gXkfPOKKf19UnS8eRB 48SgUKRMYWoDYfSVUJRMda9BLkbJbQlHG3LFXhSY2alajpPXEHcMto/XPhVAmqzL w6aSNY0Gaorow696JHpetpKqAAlL1r2GjeaPYi2aZyIAifuhay/qwA+ig0SqzGOw UdgFZWMyS5yanq8/WlLCCql6kKOzT4tEqUaleD7R1q8BTcG2+fmhWR8WwJLpIV6y 7GAqt0Cocu8sYpTNBNk8iKHxzZ2hMZKJpH9lHZuiJ/9vSercrvDy2R4/MG+KnBWb OyiFAt2mC51+63RhLOMJfg== -----END PKCS7----- ``` ACKs for top commit: laanwj: ACK 3e0df92bf216e1dce05ca9bf14049f2e42783c30 theuni: ACK 3e0df92bf216e1dce05ca9bf14049f2e42783c30. Tree-SHA512: 4210f4db1e805ab11231fbae49ea197257c6f7e44f1f6219685b63831704984d824ac2f9e0a3b1bd2655953af72636a474f077cb859fb35852551f5a9f8fbde3
2020-03-25Merge #18395: scripts: add PE dylib checking to symbol-check.pyWladimir J. van der Laan
1a0993ae354c36d6f219e67f82ca8236530d6201 scripts: add PE dylib checking to symbol-check.py (fanquake) Pull request description: Uses `objdump -x` and looks for `DLL Name:` lines. i.e: ```bash objdump -x src/qt/bitcoin-qt.exe | grep "DLL Name:" DLL Name: ADVAPI32.dll DLL Name: dwmapi.dll DLL Name: GDI32.dll DLL Name: IMM32.dll DLL Name: IPHLPAPI.DLL DLL Name: KERNEL32.dll DLL Name: msvcrt.dll DLL Name: ole32.dll DLL Name: OLEAUT32.dll DLL Name: SHELL32.dll DLL Name: SHLWAPI.dll DLL Name: USER32.dll DLL Name: UxTheme.dll DLL Name: VERSION.dll DLL Name: WINMM.dll DLL Name: WS2_32.dll ``` ACKs for top commit: dongcarl: Concept ACK 1a0993ae354c36d6f219e67f82ca8236530d6201 hebasto: ACK 1a0993ae354c36d6f219e67f82ca8236530d6201, tested on Linux Mint 19.3: Tree-SHA512: 0099a50e2c616d5239a15cafa9a7c483e9c40244af41549e4738be0f5360f27a2afb956eb50b47cf446b242f4cfc6dc9d111306a056fb83789eefbd71eddabd2
2020-03-25Merge #18331: build: Use git archive as source tarballWladimir J. van der Laan
e4d366788bc2e8dce8e6ca572fce08d913d15d6b build: Drop needless EXTRA_DIST content (Hennadii Stepanov) 6c4da59f5b5b3c40526d38965d4ffa7fd59f2ebc build: Drop SOURCEDIST reordering (Hennadii Stepanov) 5e6b8b391243016cb06e9e107c2e6a13a744b31e build: Use git archive as source tarball (Hennadii Stepanov) Pull request description: This PR: - is an alternative to #17104 - closes #16734 - closes #6753 The idea is clear described by some developers: - [MarcoFalke](https://github.com/bitcoin/bitcoin/pull/17097#issuecomment-540691850): > This whole concept of explicitly listing each and every file manually (or with a fragile wildcard) is an obvious sisyphean task. I'd say all we need to do is run git archive and be done with it forever, see #16734, #6753, #11530 ... - [laanwj](https://github.com/bitcoin/bitcoin/pull/17097#issuecomment-540706025): > I agree, I've never been a fan of it. I don't think we have any files in the git repository we don't want to ship in the source tarball. --- The suggested changes have a downside which is pointed by [**luke-jr**](https://github.com/bitcoin/bitcoin/pull/17104#issuecomment-540828045): > ... but the distfile needs to include autogen-generated files. This means that a user is not able to run `./configure && make` right away. One must run `./autogen.sh` at first. Here are opinions about mandatory use of `./autogen.sh`: - [ryanofsky](https://github.com/bitcoin/bitcoin/issues/16734#issuecomment-534139356): > It's probably ok to require autogen. I think historically configure scripts were supposed to work on obscure unix systems that would just have a generic shell + make tool + c compiler, and not necessarily need gnu packages like m4 which are needed for autogen. - [laanwj](https://github.com/bitcoin/bitcoin/issues/16734#issuecomment-540729483): > I also think it's fine to require autogen. What is one dependency more, if you're building from source. --- ~Also this PR provides Windows users with ZIP archives of the sources. Additionally the commit ID is stored in these ZIP files as a file comment:~ --- Note for reviewers: please verify is `git archive` output deterministic? ACKs for top commit: MarcoFalke: re-ACK e4d366788bc2e8dce8e6ca572fce08d913d15d6b, only change is adding two dots in a the path 🛳 laanwj: ACK e4d366788bc2e8dce8e6ca572fce08d913d15d6b Tree-SHA512: d1153d3ca4a580696019b92be3555ab004d197d9a2146aacff9d3150eb7093b7d40eebd6eea12d861d93ff62d62b68706e04e64dbe5ea796ff6757486e462193
2020-03-25[scripts] previous_release: improve failed download error messageSebastian Falbesoner
before: ------------------------------------------------------------ $ contrib/devtools/previous_release.sh -r -b v0.9.5 [...] gzip: stdin: not in gzip format tar: Child returned status 1 tar: Error is not recoverable: exiting now ------------------------------------------------------------ now: ------------------------------------------------------------ $ contrib/devtools/previous_release.sh -r -b v0.9.5 [...] curl: (22) The requested URL returned error: 404 Not Found Download failed. ------------------------------------------------------------
2020-03-24Update with new Windows code signing certificateAndrew Chow