aboutsummaryrefslogtreecommitdiff
path: root/contrib
AgeCommit message (Collapse)Author
2020-08-10Merge #19622: build: Drop ancient hack in gitian-linux descriptorfanquake
33a84e8f405ed6dd8885419cef305b4e6c7a428a build: Update and sort package list in gitian-linux.yml (Hennadii Stepanov) 95051682bedc2ef1076af5ede5bd56ca243279e3 build: Drop old hack which is unneeded now (Hennadii Stepanov) Pull request description: The hack was aimed to fix an issue in Ubuntu Trusty 14.04 (see #8188). The current hack implementation was added in #8315. On master (8db23349fe9b512e6801d59d17052c5a7a1c64df) this hack is effectively noop, and it is no longer needed. I see this PR as a step to removing `libfaketime` from gitian builds. ACKs for top commit: dongcarl: tACK 33a84e8f405e laanwj: Code review ACK 33a84e8f405ed6dd8885419cef305b4e6c7a428a Tree-SHA512: 90036c555a500649ccc3d108bf11f09a9cfd2c92c0b598f7e0c0df63a713ae7abaf78f350b68c025470619c967223f45f6a235ad37a6ce1d1a0341ed34963ba0
2020-08-05build: Update and sort package list in gitian-linux.ymlHennadii Stepanov
Virtual package 'binutils-gold' replaced with 'binutils'. Explicitly added 'patch' package.
2020-08-05contrib: Fixup valgrind suppressions fileMarcoFalke
2020-07-31build: Drop old hack which is unneeded nowHennadii Stepanov
This hack is described in #8188. The current implementation was introduced in #8315.
2020-07-30Merge #18011: Replace current benchmarking framework with nanobenchWladimir J. van der Laan
78c312c983255e15fc274de2368a2ec13ce81cbf Replace current benchmarking framework with nanobench (Martin Ankerl) Pull request description: Replace current benchmarking framework with nanobench This replaces the current benchmarking framework with nanobench [1], an MIT licensed single-header benchmarking library, of which I am the autor. This has in my opinion several advantages, especially on Linux: * fast: Running all benchmarks takes ~6 seconds instead of 4m13s on an Intel i7-8700 CPU @ 3.20GHz. * accurate: I ran e.g. the benchmark for SipHash_32b 10 times and calculate standard deviation / mean = coefficient of variation: * 0.57% CV for old benchmarking framework * 0.20% CV for nanobench So the benchmark results with nanobench seem to vary less than with the old framework. * It automatically determines runtime based on clock precision, no need to specify number of evaluations. * measure instructions, cycles, branches, instructions per cycle, branch misses (only Linux, when performance counters are available) * output in markdown table format. * Warn about unstable environment (frequency scaling, turbo, ...) * For better profiling, it is possible to set the environment variable NANOBENCH_ENDLESS to force endless running of a particular benchmark without the need to recompile. This makes it to e.g. run "perf top" and look at hotspots. Here is an example copy & pasted from the terminal output: | ns/byte | byte/s | err% | ins/byte | cyc/byte | IPC | bra/byte | miss% | total | benchmark |--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:---------- | 2.52 | 396,529,415.94 | 0.6% | 25.42 | 8.02 | 3.169 | 0.06 | 0.0% | 0.03 | `bench/crypto_hash.cpp RIPEMD160` | 1.87 | 535,161,444.83 | 0.3% | 21.36 | 5.95 | 3.589 | 0.06 | 0.0% | 0.02 | `bench/crypto_hash.cpp SHA1` | 3.22 | 310,344,174.79 | 1.1% | 36.80 | 10.22 | 3.601 | 0.09 | 0.0% | 0.04 | `bench/crypto_hash.cpp SHA256` | 2.01 | 496,375,796.23 | 0.0% | 18.72 | 6.43 | 2.911 | 0.01 | 1.0% | 0.00 | `bench/crypto_hash.cpp SHA256D64_1024` | 7.23 | 138,263,519.35 | 0.1% | 82.66 | 23.11 | 3.577 | 1.63 | 0.1% | 0.00 | `bench/crypto_hash.cpp SHA256_32b` | 3.04 | 328,780,166.40 | 0.3% | 35.82 | 9.69 | 3.696 | 0.03 | 0.0% | 0.03 | `bench/crypto_hash.cpp SHA512` [1] https://github.com/martinus/nanobench ACKs for top commit: laanwj: ACK 78c312c983255e15fc274de2368a2ec13ce81cbf Tree-SHA512: 9e18770b18b6f95a7d0105a4a5497d31cf4eb5efe6574f4482f6f1b4c88d7e0946b9a4a1e9e8e6ecbf41a3f2d7571240677dcb45af29a6f0584e89b25f32e49e
2020-07-28devtools: Add security check for separate_codeWladimir J. van der Laan
Check that sections are appropriately separated in virtual memory, based on their (expected) permissions. This checks for missing -Wl,-z,separate-code and potentially other problems. Co-authored-by: fanquake <fanquake@gmail.com>
2020-07-21contrib: Clean up previous_releases.pyMarcoFalke
* Replace curl single char options with their verbose counterpart * Stricter check for tarballHash
2020-07-21Merge #19205: script: previous_release.sh rewritten in pythonMarcoFalke
9c34aff39309b8adc99d347e07b6ddb5366498e9 Remove previous_release.sh (Brian Liotti) e1e5960e10a9329d9f55a3967d546ffbdd896030 script: Add previous_release.py (Brian Liotti) Pull request description: Closes #18132 Added functionality: 1) checks file hash before untarring when using the binary download option ACKs for top commit: fjahr: re-ACK 9c34aff39309b8adc99d347e07b6ddb5366498e9 Sjors: tACK 9c34aff39309b8adc99d347e07b6ddb5366498e9 Tree-SHA512: 323f11828736a372a47f048592de8b027ddcd75b38f312dfc73f7b495d1e078bfeb384d9cdf434b3e70f2c6c0ce2da2df48e9a6460ac0e1967c6829a411c52d5
2020-07-21Make test DoS_mapOrphans deterministicDavid Reikher
The RandomOrphan function and the function ecdsa_signature_parse_der_lax in pubkey.cpp were causing non-deterministic test coverage. Force seed in the beginning of the test to make it deterministic. The seed is selected carefully so that all branches of the function ecdsa_signature_parse_der_lax are executed. Prior to this fix, the test was exhibiting non-deterministic coverage since none of the ECDSA signatures that were generated during the test had leading zeroes in either R, S, or both, resulting in some branches of said function not being executed. The seed ensures that both conditions are hit. Removed denialofservice_tests test entry from the list of non-deterministic tests in the coverage script.
2020-07-05Remove previous_release.shBrian Liotti
2020-07-05script: Add previous_release.pyBrian Liotti
closes #18132 added GPG verify for binaries co-authored-by: bboot <bboot@cisco.com>
2020-06-29Merge #19344: docs: update testgen usage examplefanquake
e12e970df6fcae08ff8008812cdeef600d6b2db8 docs: match usage text to script and location (Peter Bushnell) Pull request description: Update the usage text in the README to match the usage text in the Python script. https://github.com/bitcoin/bitcoin/blob/02b26ba1c119c7732f09f09e3b94f75effa569c0/contrib/testgen/gen_key_io_test_vectors.py#L9 https://github.com/bitcoin/bitcoin/blob/02b26ba1c119c7732f09f09e3b94f75effa569c0/contrib/testgen/gen_key_io_test_vectors.py#L10 Also to match the file names in the actual destination. https://github.com/bitcoin/bitcoin/blob/02b26ba1c119c7732f09f09e3b94f75effa569c0/src/test/data/key_io_valid.json https://github.com/bitcoin/bitcoin/blob/02b26ba1c119c7732f09f09e3b94f75effa569c0/src/test/data/key_io_invalid.json Following the README usage text generates new files when the user is likely to have wanted to update the existing files. ACKs for top commit: fanquake: ACK e12e970df6fcae08ff8008812cdeef600d6b2db8 - this looks correct. Tree-SHA512: b7ab61e19a54597a8fbd1844b9cfaef78879e53b882eefe4e0140fa115674df7f061e468835186963b89c963244a17d922f2ad0829b10f62b84f02019ee33edb
2020-06-22macos: Bump to xcode 11.3.1 and 10.15 SDKCory Fields
This gets us a newer SDK with c++17 support and retains 10.12 back-compat. Co-authored-by: Carl Dong <contact@carldong.me>
2020-06-22contrib: macdeploy: Remove historical extraction notesCarl Dong
2020-06-22contrib: macdeploy: Use apple-sdk-tools instead of xar+pbzxCarl Dong
2020-06-22Adapt rest of tooling to new SDK naming schemeCarl Dong
2020-06-21docs: match usage text to script and locationPeter Bushnell
Update the usage text in the README to match the usage text in the Python script. https://github.com/bitcoin/bitcoin/blob/02b26ba1c119c7732f09f09e3b94f75effa569c0/contrib/testgen/gen_key_io_test_vectors.py#L9 Also to match the file names in the actual destination. https://github.com/bitcoin/bitcoin/blob/02b26ba1c119c7732f09f09e3b94f75effa569c0/src/test/data/key_io_valid.json https://github.com/bitcoin/bitcoin/blob/02b26ba1c119c7732f09f09e3b94f75effa569c0/src/test/data/key_io_invalid.json Following the README usage text generates new files when the user is likely to have wanted to update the existing files.
2020-06-19contrib: macdeploy: Correctly generate macOS SDKCarl Dong
Previously, we did not include the macOS SDK libc++ headers in our SDK creation process and instead used whichever libc++ headers shipped with the clang package we downloaded in depends. This change adds a script (which works on both GNU/Linux and macOS) to correctly generate the macOS SDK including the libc++ headers. This can be thought of as a simplified rewrite of tpoechtrager's script: https://github.com/tpoechtrager/osxcross/blob/d3392f4eae78f3fa3f1fd065fa423f2712825102/tools/gen_sdk_package.sh The location within the SDK where we place the libc++ headers is chosen such that clang's search path detection logic for sysroots would pick up the headers properly. We also document this change.
2020-06-17Merge #19287: contrib: Fix SyntaxWarning in Python base58 implementationfanquake
47b49a05eafddcaef373f70436d794e9f9f7495c contrib: Fix SyntaxWarning in Python base58 implementation (Alex Willmer) Pull request description: In Python integers should be compared for equality (`i == j`), not identity (`i is j`). Recent versions of CPython 3.x emit a SyntaxWarning when they encounter this incorrect usage, e.g. ``` $ python3 base58.py base58.py:110: SyntaxWarning: "is" with a literal. Did you mean "=="? assert get_bcaddress_version('15VjRaDX9zpbA8LVnbrCAFzrVzN7ixHNsC') is 0 Tests passed ``` ACKs for top commit: MarcoFalke: ACK 47b49a05eafddcaef373f70436d794e9f9f7495c Tree-SHA512: 9f8962025dcdfa062c0515c68a1864f5bbeb86bd0510c0ec0e413a5edb6afbfd5f41b4c0255784e53db8eaf39c68b7cfa7cc8a33a2e5214aae463fda374f8719
2020-06-16test: use subprocess.run() in test-security-check.pyfanquake
2020-06-16tests: run test-security-check.py in CIfanquake
2020-06-15contrib: Fix SyntaxWarning in Python base58 implementationAlex Willmer
In Python integers should be compared for equality (`i == j`), not identity (`i is j`). Recent versions of CPython 3.x emit a SyntaxWarning when they encounter this incorrect usage, e.g. ``` $ python3 base58.py base58.py:110: SyntaxWarning: "is" with a literal. Did you mean "=="? assert get_bcaddress_version('15VjRaDX9zpbA8LVnbrCAFzrVzN7ixHNsC') is 0 Tests passed ```
2020-06-13Replace current benchmarking framework with nanobenchMartin Ankerl
This replaces the current benchmarking framework with nanobench [1], an MIT licensed single-header benchmarking library, of which I am the autor. This has in my opinion several advantages, especially on Linux: * fast: Running all benchmarks takes ~6 seconds instead of 4m13s on an Intel i7-8700 CPU @ 3.20GHz. * accurate: I ran e.g. the benchmark for SipHash_32b 10 times and calculate standard deviation / mean = coefficient of variation: * 0.57% CV for old benchmarking framework * 0.20% CV for nanobench So the benchmark results with nanobench seem to vary less than with the old framework. * It automatically determines runtime based on clock precision, no need to specify number of evaluations. * measure instructions, cycles, branches, instructions per cycle, branch misses (only Linux, when performance counters are available) * output in markdown table format. * Warn about unstable environment (frequency scaling, turbo, ...) * For better profiling, it is possible to set the environment variable NANOBENCH_ENDLESS to force endless running of a particular benchmark without the need to recompile. This makes it to e.g. run "perf top" and look at hotspots. Here is an example copy & pasted from the terminal output: | ns/byte | byte/s | err% | ins/byte | cyc/byte | IPC | bra/byte | miss% | total | benchmark |--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:---------- | 2.52 | 396,529,415.94 | 0.6% | 25.42 | 8.02 | 3.169 | 0.06 | 0.0% | 0.03 | `bench/crypto_hash.cpp RIPEMD160` | 1.87 | 535,161,444.83 | 0.3% | 21.36 | 5.95 | 3.589 | 0.06 | 0.0% | 0.02 | `bench/crypto_hash.cpp SHA1` | 3.22 | 310,344,174.79 | 1.1% | 36.80 | 10.22 | 3.601 | 0.09 | 0.0% | 0.04 | `bench/crypto_hash.cpp SHA256` | 2.01 | 496,375,796.23 | 0.0% | 18.72 | 6.43 | 2.911 | 0.01 | 1.0% | 0.00 | `bench/crypto_hash.cpp SHA256D64_1024` | 7.23 | 138,263,519.35 | 0.1% | 82.66 | 23.11 | 3.577 | 1.63 | 0.1% | 0.00 | `bench/crypto_hash.cpp SHA256_32b` | 3.04 | 328,780,166.40 | 0.3% | 35.82 | 9.69 | 3.696 | 0.03 | 0.0% | 0.03 | `bench/crypto_hash.cpp SHA512` [1] https://github.com/martinus/nanobench * Adds support for asymptotes This adds support to calculate asymptotic complexity of a benchmark. This is similar to #17375, but currently only one asymptote is supported, and I have added support in the benchmark `ComplexMemPool` as an example. Usage is e.g. like this: ``` ./bench_bitcoin -filter=ComplexMemPool -asymptote=25,50,100,200,400,600,800 ``` This runs the benchmark `ComplexMemPool` several times but with different complexityN settings. The benchmark can extract that number and use it accordingly. Here, it's used for `childTxs`. The output is this: | complexityN | ns/op | op/s | err% | ins/op | cyc/op | IPC | total | benchmark |------------:|--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|----------:|:---------- | 25 | 1,064,241.00 | 939.64 | 1.4% | 3,960,279.00 | 2,829,708.00 | 1.400 | 0.01 | `ComplexMemPool` | 50 | 1,579,530.00 | 633.10 | 1.0% | 6,231,810.00 | 4,412,674.00 | 1.412 | 0.02 | `ComplexMemPool` | 100 | 4,022,774.00 | 248.58 | 0.6% | 16,544,406.00 | 11,889,535.00 | 1.392 | 0.04 | `ComplexMemPool` | 200 | 15,390,986.00 | 64.97 | 0.2% | 63,904,254.00 | 47,731,705.00 | 1.339 | 0.17 | `ComplexMemPool` | 400 | 69,394,711.00 | 14.41 | 0.1% | 272,602,461.00 | 219,014,691.00 | 1.245 | 0.76 | `ComplexMemPool` | 600 | 168,977,165.00 | 5.92 | 0.1% | 639,108,082.00 | 535,316,887.00 | 1.194 | 1.86 | `ComplexMemPool` | 800 | 310,109,077.00 | 3.22 | 0.1% |1,149,134,246.00 | 984,620,812.00 | 1.167 | 3.41 | `ComplexMemPool` | coefficient | err% | complexity |--------------:|-------:|------------ | 4.78486e-07 | 4.5% | O(n^2) | 6.38557e-10 | 21.7% | O(n^3) | 3.42338e-05 | 38.0% | O(n log n) | 0.000313914 | 46.9% | O(n) | 0.0129823 | 114.4% | O(log n) | 0.0815055 | 133.8% | O(1) The best fitting curve is O(n^2), so the algorithm seems to scale quadratic with `childTxs` in the range 25 to 800.
2020-06-07ci: Switch to bitcoincore.org downloadMarcoFalke
2020-06-03test: Make valgrind.supp work on aarch64MarcoFalke
The equivalent suppression on aarch64 looks like: { <insert_a_suppression_name_here> Memcheck:Param pwrite64(buf) fun:__libc_pwrite64 fun:pwrite fun:__os_io obj:/usr/lib/aarch64-linux-gnu/libdb_cxx-5.3.so fun:__log_flush_int fun:__log_flush fun:__memp_sync_int fun:__db_sync fun:__db_refresh fun:__db_close fun:__fop_subdb_setup fun:__db_open fun:__db_open_pp }
2020-05-21Merge #18958: guix: Make V=1 more powerful for debuggingfanquake
f852761aec81ed23c7b9e4546c08d1ef303f2507 guix: Add clarifying documentation for V env var (Carl Dong) 85f4a4b0822e3aa10310c4623eff719f301e9263 guix: Make V=1 more powerful for debugging (Carl Dong) Pull request description: ``` - Print commands in both unexpanded and expanded forms - Set VERBOSE=1 for CMake ``` Ping MarcoFalke hopefully you use `V=1` already for the Guix builds on DrahtBot? ACKs for top commit: fanquake: ACK f852761aec81ed23c7b9e4546c08d1ef303f2507. Ran a Windows Guix build and compared the output from master and this PR when using `V=1`. i.e `HOSTS=x86_64-w64-mingw32 PATH="/root/.config/guix/current/bin${PATH:+:}$PATH" V=1 ./contrib/guix/guix-build.sh`. Tree-SHA512: 8bc466fa7b869618bbd5a0a91c6b23d4785009289f8dfb93b0349317463a9ab9ece128c72436e02a0819722a63e703100aed15807867a716fda891292fcb9d9d
2020-05-20guix: Add clarifying documentation for V env varCarl Dong
2020-05-14scripts: add additional type annotations to security-check.pyfanquake
2020-05-14scripts: add run_command to security-check.pyfanquake
Deduplicate all the subprocess code as mentioned in 18713.
2020-05-14scripts: remove NONFATAL from security-check.pyfanquake
2020-05-14scripts: no-longer check for 32 bit windows in security-check.pyfanquake
2020-05-12guix: Make V=1 more powerful for debuggingCarl Dong
- Print commands in both unexpanded and expanded forms - Set VERBOSE=1 for CMake
2020-05-06Merge #18741: guix: Make source tarball using git-archivefanquake
bfe1ba2f5b36056e0c41edf8206b93d3d83098df rel-builds: Specify core.abbrev for git-rev-parse (Carl Dong) 27e63e01cce368d67092de8f0c736927d6f6aa69 build: Accomodate makensis v2.x (Carl Dong) 1f2c39a30e0f82046c7aecddfda3eb99cb536816 guix: Remove logical cores requirement (Carl Dong) a4f6ffa71e335d4b2a6bf525b7f416968f9cd9f7 lint: Also enable source statements for non-gitian (Carl Dong) d256f91cb1b0d6ff5170106b99b0266cbe51f5a2 rel-builds: Directly deploy win installer to OUTDIR (Carl Dong) fa791da02f9684e3fd554b687fb692ae6a23d65a nsis: Specify OutFile path only once (Carl Dong) 14701604d0904bc5bbf1c67de08f8ee6d3215523 guix: Expose GIT_COMMON_DIR in container as readonly (Carl Dong) f5a6ac4f48b18f93050d77bcb23f9cf45ec34647 guix: Make source tarball using git-archive (Carl Dong) 395c1137f630dc495ffb2752a23bc1dfd470ee53 gitian: Limit sourced script to just assignments (Carl Dong) Pull request description: Based on: #18556 Related: https://github.com/bitcoin/bitcoin/pull/17595#discussion_r399728721 ACKs for top commit: fanquake: ACK bfe1ba2f5b36056e0c41edf8206b93d3d83098df - I agree with Carl, and am going to merge this. I'd like for Linux Guix builds to be working again, and we can rebase #18818. Tree-SHA512: c87ada7e3de17ca0b692a91029b86573442ded5780fc081c214773f6b374a0cdbeaf6f6898c36669c2e247ee32aa7f82defb1180f8decac52c65f0c140f18674
2020-05-05Merge #18885: contrib: Move optimize-pngs.py script to the maintainer repoMarcoFalke
fa13090d2048c9a1e475eb25de756763f8adddb8 contrib: Remove optimize-pngs.py script, which lives in the maintainer repo (MarcoFalke) Pull request description: Moved to https://github.com/bitcoin-core/bitcoin-maintainer-tools/blob/master/optimize-pngs.py Bitcoin Core should focus on the full node implementation, not on scripts to compress png images. This script is only used when new PNG files are added to the repo. This happens about once every two years. So fetching the script from the other repo should not be a burden, but removing it from this repo is going to cut down on the meta files we need to maintain in the main repo. ACKs for top commit: practicalswift: ACK fa13090d2048c9a1e475eb25de756763f8adddb8 -- `+0 lines, -82 lines` :) promag: ACK fa13090d2048c9a1e475eb25de756763f8adddb8. hebasto: ACK fa13090d2048c9a1e475eb25de756763f8adddb8, verified that script is already [moved](https://github.com/bitcoin-core/bitcoin-maintainer-tools/pull/56). Tree-SHA512: 37d111adae769bcddc6ae88041032d5a2b8b228fec67f555c8333c38de3992f5138b30bea868d7d6d6b7f966a47133e5853134373b149ab23cba3b8b560ecb31
2020-05-05contrib: Remove optimize-pngs.py script, which lives in the maintainer repoMarcoFalke
https://github.com/bitcoin-core/bitcoin-maintainer-tools/blob/master/optimize-pngs.py
2020-05-05valgrind : remove duplicate BCLog::Logger suppressionfanquake
This was added in #17770, but is identical to the supression above.
2020-05-05valgrind: remove rest_blockhash_by_height suppressionfanquake
This should no-longer be necessary after #18785.
2020-05-04rel-builds: Specify core.abbrev for git-rev-parseCarl Dong
Chose 12 because the kernel uses it: https://public-inbox.org/git/CA+55aFy0_pwtFOYS1Tmnxipw9ZkRNCQHmoYyegO00pjMiZQfbg@mail.gmail.com/raw And also because it's a nice number.
2020-05-01guix: Remove logical cores requirementCarl Dong
Thanks MarcoFalke for pushing this to its limits and testing :-)
2020-05-01rel-builds: Directly deploy win installer to OUTDIRCarl Dong
2020-04-28guix: Expose GIT_COMMON_DIR in container as readonlyCarl Dong
When using worktrees or submodules, you'll see a `.git' plain text file at the root of your working tree instead of the usual `.git' directory. This plain text file will point to the real GIT_DIR, under the GIT_COMMON_DIR. From experimentation, the full GIT_COMMON_DIR is required to exist for operations such as git-archive(1), so we expose it as readonly inside the container.
2020-04-28guix: Make source tarball using git-archiveCarl Dong
2020-04-28gitian: Limit sourced script to just assignmentsCarl Dong
Previously, the sourced script would create the source tarball. Now, it only assigns variables and the source-ing script has more flexibility in determining what to do with these variables. See later commit showing how this flexibility is useful in our Guix builds.
2020-04-28Merge #18556: build: Drop make dist in gitian buildsfanquake
2aa48edec0101f8a77a2189244fc62722ff7a123 refactor: Drop unused ${WRAP_DIR}/${HOST} directory (Hennadii Stepanov) 1362be044724bb49d785ca2e296a3b43343c1690 build: Drop make dist in gitian builds (Hennadii Stepanov) Pull request description: After the merge of #18331, the packaged source tarball is created by `git archive`, but the binaries are built from another one which is made by `make dist`. With this PR the only source tarball, created by `git archive`, is used both for binaries building and for packaging to users. Close #16588. Close #18547. As a good side-effect, #18349 becomes redundant. **Change in behavior** The following variables https://github.com/bitcoin/bitcoin/blob/1b151e3ffce7c1a2ee46bf280cc1d96775d1f91e/configure.ac#L2-L6 are no longer used for naming of directories and tarballs. Instead of them the gitian descriptors use a git tag (if available) or a commit hash. --- Also a small refactor commit picked from #18404. ACKs for top commit: dongcarl: ACK 2aa48edec0101f8a77a2189244fc62722ff7a123 MarcoFalke: ACK 2aa48edec0101f8a77a2189244fc62722ff7a123 fanquake: ACK 2aa48edec0101f8a77a2189244fc62722ff7a123 - I've had a quick look over this, and don't want to block merging if this actually gets as closer to finally having this all sorted out. Obviously we've still got #18741, and after speaking to Carl this morning, there will likely be even more changes after that (not Guix specific). Tree-SHA512: d3b16f87e48d1790a3264940c28acd5d881bfd10f3ce94fb0c8a6af76d8039289d01e0cd4972adac49ae24362857251f6c1e5e09e3e9fbf636c10708b4015a7c
2020-04-28Merge #18629: scripts: add PE .reloc section check to security-check.pyfanquake
3e38023af724a76972d39cbccfb0bba4c54a0323 scripts: add PE .reloc section check to security-check.py (fanquake) Pull request description: The `ld` in binutils has historically had a few issues with PE binaries, there's a good summary in this [thread](https://sourceware.org/bugzilla/show_bug.cgi?id=19011). One issue in particular was `ld` stripping the `.reloc` section out of PE binaries, even though it's required for functioning ASLR. This was [reported by a Tor developer in 2014](https://sourceware.org/bugzilla/show_bug.cgi?id=17321) and they have been patching their [own binutils](https://gitweb.torproject.org/builders/tor-browser-build.git/tree/projects/binutils) ever since. However their patch only made it into binutils at the [start of this year](https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=commit;h=dc9bd8c92af67947db44b3cb428c050259b15cd0). It adds an `--enable-reloc-section` flag, which is turned on by default if you are using `--dynamic-base`. In the mean time this issue has also been worked around by other projects, such as FFmpeg, see [this commit](https://github.com/TheRyuu/FFmpeg/commit/91b668acd6decec0a6f8d20bf56e2644f96adcb9). I have checked our recent supported Windows release binaries, and they do contain a `.reloc` section. From what I understand, we are using all the right compile/linker flags, including `-pie` & `-fPIE`, and have never run into the crashing/entrypoint issues that other projects might have seen. One other thing worth noting here, it how Debian/Ubuntu patch the binutils that they distribute, because that's what we end up using in our gitian builds. In the binutils-mingw-w64 in Bionic (18.04), which we currently use in gitian, PE hardening options/security flags are enabled by default. See the [changelog](https://changelogs.ubuntu.com/changelogs/pool/universe/b/binutils-mingw-w64/binutils-mingw-w64_8ubuntu1/changelog) and the [relevant commit](https://salsa.debian.org/mingw-w64-team/binutils-mingw-w64/-/commit/452b3013b8280cbe35eaeb166a43621b88d5f8b7). However in Focal (20.04), this has now been reversed. PE hardening options are no-longer the default. See the [changelog](https://changelogs.ubuntu.com/changelogs/pool/universe/b/binutils-mingw-w64/binutils-mingw-w64_8.8/changelog) and [relevant commit](https://salsa.debian.org/mingw-w64-team/binutils-mingw-w64/-/commit/7bd8b2fbc242a8c2fc2217f29fd61f94d3babf6f), which cites same .reloc issue mentioned here. Given that we explicitly specify/opt-in to everything that we want to use, the defaults aren't necessarily an issue for us. However I think it highlights the importance of continuing to be explicit about what we want, and not falling-back or relying on upstream. This was also prompted by the possibility of us doing link time garbage collection, see #18579 & #18605. It seemed some sanity checks would be worthwhile in-case the linker goes haywire while garbage collecting. I think Guix is going to bring great benefits when dealing with these kinds of issues. Carl you might have something to say in that regard. ACKs for top commit: dongcarl: ACK 3e38023af724a76972d39cbccfb0bba4c54a0323 Tree-SHA512: af14d63bdb334bde548dd7de3e0946556b7e2598d817b56eb4e75b3f56c705c26aa85dd9783134c4b6a7aeb7cb4de567eed996e94d533d31511f57ed332287da
2020-04-24Merge #18589: Fix naming of macOS SDK and clarify versionfanquake
eb37275a6f972c81caef010b4ee9c5dc88edc759 Fix naming of macOS SDK and clarify version (Andrew Chow) Pull request description: Fixes the `MacOSX10.14.sdk.tar.gz` creation command to have `MacOSX.sdk` be correctly named as `MacOSX10.14.sdk` and for the resulting file to be placed in the current directory. Gitian requires that `tar.gz` contains a folder named `MacOSX10.14.sdk` and the command did not do this originally. Having the file be placed in the current directory is a convenience so builders don't have to go find it. Also clarifies which version of Xcode to download and where it can be downloaded. ACKs for top commit: fanquake: ACK eb37275a6f972c81caef010b4ee9c5dc88edc759 - tested the macOS and Linux SDK extraction. Also noticed something seemingly broken with Apple `tar`, but will open an issue to follow up. Sjors: ACK eb37275 for the macOS instruction Tree-SHA512: d691e14711cf195999291dd6fb7ffe552c86f8b30d2b1a77e88b4db6050dd817ba128b047cf36d29b0bb0d4183e709b7c03aa27f31b64e562ea8cd948434ca55
2020-04-23Fix naming of macOS SDK and clarify versionAndrew Chow
2020-04-23scripts: add PE .reloc section check to security-check.pyfanquake
2020-04-21scripts: add MACHO LAZY_BINDINGS test to test-security-check.pyfanquake
I didn't add the relevant test in #18295.
2020-04-21scripts: add MACHO Canary check to security-check.pyfanquake
generated by cgit v1.2.3 (git 2.26.2) at 2024-11-06 19:53:25 +0000