aboutsummaryrefslogtreecommitdiff
path: root/src/Makefile.am
AgeCommit message (Collapse)Author
2020-06-15walletdb: Move BDB specific things into bdb.{cpp/h}Andrew Chow
Leave wallet/db.{cpp/h} for generic WalletDatabase stuff. The BDB specific stuff goes into bdb.{cpp/h}
2020-05-28Remove g_rpc_chain globalRussell Yanofsky
Replace with RPC request reference to new WalletContext struct similar to the existing NodeContext struct and reference. This PR is a followup to 25ad2c623af30056ffb36dcd203a52edda2b170f https://github.com/bitcoin/bitcoin/pull/18740 removing the g_rpc_node global. Some later PRs will follow this up and move more wallet globals to the WalletContext struct. Co-authored-by: João Barbosa <joao.paulo.barbosa@gmail.com>
2020-05-27Merge #18918: wallet: Move salvagewallet into wallettoolSamuel Dobson
84ae0578b6c68dda145ca65fef510ce0fdac0d7b Add release notes about salvage changes (Andrew Chow) ea337f2d0318a860f695698cfb3aa91c03ded858 Move RecoverKeysOnlyFilter into RecoverDataBaseFile (Andrew Chow) 9ea2d258b46e8a9776100633585ed0feede5c2a4 Move RecoverDatabaseFile and RecoverKeysOnlyFilter into salvage.{cpp/h} (Andrew Chow) b426c7764d26e280e1f814cf36e050743c45cd12 Make BerkeleyBatch::Recover and WalletBatch::RecoverKeysOnlyFilter standalone (Andrew Chow) 2741774214168eb287c7066d6823afe5e570381d Expose a version of ReadKeyValue and use it in RecoverKeysOnlyFilter (Andrew Chow) ced95d0e43389fe62b5d30fcc7c42dbca0e88242 Move BerkeleyEnvironment::Salvage into BerkeleyBatch::Recover (Andrew Chow) 07250b8dcebe2b97ed0fd900ad35cba4091b8ecf walletdb: remove fAggressive from Salvage (Andrew Chow) 8ebcbc85c652665b78dcfd2ad55fa67cafd42c73 walletdb: don't automatically salvage when corruption is detected (Andrew Chow) d321046f4bb4887742699c586755a21f3a2edbe1 wallet: remove -salvagewallet (Andrew Chow) cdd955e580dff99f3fa440494ed2b348f7f094af Add basic test for bitcoin-wallet salvage (Andrew Chow) c87770915b88d195d264b58111c64142b1965cfa wallettool: Add a salvage command (Andrew Chow) Pull request description: Removes the `-salvagewallet` startup option and adds a `salvage` command to the `bitcoin-wallet` tool. As such, `-salvagewallet` is removed. Additionally, the automatic salvage that is done if the wallet file fails to load is removed. Lastly the salvage code entirely is moved out entirely into `bitcoin-wallet` from `walletdb.{cpp/h}` and `db.{cpp/h}`. ACKs for top commit: jonatack: ACK 84ae0578b6c68dda145ca65fef510ce0fdac0d7b feedback taken, and compared to my previous review, the bitcoin-wallet salvage command now seems to run and it exits without raising. The new test passes at both 9454105 and 84ae057 so as a sanity check I'd agree there is room for improvement, if possible. MarcoFalke: re-ACK 84ae0578b6 🏉 Empact: Code Review ACK https://github.com/bitcoin/bitcoin/pull/18918/commits/84ae0578b6c68dda145ca65fef510ce0fdac0d7b ryanofsky: Code review ACK 84ae0578b6c68dda145ca65fef510ce0fdac0d7b. Lot of small changes since previous review: added verify step before salvage, added basic test in new commit, removed unused scanstate variable and warnings parameter, tweaked various comments and strings, moved fsuccess variable declaration meshcollider: Concept / light code review ACK 84ae0578b6c68dda145ca65fef510ce0fdac0d7b Tree-SHA512: 05be116b56ecade1c58faca1728c8fe4b78f0a082dbc2544a3f7507dd155f1f4f39070bd1fe90053444384337bc48b97149df5c1010230d78f8ecc08e69d93af
2020-05-25Move RecoverDatabaseFile and RecoverKeysOnlyFilter into salvage.{cpp/h}Andrew Chow
2020-05-21Merge #18740: Remove g_rpc_node globalMarcoFalke
b3f7f375efb9a9ca9a7a4f2caf41fe3df2262520 refactor: Remove g_rpc_node global (Russell Yanofsky) ccb5059ee89f6e8dc31ba5b82830b384890bb65e scripted-diff: Remove g_rpc_node references (Russell Yanofsky) 6fca33b2edc09ed62dab2323c780b31585de1750 refactor: Pass NodeContext to RPC and REST methods through util::Ref (Russell Yanofsky) 691c817b340d10e806dc3b1834d2a8fcc5e681fd Add util::Ref class as temporary alternative for c++17 std::any (Russell Yanofsky) Pull request description: This PR removes the `g_rpc_node` global, to get same benefits we see removing other globals and make RPC code more testable, modular, and reusable. This uses a hybrid of the approaches suggested in #17548. Instead of using `std::any`, which isn't available in c++11, or `void*`, which isn't type safe, it uses a small new `util::Ref` helper class, which acts like a simplified `std::any` that only holds references, not values. Motivation for writing this was to provide an simpler alternative to #18647 by Harris Brakmić (brakmic) which avoids some shortcomings of that PR (https://github.com/bitcoin/bitcoin/pull/18647#issuecomment-617878826) ACKs for top commit: MarcoFalke: re-ACK b3f7f375ef, only change is adding back const and more tests 🚾 ajtowns: ACK b3f7f375efb9a9ca9a7a4f2caf41fe3df2262520 Tree-SHA512: 56292268a001bdbe34d641db1180c215351503966ff451e55cc96c9137f1d262225d7d7733de9c9da7ce7d7a4b34213a98c2476266b58c89dbbb0f3cb5aa5d70
2020-05-21Merge #18677: Multiprocess build supportfanquake
e2bab2aa162ae38b2bf8195b577c982402fbee9d multiprocess: add multiprocess travis configuration (Russell Yanofsky) 603fd6a2e708c04ef6c9880f89d0a4cbaa6fc7c5 depends: add MULTIPROCESS depends option (Russell Yanofsky) 5d1377b52bfcd4edf8553aaf332bfeb92fc554cc build: multiprocess autotools changes (Russell Yanofsky) Pull request description: This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10). --- This PR consists of build changes only. It adds an `--enable-multiprocess` autoconf option (off by default and marked experimental), that builds new `bitcoin-node` and `bitcoin-gui` binaries. These currently function the same as existing `bitcoind` and `bitcoin-qt` binaries, but are extended in #10102 with IPC features to execute node, wallet, and gui functions in separate processes. In addition to adding the `--enable-multiprocess` config flag, it also adds a depends package and autoconf rules to build with the [libmultiprocess](https://github.com/chaincodelabs/libmultiprocess) library, and it adds new travis configuration to exercise the build code and run functional tests with the new binaries. The changes in this PR were originally part of #10102 but were moved into #16367 to be able to develop and review the multiprocess build changes independently of the code changes. #16367 was briefly merged and then reverted in #18588. Only change since #16367 has been dropping the `native_boost.mk` depends package which was pointed out to be no longer necessary in https://github.com/bitcoin/bitcoin/pull/16367#issuecomment-596484337 and https://github.com/bitcoin/bitcoin/pull/18588#pullrequestreview-391765649 ACKs for top commit: practicalswift: ACK e2bab2aa162ae38b2bf8195b577c982402fbee9d Sjors: tACK e2bab2aa162ae38b2bf8195b577c982402fbee9d on macOS 10.15.4 hebasto: ACK e2bab2aa162ae38b2bf8195b577c982402fbee9d, tested on Linux Mint 19.3 (x86_64): Tree-SHA512: b5a76eab5abf63d9d8b6d628cbdff4cc1888eef15cafa0a5d56369e2f9d02595fed623f4b74b2cf2830c42c05a774f0943e700f9c768a82d9d348cad199e135c
2020-05-13Add util::Ref class as temporary alternative for c++17 std::anyRussell Yanofsky
This commit does not change behavior
2020-05-12build: multiprocess autotools changesRussell Yanofsky
autoconf and automake changes to support multiprocess gui/node/wallet execution. This adds a new --enable-multiprocess flag, and build configuration code to detect libraries needed for multiprocess support. The --enable-multiprocess flag builds new bitcoin-node and bitcoin-gui executables, which are updated in https://github.com/bitcoin/bitcoin/pull/10102 to communicate across processes. But for now they are functionally equivalent to existing bitcoind and bitcoin-qt executables.
2020-05-07test: remove glibc fdelt sanity checkfanquake
As is, this sanity check doesn't seem to be testing fdelt_chk, because passing a value of "0" to FD_SET wont cause the compiler to insert any calls to fdelt_chk(). The documentation is a little misleading. If we actually triggered fdelt_chk at runtime, bitcoind would abort. I think this check would be better replaced (if possible) by additional checks in security-check.py. The compiler may insert a call to fdelt_warn() (aliased with fdelt_chk in glibc) at compile time if it can determine that an invalid value is being passed to FD_SET. These checks are essentially; value < 0 or value >= FD_SETSIZE along with a check for wether the value is a compile time constant. If the compiler can determine an invalid value is being passed, a call to fdelt_warn will be inserted. Passing 0 should never cause a call to be inserted. You can check this after compiling: ```bash objdump -dC bitcoind | grep sanity_fdelt ... 0000000000399d20 <sanity_test_fdelt()>: 399d20: 48 81 ec 98 00 00 00 sub $0x98,%rsp 399d27: b9 10 00 00 00 mov $0x10,%ecx 399d2c: 64 48 8b 04 25 28 00 mov %fs:0x28,%rax 399d33: 00 00 399d35: 48 89 84 24 88 00 00 mov %rax,0x88(%rsp) 399d3c: 00 399d3d: 31 c0 xor %eax,%eax 399d3f: 48 89 e7 mov %rsp,%rdi 399d42: fc cld 399d43: f3 48 ab rep stos %rax,%es:(%rdi) 399d46: 48 8b 84 24 88 00 00 mov 0x88(%rsp),%rax 399d4d: 00 399d4e: 64 48 33 04 25 28 00 xor %fs:0x28,%rax 399d55: 00 00 399d57: 75 0d jne 399d66 <sanity_test_fdelt()+0x46> 399d59: b8 01 00 00 00 mov $0x1,%eax 399d5e: 48 81 c4 98 00 00 00 add $0x98,%rsp 399d65: c3 retq 399d66: e8 85 df c8 ff callq 27cf0 <__stack_chk_fail@plt> 399d6b: 0f 1f 44 00 00 nopl 0x0(%rax,%rax,1) ``` To test, you could modify this test to pass -1 to FD_SET, and check that a call to fdelt_warn() is inserted, and that running bitcoind fails. i.e: ```bash 0000000000399d20 <sanity_test_fdelt()>: 399d20: 48 81 ec 98 00 00 00 sub $0x98,%rsp 399d27: b9 10 00 00 00 mov $0x10,%ecx 399d2c: 64 48 8b 04 25 28 00 mov %fs:0x28,%rax 399d33: 00 00 399d35: 48 89 84 24 88 00 00 mov %rax,0x88(%rsp) 399d3c: 00 399d3d: 31 c0 xor %eax,%eax 399d3f: 48 89 e7 mov %rsp,%rdi 399d42: fc cld 399d43: f3 48 ab rep stos %rax,%es:(%rdi) 399d46: 48 c7 c7 ff ff ff ff mov $0xffffffffffffffff,%rdi 399d4d: e8 3e ff ff ff callq 399c90 <__fdelt_warn> 399d52: 0f b6 04 24 movzbl (%rsp),%eax 399d56: 83 e0 01 and $0x1,%eax 399d59: 48 8b 94 24 88 00 00 mov 0x88(%rsp),%rdx 399d60: 00 399d61: 64 48 33 14 25 28 00 xor %fs:0x28,%rdx 399d68: 00 00 399d6a: 75 08 jne 399d74 <sanity_test_fdelt()+0x54> 399d6c: 48 81 c4 98 00 00 00 add $0x98,%rsp 399d73: c3 retq 399d74: e8 77 df c8 ff callq 27cf0 <__stack_chk_fail@plt> 399d79: 0f 1f 80 00 00 00 00 nopl 0x0(%rax) ``` ```bash ./src/bitcoind *** buffer overflow detected ***: src/bitcoind terminated Aborted ```
2020-04-20Merge #18190: tests: Add fuzzing harness for Golomb-Rice coding ↵MarcoFalke
(GolombRiceEncode/GolombRiceDecode) 69749fbe6a95f45eb7a695a5f89be87e55c91fb8 tests: Add fuzzing harness for Golomb-Rice coding (GolombRiceEncode/GolombRiceDecode) (practicalswift) Pull request description: Add fuzzing harness for Golomb-Rice coding (`GolombRiceEncode`/`GolombRiceDecode`). Test this PR using: ``` $ make distclean $ ./autogen.sh $ CC=clang CXX=clang++ ./configure --enable-fuzz \ --with-sanitizers=address,fuzzer,undefined $ make $ src/test/fuzz/golomb_rice … ``` Top commit has no ACKs. Tree-SHA512: 1b26512301b8c22ab3b804d9b9e4baf933f26f8c05e462d583863badcec7e694548a34849a0d7c4ff7d58b19f6338b51819976ecf642bc4659b04ef71182d748
2020-04-20tests: Add fuzzing harness for Golomb-Rice coding ↵practicalswift
(GolombRiceEncode/GolombRiceDecode)
2020-04-10Revert "Merge #16367: Multiprocess build support"MarcoFalke
This reverts the changes made in merge commit 1b307613604883daea4913a65da30ae073c9dc4d: This reverts commit b919efadff3d0393f4a8c3c1dc735f7ac5c665bb. This reverts commit d54f64c6c700be0604190f52c84fc5f1cdd9f02f. This reverts commit 787f40668dc15980c3d6de028d7950c08175d84a. This reverts commit d6306466626635e6fee44385e6a688c8dc118eb5. This reverts commit e6e44eedd56ecaf59f3fabf8e07ab7dee0ddb1b6.
2020-04-10Merge #16367: Multiprocess build supportMarcoFalke
b919efadff3d0393f4a8c3c1dc735f7ac5c665bb depends: Use default macos clang compiler (Russell Yanofsky) d54f64c6c700be0604190f52c84fc5f1cdd9f02f Add multiprocess travis configuration (Russell Yanofsky) 787f40668dc15980c3d6de028d7950c08175d84a Set LD_LIBRARY_PATH consistently in travis tests (Russell Yanofsky) d6306466626635e6fee44385e6a688c8dc118eb5 libmultiprocess depends build (Russell Yanofsky) e6e44eedd56ecaf59f3fabf8e07ab7dee0ddb1b6 Multiprocess build changes (Russell Yanofsky) Pull request description: This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10). This splits autotools, depends build, and travis changes out of #10102, so code changes and build system changes can be reviewed separately. ACKs for top commit: hebasto: re-ACK b919efadff3d0393f4a8c3c1dc735f7ac5c665bb, rebased only since my [previous](https://github.com/bitcoin/bitcoin/pull/16367#issuecomment-605514556) review. Tree-SHA512: ebc5e403cc99a0d9629ed7fe1595e01d57e6d1255cbf03968a3196ff6f528f734c78060fdc065724ee1f923bcc5aa2b29470fcb36a7f15957eb57c76d58178a4
2020-04-10Merge #18504: build: Drop bitcoin-tx and bitcoin-wallet dependencies on libeventMarcoFalke
01a3392b1b778fa4fcf568013326d6ea1de4fb3b Drop bitcoin-wallet dependency on libevent (Russell Yanofsky) 0660119ac372c2863d14060ac1bc9bc243771f94 Drop unintended bitcoin-tx dependency on libevent (Russell Yanofsky) Pull request description: This fixes compile errors trying to build bitcoin-tx and bitcoin-wallet without libevent, which were reported by Luke Dashjr in https://github.com/bitcoin/bitcoin/issues/18465 The fix avoiding `bitcoin-tx` dependency on libevent just adds a conditional build rule. This is implemented in the first commit (more details in commit description). The fix avoiding `bitcoin-wallet` dependency on libevent requires minor code changes, because `bitcoin-wallet` (unlike `bitcoin-tx`) links against code that calls `urlDecode` / `evhttp_uridecode`. This fix is implemented in the second commit (again details in the commit description). ACKs for top commit: jonasschnelli: utACK 01a3392b1b778fa4fcf568013326d6ea1de4fb3b. Tree-SHA512: d2245e912ab494cccceeb427a1eca8e55b01a0006ff93eebcfb5461ae7cecd1083ac2de443d9db036b18bdc6f0fb615546caaa20c585046f66d234937f74870a
2020-04-05Multiprocess build changesRussell Yanofsky
autotools and automake changes to support multiprocess execution. This adds a new --enable-multiprocess flag, and build configuration code to detect libraries needed for multiprocess support. The --enable-multiprocess flag builds new bitcoin-node and bitcoin-gui executables, which are updated in https://github.com/bitcoin/bitcoin/pull/10102 to communicate across processes. But for now they are functionally equivalent to existing bitcoind and bitcoin-qt executables.
2020-04-05build: create test_fuzz library from src/test/fuzz/fuzz.cppHarris
2020-04-02Drop unintended bitcoin-tx dependency on libeventRussell Yanofsky
Don't include util/url.cpp to libbitcoin_util.a when libevent isn't available. This fixes a compile error trying to build bitcoin-tx without libevent reported by Luke Dashjr in https://github.com/bitcoin/bitcoin/issues/18465 Fixes #18465
2020-03-22scripts: add PE dylib checking to symbol-check.pyfanquake
2020-03-09Move FillPSBT to be a member of CWalletAndrew Chow
2020-03-06Drop unused reverselock.hAnthony Towns
2020-02-27Refactor FormatStateMessage into ValidationStateJeffrey Czyz
2020-02-14Deduplicate the message verifying codeVasil Dimov
The logic of verifying a message was duplicated in 2 places: src/qt/signverifymessagedialog.cpp SignVerifyMessageDialog::on_verifyMessageButton_VM_clicked() src/rpc/misc.cpp verifymessage() with the only difference being the result handling. Move the logic into a dedicated src/util/message.cpp MessageVerify() which returns a set of result codes, call it from the 2 places and just handle the results differently in the callers.
2020-02-10Merge #17398: build: Update leveldb to 1.22+Wladimir J. van der Laan
677fb8e92380d4deb6a3753047c01f7cf7b5af91 test: Add ubsan surpression for crc32c (Wladimir J. van der Laan) 8e68bb1ddeca504bedd40aee8492b5478a88c1e5 build: Disable msvc warning 4722 for leveldb build (Aaron Clauson) be23949765e1b2e050574c6c2a136658a89dee5d build: MSVC changes for leveldb update (Aaron Clauson) 9ebdf047578f0da7e6578d0c51c32f55e84ac157 build: CRC32C build system integration (Wladimir J. van der Laan) 402252a8081e25f22aa1a5c60708714cf1d84ec4 build: Add LCOV exception for crc32c (Wladimir J. van der Laan) 3a037d0067c2c12a1c2c800fb85613a0a2911253 test: Add crc32c exception to various linters and generation scripts (Wladimir J. van der Laan) 84ff1b2076ef91ce688930d0aa0a7f4078ef3e1d test: Add crc32c to subtree check linter (Wladimir J. van der Laan) 7cf13a513409c18d18dff2f6203b3630937b487d doc: Add crc32c subtree to developer notes (Wladimir J. van der Laan) 24d02a9ac00a82d172b171f73554a882df264c80 build: Update build system for new leveldb (Wladimir J. van der Laan) 2e1819311a59fb5cb26e3ca50a510bfe01358350 Squashed 'src/crc32c/' content from commit 224988680f7673cd7c769963d4035cb315aa3388 (Wladimir J. van der Laan) 66480821b36c839ab7615cb9309850015bceadb0 Squashed 'src/leveldb/' changes from f545dfabff4c2e9836efed094dba99a34fbc6b88..f8ae182c1e5176d12e816fb2217ae33a5472fdd7 (Wladimir J. van der Laan) Pull request description: This updates leveldb to currently newest upstream commit https://github.com/bitcoin-core/leveldb/commit/0c40829872a9f00f38e11dc370ff8adb3e19f25b: - CRC32C hardware acceleration is now an external library [crc32c](https://github.com/google/crc32c). This adds acceleration on ARM, and should be faster on x86 because of using prefetch. It also makes it easy to support similar instruction sets on other platforms in the future. - Thread handling uses C++11, instead of platform specific code. - Native windows environment was added. No need to maintain our own hacky one, anymore. - Upstream now builds using CMake. This doesn't mean we need to use that (phew), but internal configuration changed to a a series of checks, instead of OS profiles. This means the blanket error "Cannot build leveldb for $host. Please file a bug report' is removed. All changes: https://github.com/google/leveldb/compare/a53934a3ae1244679f812d998a4f16f2c7f309a6...0c40829872a9f00f38e11dc370ff8adb3e19f25b Pretty much all our changes have been subsumed by upstream, so we figured it was cleaner to start over with a new branch from upstream with the still-relevant patches applied: https://github.com/bitcoin-core/leveldb/tree/bitcoin-fork-new There's quite some testing to be done (see below). See https://github.com/bitcoin-core/leveldb/issues/25 and https://github.com/bitcoin-core/leveldb/pull/26 for more history and context. TODO: - [x] Subtree `crc32c` - [x] Make linters happy about crc32 subtree - [x] Integrate `crc32c` library into build system - [x] MSVC build system ACKs for top commit: sipa: ACK 677fb8e92380d4deb6a3753047c01f7cf7b5af91 Tree-SHA512: 37ee92a750e053e924bc4626b12bb3fd81faa9f8c5ebaa343931fee810c45ba05aa6051fdea82535fa351bf2be7297801b98af9469865fc5ead771650a5d6240
2020-01-29Merge #16702: p2p: supplying and using asmap to improve IP bucketing in addrmanWladimir J. van der Laan
3c1bc40205a3fcab606e70b0e3c13d68b2860e34 Add extra logging of asmap use and bucketing (Gleb Naumenko) e4658aa8eaf1629dd5af8cf7b9717a8e72028251 Return mapped AS in RPC call getpeerinfo (Gleb Naumenko) ec45646de9e62b3d42c85716bfeb06d8f2b507dc Integrate ASN bucketing in Addrman and add tests (Gleb Naumenko) 8feb4e4b667361bf23344149c01594abebd56fdb Add asmap utility which queries a mapping (Gleb Naumenko) Pull request description: This PR attempts to solve the problem explained in #16599. A particular attack which encouraged us to work on this issue is explained here [[Erebus Attack against Bitcoin Peer-to-Peer Network](https://erebus-attack.comp.nus.edu.sg/)] (by @muoitranduc) Instead of relying on /16 prefix to diversify the connections every node creates, we would instead rely on the (ip -> ASN) mapping, if this mapping is provided. A .map file can be created by every user independently based on a router dump, or provided along with the Bitcoin release. Currently we use the python scripts written by @sipa to create a .map file, which is no larger than 2MB (awesome!). Here I suggest adding a field to peers.dat which would represent a hash of asmap file used while serializing addrman (or 0 for /16 prefix legacy approach). In this case, every time the file is updated (or grouping method changed), all buckets will be re-computed. I believe that alternative selective re-bucketing for only updated ranges would require substantial changes. TODO: - ~~more unit tests~~ - ~~find a way to test the code without including >1 MB mapping file in the repo.~~ - find a way to check that mapping file is not corrupted (checksum?) - comments and separate tests for asmap.cpp - make python code for .map generation public - figure out asmap distribution (?) ~Interesting corner case: I’m using std::hash to compute a fingerprint of asmap, and std::hash returns size_t. I guess if a user updates the OS to 64-bit, then the hash of asap will change? Does it even matter?~ ACKs for top commit: laanwj: re-ACK 3c1bc40205a3fcab606e70b0e3c13d68b2860e34 jamesob: ACK 3c1bc40205a3fcab606e70b0e3c13d68b2860e34 ([`jamesob/ackr/16702.3.naumenkogs.p2p_supplying_and_using`](https://github.com/jamesob/bitcoin/tree/ackr/16702.3.naumenkogs.p2p_supplying_and_using)) jonatack: ACK 3c1bc40205a3fcab606e70b0e3c13d68b2860e34 Tree-SHA512: e2dc6171188d5cdc2ab2c022fa49ed73a14a0acb8ae4c5ffa970172a0365942a249ad3d57e5fb134bc156a3492662c983f74bd21e78d316629dcadf71576800c
2020-01-28build: CRC32C build system integrationWladimir J. van der Laan
2020-01-22Merge #17863: scripts: Add MACHO dylib checks to symbol-check.pyWladimir J. van der Laan
c491368d8cfddf3a5b6d574f10ed67492fcecbed scripts: add MACHO dylib checking to symbol-check.py (fanquake) 76bf97213f4b153dd3ccf1314088a73c4804601d scripts: fix check-symbols & check-security argument passing (fanquake) Pull request description: Based on #17857. This adds dynamic library checks for MACHO executables to symbol-check.py. The script has been modified to function more like `security-check.py`. The error output is now also slightly different. i.e: ```bash # Linux x86 bitcoin-cli: symbol operator new[](unsigned long) from unsupported version GLIBCXX_3.4 bitcoin-cli: export of symbol vtable for std::basic_ios<char, std::char_traits<char> > not allowed bitcoin-cli: NEEDED library libstdc++.so.6 is not allowed bitcoin-cli: failed IMPORTED_SYMBOLS EXPORTED_SYMBOLS LIBRARY_DEPENDENCIES # RISCV (skips exported symbols checks) bitcoin-tx: symbol operator new[](unsigned long) from unsupported version GLIBCXX_3.4 bitcoin-tx: NEEDED library libstdc++.so.6 is not allowed bitcoin-tx: failed IMPORTED_SYMBOLS LIBRARY_DEPENDENCIES # macOS Checking macOS dynamic libraries... libboost_filesystem.dylib is not in ALLOWED_LIBRARIES! bitcoind: failed DYNAMIC_LIBRARIES ``` Compared to `v0.19.0.1` the macOS allowed dylibs has been slimmed down somewhat: ```diff src/qt/bitcoin-qt: /usr/lib/libSystem.B.dylib -/System/Library/Frameworks/DiskArbitration.framework/Versions/A/DiskArbitration /System/Library/Frameworks/IOKit.framework/Versions/A/IOKit /System/Library/Frameworks/Foundation.framework/Versions/C/Foundation /System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices /System/Library/Frameworks/AppKit.framework/Versions/C/AppKit /System/Library/Frameworks/ApplicationServices.framework/Versions/A/ApplicationServices /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation -/System/Library/Frameworks/Security.framework/Versions/A/Security -/System/Library/Frameworks/SystemConfiguration.framework/Versions/A/SystemConfiguration /System/Library/Frameworks/CoreGraphics.framework/Versions/A/CoreGraphics -/System/Library/Frameworks/OpenGL.framework/Versions/A/OpenGL -/System/Library/Frameworks/AGL.framework/Versions/A/AGL /System/Library/Frameworks/Carbon.framework/Versions/A/Carbon /usr/lib/libc++.1.dylib -/System/Library/Frameworks/CFNetwork.framework/Versions/A/CFNetwork /System/Library/Frameworks/CoreText.framework/Versions/A/CoreText /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO /usr/lib/libobjc.A.dylib ``` ACKs for top commit: laanwj: ACK c491368d8cfddf3a5b6d574f10ed67492fcecbed Tree-SHA512: f8624e4964e80b3e0d34e8d3cc33f3107938f3ef7a01c07828f09b902b5ea31a53c50f9be03576e1896ed832cf2c399e03a7943a4f537a1e1c705f3804aed979
2020-01-12build: remove double LIBBITCOIN_SERVER linkingfanquake
2020-01-04scripts: add MACHO dylib checking to symbol-check.pyfanquake
2020-01-04scripts: fix check-symbols & check-security argument passingfanquake
The first argument in bin_PROGRAMS (bitcoind) was being silently dropped and never passed into the check-security.py or check-symbols.py scripts. This has been the case since the scripts were added to the makefile in https://github.com/bitcoin/bitcoin/commit/f3d3eaf78eb51238d799d8f20a585550d1567719. Example of the behavior: ```python # touch a, touch b, touch c # python3 args.py < a b c import sys if __name__ == '__main__': print(sys.argv) # ['args.py', 'b', 'c'] # if you add some lines to "a", # you'll see them here.. for line in sys.stdin: print(line) ```
2020-01-03scripts: fix check-symbols & check-security argument passingfanquake
The first argument in bin_PROGRAMS (bitcoind) was being silently dropped and never passed into the check-security.py or check-symbols.py scripts. This has been the case since the scripts were added to the makefile in https://github.com/bitcoin/bitcoin/commit/f3d3eaf78eb51238d799d8f20a585550d1567719. Example of the behavior: ```python # touch a, touch b, touch c # python3 args.py < a b c import sys if __name__ == '__main__': print(sys.argv) # ['args.py', 'b', 'c'] # if you add some lines to "a", # you'll see them here.. for line in sys.stdin: print(line) ```
2020-01-02scripts: add MACHO PIE check to security-check.pyfanquake
2019-12-18 Add asmap utility which queries a mappingGleb Naumenko
The scripts for creating a compact IP->ASN mapping are here: https://github.com/sipa/asmap Co-authored-by: Pieter Wuille <pieter.wuille@gmail.com>
2019-11-21build: Create test utility library from src/test/util/Harris
2019-11-18build: remove OpenSSL detection and libsfanquake
2019-11-18Merge #17270: Feed environment data into RNG initializersWladimir J. van der Laan
d1c02775aa74a0610809ac54bb241ddad61d2d8c Report amount of data gathered from environment (Pieter Wuille) 64e1e022cedf6776c5dffd488ca2e766adca5dc3 Use thread-safe atomic in perfmon seeder (Pieter Wuille) d61f2bb076d8f17840a8e79f1583d7f6e3e6d09a Run background seeding periodically instead of unpredictably (Pieter Wuille) 483b94292e89587e5ab40a30b8a90e2f56e847f3 Add information gathered through getauxval() (Pieter Wuille) 11793ea22e1298fa7d3b44a5b6d20830248d8cf4 Feed CPUID data into RNG (Pieter Wuille) a81c494b4c9a8c2f1a319a03375826f12863706f Use sysctl for seeding on MacOS/BSD (Pieter Wuille) 2554c1b81bb8c40e1989025c6f18e7935720b156 Gather additional entropy from the environment (Pieter Wuille) c2a262a78c3bcc4d5e13612ab0214874abe15de0 Seed randomness with process id / thread id / various clocks (Pieter Wuille) 723c79666770b30cce9f962bed5ece8cc7d74580 [MOVEONLY] Move cpuid code from random & sha256 to compat/cpuid (Pieter Wuille) cea3902015185adc88adbd031d919f91bc844fd7 [MOVEONLY] Move perfmon data gathering to new randomenv module (Pieter Wuille) b51bae1a5a4fa8ef7825dd1bb09e3f47f96d7a5a doc: minor corrections in random.cpp (fanquake) Pull request description: This introduces a new `randomenv` module that queries varies non-cryptographic (and non-RNG) sources of entropy available on the system; things like user IDs, system configuration, time, statistics, CPUID data. The idea is that these provide a fallback in scenarios where system entropy is somehow broken (note that if system entropy *fails* we will abort regardless; this is only meant to function as a last resort against undetected failure). It includes some data sources OpenSSL currently uses, and more. The separation between random and randomenv is a bit arbitrary, but I felt that all this "non-essential" functionality deserved to be separated from the core random module. ACKs for top commit: TheBlueMatt: utACK d1c02775aa74a0610809ac54bb241ddad61d2d8c. Certainly no longer measuring the time elapsed between a 1ms sleep (which got removed in the latest change) is a fair tradeoff for adding about 2 million other actually-higher-entropy bits :). laanwj: ACK d1c02775aa74a0610809ac54bb241ddad61d2d8c Tree-SHA512: d290a8db6538a164348118ee02079e4f4c8551749ea78fa44b2aad57f5df2ccbc2a12dc7d80d8f3e916d68cdd8e204faf9e1bcbec15f9054eba6b22f17c66ae3
2019-11-12[MOVEONLY] Move cpuid code from random & sha256 to compat/cpuidPieter Wuille
2019-11-12[MOVEONLY] Move perfmon data gathering to new randomenv modulePieter Wuille
2019-11-07Add util::Settings struct and helper functions.Russell Yanofsky
Implement merging of settings from different sources (command line and config file) separately from parsing code in system.cpp, so it is easier to add new sources. Document current inconsistent merging behavior without changing it. This commit only adds new settings code without using it. The next commit calls the new code to replace existing code in system.cpp. Co-authored-by: John Newbery <john@johnnewbery.com>
2019-11-05Merge #16805: logs: add timing information to FlushStateToDisk()Wladimir J. van der Laan
dcef9a2922317cb2849f397366b6c56d755db6c9 logs: add timing information to FlushStateToDisk() (James O'Beirne) 41edaf227a69bc4846d5996675e8763fdfe0f367 logs: add BCLog::Timer and related macros (James O'Beirne) Pull request description: It's currently annoying to detect FlushStateToDisk() calls when benchmarking since they have to be inferred from a drop in coins count from the `UpdateTip: ` log messages. This adds a new logging utility, `BCLog::Timer`, and some related macros that are generally useful for printing timing-related logging messages, and a message that is unconditionally written when the coins cache is flushed to disk. ``` 2019-09-04T20:17:51Z FlushStateToDisk: write block and undo data to disk completed (3ms) 2019-09-04T20:17:51Z FlushStateToDisk: write block index to disk completed (370ms) 2019-09-04T20:17:51Z FlushStateToDisk: write coins cache to disk (2068451 coins, 294967kB) completed (21481ms) ``` ACKs for top commit: laanwj: Thanks, ACK dcef9a2922317cb2849f397366b6c56d755db6c9 ryanofsky: Code review ACK dcef9a2922317cb2849f397366b6c56d755db6c9. No changes since last review other than moving code to new timer.h header Tree-SHA512: 6d61e48a062d3edb48d0e056a6f0b1f8031773cc99289ee4544f8349d24526b88519e1e304009d56e428f1eaf76c857bf8e7e1c0b6873a6f270306accb5edc3d
2019-11-05Merge #16899: UTXO snapshot creation (dumptxoutset)Wladimir J. van der Laan
92b2f5306ba0b3f031293cb8f415b67cb002c2f1 test: add dumptxoutset RPC test (James O'Beirne) c1ccbc3ddef931896a7e9dcfa6704e305a69fbff devtools: add utxo_snapshot.sh (James O'Beirne) 57cf74c9918d10c69a46e6ceb3cb1a5e04edf5bc rpc: add dumptxoutset (James O'Beirne) 92fafb3a7da66f737e960e541fcfbcadedf6043a coinstats: add coins_count (James O'Beirne) 707fde7b9ba522c22179e2db0ed7b462c65138d9 add unused SnapshotMetadata class (James O'Beirne) Pull request description: This is part of the [assumeutxo project](https://github.com/bitcoin/bitcoin/projects/11): Parent PR: #15606 Issue: #15605 Specification: https://github.com/jamesob/assumeutxo-docs/tree/master/proposal --- This changeset defines the serialization format for UTXO snapshots and adds an RPC command for creating them, `dumptxoutset`. It also adds a convenience script for generating and verifying snapshots at a certain height, since that requires doing a hacky rewind of the chain via `invalidateblock`. All of this is unused at the moment. ACKs for top commit: laanwj: ACK 92b2f5306ba0b3f031293cb8f415b67cb002c2f1 Tree-SHA512: 200dff87767f157d627e99506ec543465d9329860a6cd49363081619c437163a640a46d008faa92b1f44fd403bfc7a7c9e851c658b5a4849efa9a34ca976bf31
2019-11-04logs: add BCLog::Timer and related macrosJames O'Beirne
Makes logging timing information about a block of code easier.
2019-11-04Merge #17297: refactor: Remove addrdb.h dependency from node.hWladimir J. van der Laan
f44abe4bed25a40145ab168adc1589f5df4146f3 refactor: Remove addrdb.h dependency from node.h (Hennadii Stepanov) Pull request description: `node.h` includes `addrdb.h` just for the sake of `banmap_t` type. This PR makes dependencies simpler and explicit. ~Also needless `typedef` has been removed from `enum BanReason`.~ ACKs for top commit: laanwj: ACK f44abe4bed25a40145ab168adc1589f5df4146f3 practicalswift: ACK f44abe4bed25a40145ab168adc1589f5df4146f3 Tree-SHA512: 33a1be20e5c629daf4a61ebbf93ea6494b9256887cebd4974de4782f6d324404b6cc84909533d9502b2cc19902083f1f9307d4fb7231e67db5b412b842d13072
2019-10-30Merge #16839: Replace Connman and BanMan globals with NodeContext localWladimir J. van der Laan
362ded410b8cb1104b7ef31ff8488fec4824a7d5 Avoid using g_rpc_node global in wallet code (Russell Yanofsky) 8922d7f6b751a3e6b3b9f6fb7961c442877fb65a scripted-diff: Remove g_connman, g_banman globals (Russell Yanofsky) e6f4f895d5e42feaf7bfa5f41e80292aaa73cd7d Pass NodeContext, ConnMan, BanMan references more places (Russell Yanofsky) 4d5448c76b71c9d91399c31b043237091be2e5e7 MOVEONLY: Move NodeContext struct to node/context.h (Russell Yanofsky) 301bd41a2e6765b185bd55f4c541f9e27aeea29d scripted-diff: Rename InitInterfaces to NodeContext (Russell Yanofsky) Pull request description: This change is mainly a naming / organization change intended to simplify #10102. It: - Renames struct InitInterfaces to struct NodeContext and moves it from src/init.h to src/node/context.h. This is a cosmetic change intended to make the point of the struct more obvious. - Gets rid of BanMan and ConnMan globals making them NodeContext members instead. Getting rid of these globals has been talked about in past as a way to implement testing and simulations. Making them NodeContext members is a way of keeping them accessible without the globals. - Splits g_rpc_interfaces global into g_rpc_node and g_rpc_chain globals. This better separates node and wallet rpc methods. Node RPC methods should have access NodeContext, while wallet RPC methods should only have indirect access to node functionality via interfaces::Chain. - Adds NodeContext& references to interfaces::Chain class and the interfaces::MakeChain() function. This is needed to access ConnMan and BanMan instances without the globals. - Gets rid of redundant Node and Chain instances in Qt tests. This is needed due to the previous MakeChain change, and also makes test setup a little more straightforward. More cleanup could be done in the future, but it will require deduplication of bitcoind, bitcoin-qt, and TestingSetup init code. ACKs for top commit: laanwj: ACK 362ded410b8cb1104b7ef31ff8488fec4824a7d5 Tree-SHA512: 9ae6ff1e33423291d1e52056bac95e0874538390892a6e83c4c115b3c73155a8827c0191b46eb3d14e3b3f6c23ccb08095490880fbc3188026319c71739f7db2
2019-10-29add unused SnapshotMetadata classJames O'Beirne
2019-10-29Merge #17260: Split some CWallet functions into new LegacyScriptPubKeyManMarcoFalke
f201ba59ffd2e071a36a688b80d2cff9a9c44bb2 Refactor: Split up CWallet and LegacyScriptPubKeyMan and classes (Andrew Chow) 6702048f91089d7a565e5ca5f7c8dcd2ca405a85 MOVEONLY: Move key handling code out of wallet to keyman file (Andrew Chow) ab053ec6d1e766402f88947d29cd875a285e7280 Move wallet enums to walletutil.h (Andrew Chow) Pull request description: Moves key management functions into a new class LegacyScriptPubKeyMan. First two commits are move-only commits which move stuff out of wallet.{h/cpp} and into newly created scriptpubkeyman.{h/cpp}. Third commit changes several things in CWallet to use LegacyScriptPubKeyMan. First step in the wallet boxes refactor. Note that LegacyScriptPubKeyMan and ScriptPubKeyMan cannot be used standalone yet and are still very much tied into CWallet with both accessing functions within each other. This PR is to help reduce review burden. ACKs for top commit: Sjors: Code review ACK f201ba5. promag: Code review ACK f201ba59ffd2e071a36a688b80d2cff9a9c44bb2. ryanofsky: Code review ACK f201ba59ffd2e071a36a688b80d2cff9a9c44bb2 MarcoFalke: ACK f201ba59ffd2e071a36a688b80d2cff9a9c44bb2 Tree-SHA512: bdc0d8595a06233fe003afcf968a38e0e8cc584a6a89c5bcd05309ac29dca852391802d46763ef81a108d146d0f40c79ea5438e87234ed12b4b8360c9aec94c0
2019-10-29refactor: Remove addrdb.h dependency from node.hHennadii Stepanov
2019-10-28Pass NodeContext, ConnMan, BanMan references more placesRussell Yanofsky
So g_connman and g_banman globals can be removed next commit.
2019-10-28MOVEONLY: Move NodeContext struct to node/context.hRussell Yanofsky
2019-10-28Merge #17192: util: Add CHECK_NONFATAL and use it in src/rpcWladimir J. van der Laan
faeb6665362e35f573ad715ade0ef2db62d71839 util: Add CHECK_NONFATAL and use it in src/rpc (MarcoFalke) Pull request description: Fixes #17181 Currently, we use `assert` in RPC code to document logic and code assumptions. However, it seems a bit extreme to abort all of Bitcoin Core on an assert failure in one of the RPC threads. I suggest to replace all `assert`s with a macro `CHECK_NONFATAL(condition)` that throws a runtime error when the condition evaluates to `false`. That runtime error will then be returned to the rpc caller and will include instructions to report the error to our issue tracker. ACKs for top commit: practicalswift: ACK faeb6665362e35f573ad715ade0ef2db62d71839 laanwj: ACK faeb6665362e35f573ad715ade0ef2db62d71839 ryanofsky: Code review ACK faeb6665362e35f573ad715ade0ef2db62d71839 Tree-SHA512: 9b748715a5e0767ac11f1324a95a3a6ec672a0e0658013492219223bda83ce4b1b447fd8183bbb235f7df5ef7dddda7666ad569544b4d61cc65f232ca7a800ec
2019-10-25MOVEONLY: Move key handling code out of wallet to keyman fileAndrew Chow
Start moving wallet and ismine code to scriptpubkeyman.h, scriptpubkeyman.cpp The easiest way to review this commit is to run: git log -p -n1 --color-moved=dimmed_zebra And check that everything is a move (other than includes and copyrights comments). This commit is move-only and doesn't change code or affect behavior.
generated by cgit v1.2.3 (git 2.26.2) at 2024-11-01 07:19:08 +0000