aboutsummaryrefslogtreecommitdiff
path: root/src/Makefile.am
AgeCommit message (Collapse)Author
2021-03-29remove unused class util::Ref and its unit testSebastian Falbesoner
2021-03-17Merge #21415: refactor: remove Optional & nulloptWladimir J. van der Laan
ebc4ab721b0371c0ef217c0f5bd7d42613e951e6 refactor: post Optional<> removal cleanups (fanquake) 57e980d13ca488031bde6ef197cf34d493d36796 scripted-diff: remove Optional & nullopt (fanquake) Pull request description: Same rationale & motivation as #21404, which turned out to be quite low in the number of potential conflicts. Lets see what the bot has to say here. ACKs for top commit: practicalswift: cr ACK ebc4ab721b0371c0ef217c0f5bd7d42613e951e6: patch looks correct jnewbery: utACK ebc4ab721b0371c0ef217c0f5bd7d42613e951e6 laanwj: Code review ACK ebc4ab721b0371c0ef217c0f5bd7d42613e951e6 Tree-SHA512: 550fbeef09b9d35ddefaa805d1755c18c8fd499c4b0f77ebfece8c20296a7abd1cf6c699e2261f92fe3552deeb7555ec2a2287ffe3ab9e98bb9f8612a4d43be3
2021-03-16Merge #21405: compat: remove memcpy -> memmove backwards compatibility aliasWladimir J. van der Laan
52f0be3a933225429b9b59b58de4fe54c2b20a02 compat: remove memcpy -> memmove backwards compatibility alias (fanquake) Pull request description: In glib 2.13 memcpy was changed such that the way it copied bytes was reversed. This caused all sorts of issues for existing software, which depended on the existing behavior (when they should have been using memmove). See: https://sourceware.org/bugzilla/show_bug.cgi?id=12518 https://bugzilla.redhat.com/show_bug.cgi?id=638477 Now that we require glibc 2.17+ (#17538), we should be well clear of having to maintain our memcpy -> memmove aliasing, which was introduced in #4339. Gitian builds: ```bash # Linux: 52dee59c8c7d5620ac9b140b79fcaf3d2f15a219293140190f9283ba871f5391 bitcoin-52f0be3a9332-aarch64-linux-gnu-debug.tar.gz 8963473b8791c5c6033a992d7dd761832fe1fb5732be790a6e9f8c11d67ad8ae bitcoin-52f0be3a9332-aarch64-linux-gnu.tar.gz 1fb3365c1ef60ecd1eb2d18f671f8f1e8cde0585de7de74aa0c5121093100c26 bitcoin-52f0be3a9332-arm-linux-gnueabihf-debug.tar.gz 305c5b032d51ba97459715211112204a09d119edd6ec2a12b796559ad3fde761 bitcoin-52f0be3a9332-arm-linux-gnueabihf.tar.gz 1f950a3e3979a4e1a67696b3fddc3090a0489a43b49e2b58a348d4b02ada2aa8 bitcoin-52f0be3a9332-powerpc64-linux-gnu-debug.tar.gz 0b9731dba768b30c91dadec4cd7a98c86e06fbf6354555f798b46b7c4fab7b5f bitcoin-52f0be3a9332-powerpc64-linux-gnu.tar.gz c4a37aae56cc023964f8d9e82d1b66913079cab559cbfc1c9127969aa968a06f bitcoin-52f0be3a9332-powerpc64le-linux-gnu-debug.tar.gz dfbaa4f3bf12988a0a7f82c4b10162e5e7a63382a7e29d0170bc32ce344c97c3 bitcoin-52f0be3a9332-powerpc64le-linux-gnu.tar.gz 3a0280d2c06516e50b0841d6f42d9589355dc9a1f8bb9a0b123554cd91b08004 bitcoin-52f0be3a9332-riscv64-linux-gnu-debug.tar.gz cc199a0f254b2366e80a6a884120ec3ea442983990ba1a5eb993c36060686eba bitcoin-52f0be3a9332-riscv64-linux-gnu.tar.gz eb8e7ca673cc06c167ab082fe457a41f73758ecd5b34941300e3cd378c29b197 bitcoin-52f0be3a9332-x86_64-linux-gnu-debug.tar.gz dad19226c0e4c54b78ca2fa85fc28c5bfd1e1178e3f765472bd2f895a1d57145 bitcoin-52f0be3a9332-x86_64-linux-gnu.tar.gz ef89be95b84bb7c6fef055634cd20caf2fa5b42441502918dbfbf758bb2daab6 src/bitcoin-52f0be3a9332.tar.gz dc61f5ca33330c1609bc56b23f39fef3c1ff5ec6a1799d5b7a18f3c3b3acc9f9 bitcoin-core-linux-22-res.yml ``` Guix builds: ```bash b50d6399cb59e5e4a9247b12a3eda61de6e51bd87ef1f27b388b75b71dfccf92 output/bitcoin-52f0be3a9332-aarch64-linux-gnu-debug.tar.gz 23d845dc13e60a581ebdfbaa6063f559a56cce06734e1b50790d2fc13e257793 output/bitcoin-52f0be3a9332-aarch64-linux-gnu.tar.gz 79094406fe00939bbce17a6d65de5a2686625e871432350c69e674cc80b1491c output/bitcoin-52f0be3a9332-arm-linux-gnueabihf-debug.tar.gz 65a91913249a743015eceea5a56c497d606af17270cb7e8a3df10cf729b757ec output/bitcoin-52f0be3a9332-arm-linux-gnueabihf.tar.gz 5e75ca5e8cf6934ba5a5a1b4d26c1b361b118e10ef34b73845d038035ddb9b85 output/bitcoin-52f0be3a9332-osx-unsigned.dmg 774b372696cde8ceab40f6909dadea3fc87b375b495fcfb4ee8a963afd7fbd3a output/bitcoin-52f0be3a9332-osx-unsigned.tar.gz dc4bdfb7b32dcc0b6e876d6d7ab3cb8d1472f21f66546ab70515f96262292e21 output/bitcoin-52f0be3a9332-osx64.tar.gz ea178ff9e28439f80129445cf260215c74eea2e610f62ff045061f287675d3ff output/bitcoin-52f0be3a9332-powerpc64-linux-gnu-debug.tar.gz 0390687a7aaa3f0a8a78be2deab21116599e5b332f00a2d1fdce97a5bd30e3eb output/bitcoin-52f0be3a9332-powerpc64-linux-gnu.tar.gz 52c948719a27f252f5969558abc2718c1e365ea85496322cb4ec97eab8a234cc output/bitcoin-52f0be3a9332-powerpc64le-linux-gnu-debug.tar.gz 5a4a8748dffe7e6a5bd07f3f564b1f2052440c4199fe25aaa41675bfb69e61db output/bitcoin-52f0be3a9332-powerpc64le-linux-gnu.tar.gz ba521bd2b4e73aea317821a9e08da9a326c0be3b38d923b35ba14bc68ee6c814 output/bitcoin-52f0be3a9332-riscv64-linux-gnu-debug.tar.gz 783ea81ab2f6b642b13ebf7882aa822d12f95936574a8848a74b1b8978e6801d output/bitcoin-52f0be3a9332-riscv64-linux-gnu.tar.gz 376706fc12e58d7d559a87e1ce64be22eaac3fc32d95c60d603ad893d9128cc1 output/bitcoin-52f0be3a9332-win-unsigned.tar.gz 7aa48242fb71e29b00992b2be8677f1ea49f2ca82c5355bf0c1d4c8d14635596 output/bitcoin-52f0be3a9332-win64-debug.zip 41e6461ab573fa8f6ac0f198193e72a4a047bb7a4193f743b937e81739c929cc output/bitcoin-52f0be3a9332-win64-setup-unsigned.exe e2c4ecb05f24577da12f722d848bf6ac89f3f549d6d2bfd30d65676099c0725b output/bitcoin-52f0be3a9332-win64.zip 60ed63b3b562fa2141f18f1556a03c2474b75797088cd68fdb3e7d057a6983a3 output/bitcoin-52f0be3a9332-x86_64-linux-gnu-debug.tar.gz adb0bb62dc8b99d025a863e921b8e670f4c8f4b5600cd6d79eb552ede10bc8b8 output/bitcoin-52f0be3a9332-x86_64-linux-gnu.tar.gz ef89be95b84bb7c6fef055634cd20caf2fa5b42441502918dbfbf758bb2daab6 output/src/bitcoin-52f0be3a9332.tar.gz ``` ACKs for top commit: laanwj: Concept and code review ACK 52f0be3a933225429b9b59b58de4fe54c2b20a02 Tree-SHA512: 851634a633cc7d27b10f11436768f3695a7615d5850166c3718028c36d3a7dd56baa2dd1028f47802891703e9f5a1d382f559e388ecef2249e2004edc62d97bf
2021-03-15scripted-diff: remove Optional & nulloptfanquake
-BEGIN VERIFY SCRIPT- git rm src/optional.h sed -i -e 's/Optional</std::optional</g' $(git grep -l 'Optional<' src) sed -i -e 's/{nullopt}/{std::nullopt}/g' $(git grep -l 'nullopt' src) sed -i -e 's/ nullopt;/ std::nullopt;/g' $(git grep -l 'nullopt' src) sed -i -e 's/ nullopt)/ std::nullopt)/g' $(git grep -l 'nullopt' src) sed -i -e 's/(nullopt)/(std::nullopt)/g' $(git grep -l 'nullopt' src) sed -i -e 's/ nullopt,/ std::nullopt,/g' $(git grep -l 'nullopt' src) sed -i -e 's/? nullopt :/? std::nullopt :/g' $(git grep -l 'nullopt' src) sed -i -e 's/: nullopt}/: std::nullopt}/g' $(git grep -l 'nullopt' src) sed -i -e '/optional.h \\/d' src/Makefile.am sed -i -e '/#include <optional.h>/d' src/test/fuzz/autofile.cpp src/test/fuzz/buffered_file.cpp src/test/fuzz/node_eviction.cpp sed -i -e 's/#include <optional.h>/#include <optional>/g' $(git grep -l '#include <optional.h>' src) -END VERIFY SCRIPT-
2021-03-12Merge #21404: refactor: Remove MakeUnique<T>()MarcoFalke
1a6323bdbe20bdb7b1c907d8fa0333ffa88b21ff doc: update developer notes for removal of MakeUnique (fanquake) 3ba2840e7ee81341b0748c0121aedc2e9305041a scripted-diff: remove MakeUnique<T>() (fanquake) Pull request description: Since requiring C++17, this is just pointless abstraction. I think we should just "tear the band-aid off" and remove it. Similar to the changes happening in #21366. Also, having a comment saying this is deprecated doesn't prevent it's usage in new code. i.e : https://github.com/bitcoin/bitcoin/pull/20946#discussion_r561949731. The repository is fairly quiet at the moment, so any potential complaints about having to rebase should be minimal. Might as well get this over and done with. ACKs for top commit: jnewbery: utACK 1a6323bdbe20bdb7b1c907d8fa0333ffa88b21ff practicalswift: cr ACK 1a6323bdbe20bdb7b1c907d8fa0333ffa88b21ff: patch looks correct ajtowns: ACK 1a6323bdbe20bdb7b1c907d8fa0333ffa88b21ff -- code review only glozow: ACK https://github.com/bitcoin/bitcoin/commit/1a6323bdbe20bdb7b1c907d8fa0333ffa88b21ff looks correct Tree-SHA512: 4a14b9611b60b9b3026b54d6f5a2dce4c5d9b63a7b93d7de1307512df736503ed84bac66e7b93372c76e3117f49bf9f29cd473d3a47cb41fb2775bc10234736f
2021-03-11Merge #21007: bitcoind: Add -daemonwait option to wait for initializationWladimir J. van der Laan
e017a913d0d78ef0766cf73586fe7a38488e1a26 bitcoind: Add -daemonwait option to wait for initialization (Wladimir J. van der Laan) c3e6fdee6d39d3f52dec421b48a0ac8bad5006f7 shutdown: Use RAII TokenPipe in shutdown (Wladimir J. van der Laan) 612f746a8ffa265b6877bedbbe21fcbb392f1516 util: Add RAII TokenPipe (Wladimir J. van der Laan) Pull request description: This adds a `-daemonwait` flag that does the same as `-daemon` except that it, from a user perspective, backgrounds the process only after initialization is complete. This is similar to the behaviour of some other software such as c-lightning. This can be useful when the process launching bitcoind wants to guarantee that either the RPC server is running, or that initialization failed, before continuing. The exit code indicates the initialization result. The use of the libc function `daemon()` is replaced by a custom implementation which is inspired by the [glibc implementation](https://github.com/lattera/glibc/blob/master/misc/daemon.c#L44), but which also creates a pipe from the child to the parent process for communication. An additional advantage of having our own `daemon()` implementation is that no MACOS-specific pragmas are needed anymore to silence a deprecation warning. TODO: - [x] Factor out `token_read` and `token_write` to an utility, and use them in `shutdown.cpp` as well—this is exactly the same kind of communication mechanism. - [x] RAII-ify pipe endpoints. - [x] Improve granularity of the `configure.ac` checks. This currently still checks for the function `daemon()` which makes no sense as it's not used. It should check for individual functions such as `fork()` and `setsid()` etc—the former being required, the second optional. - [-] ~~Signal propagation during initialization: if say, pressing Ctrl-C during `-daemonwait` it would be good to pass this SIGINT on to the child process instead of detaching the parent process and letting the child run free.~~ This is not necessary, see https://github.com/bitcoin/bitcoin/pull/21007#issuecomment-769007341. Future: - Consider if it makes sense to use this in the RPC tests (there would be no more need for "is RPC ready" polling loops). I think this is out of scope for this PR. ACKs for top commit: jonatack: Tested ACK e017a913d0d78ef0766cf73586fe7a38488e1a26 checked change since previous review is move-only Tree-SHA512: 53369b8ca2247e4cf3af8cb2cfd5b3399e8e0e3296423d64be987004758162a7ddc1287b01a92d7692328edcb2da4cf05d279b1b4ef61a665b71440ab6a6dbe2
2021-03-11scripted-diff: remove MakeUnique<T>()fanquake
-BEGIN VERIFY SCRIPT- git rm src/util/memory.h sed -i -e 's/MakeUnique/std::make_unique/g' $(git grep -l MakeUnique src) sed -i -e '/#include <util\/memory.h>/d' $(git grep -l '#include <util/memory.h>' src) sed -i -e '/util\/memory.h \\/d' src/Makefile.am -END VERIFY SCRIPT-
2021-03-10compat: remove memcpy -> memmove backwards compatibility aliasfanquake
In glib 2.13 memcpy was changed such that the way it copied bytes was reversed. This caused all sorts of issues for existing software, which depended on the existing behavior (when they should have been using memmove). See: https://sourceware.org/bugzilla/show_bug.cgi?id=12518 https://bugzilla.redhat.com/show_bug.cgi?id=638477 Now that we require glibc 2.17+ (#17538), we should be well clear of having to maintain our memcpy -> memmove aliasing, which was introduced in #4339.
2021-03-06Merge #21209: build: use newer source for libnatpmpfanquake
7af25024e9563241c72797d4eeabdf660e548f53 build: compile libnatpmp with -DNATPMP_STATICLIB on Windows (fanquake) ee35745754075d862e1855c77b20a09b260cce75 build: use newer source for libnatpmp (fanquake) Pull request description: The source we are currently using is from 2015. The upstream repo has received a small number of bug fixes and improvements since then. Including one that fixes an issue for Windows users: https://github.com/miniupnp/libnatpmp/pull/13. The source we are currently using is the most recent "official" release, however I don't think it's worth waiting for a new one. The maintainer was prompted to do so in Oct 2020, then again in Jan of this year, and no release has eventuated. Given libnatpmp is a new inclusion into our repository, I think we should be using this newer source. This also cleans up a few warnings we currently see in Windows depends builds: ```bash Extracting libnatpmp... /home/ubuntu/bitcoin/depends/sources/libnatpmp-20150609.tar.gz: OK Preprocessing libnatpmp... Configuring libnatpmp... Building libnatpmp... make[1]: Entering directory '/home/ubuntu/bitcoin/depends/work/build/x86_64-w64-mingw32/libnatpmp/20150609-13efa1beb87' x86_64-w64-mingw32-gcc -Os -fPIC -Wall -DENABLE_STRNATPMPERR -c -o natpmp.o natpmp.c x86_64-w64-mingw32-gcc -Os -fPIC -Wall -DENABLE_STRNATPMPERR -c -o getgateway.o getgateway.c natpmp.c:42: warning: "EWOULDBLOCK" redefined 42 | #define EWOULDBLOCK WSAEWOULDBLOCK | In file included from natpmp.c:38: /usr/share/mingw-w64/include/errno.h:166: note: this is the location of the previous definition 166 | #define EWOULDBLOCK 140 | natpmp.c:43: warning: "ECONNREFUSED" redefined 43 | #define ECONNREFUSED WSAECONNREFUSED | In file included from natpmp.c:38: /usr/share/mingw-w64/include/errno.h:110: note: this is the location of the previous definition 110 | #define ECONNREFUSED 107 | natpmp.c:271:5: warning: ‘readnatpmpresponseorretry’ redeclared without dllimport attribute: previous dllimport ignored [-Wattributes] 271 | int readnatpmpresponseorretry(natpmp_t * p, natpmpresp_t * response) | ^~~~~~~~~~~~~~~~~~~~~~~~~ ar crs libnatpmp.a natpmp.o getgateway.o make[1]: Leaving directory '/home/ubuntu/bitcoin/depends/work/build/x86_64-w64-mingw32/libnatpmp/20150609-13efa1beb87' Staging libnatpmp... Postprocessing libnatpmp... Caching libnatpmp... ``` ACKs for top commit: hebasto: ACK 7af25024e9563241c72797d4eeabdf660e548f53 Tree-SHA512: 6939014ea986149a5bfdd42b516d563a65ae643516e234579d3f28e7c2f877b0270cc4305ae7c7cb131d6d946a6e0aedc84b4cc880a412612a878a333398b9d7
2021-03-04util: Add RAII TokenPipeWladimir J. van der Laan
2021-03-04Merge #21148: Split orphan handling from net_processing into txorphanageWladimir J. van der Laan
5e50e2d1b95e7ca7709a9671ab21f1164b8d0cb8 txorphanage: comment improvements (Anthony Towns) eeeafb324ef6057f40b5c5fdd8464110e809b0f7 net_processing: move AddToCompactExtraTransactions into PeerManagerImpl (Anthony Towns) f8c0688b9490c8d4902530ba3c3b6fbd8b48e0de scripted-diff: Update txorphanage naming convention (Anthony Towns) 6bd4963c069bfd0af420e8a3fb724c3b693a1e76 txorphanage: Move functions and data into class (Anthony Towns) 03257b832debcb1470420d8657d30ba30f4be770 txorphanage: Extract EraseOrphansForBlock (Anthony Towns) 3c4c3c2fdda3a361e3802e97bc3566f815b75de1 net_processing: drop AddOrphanTx (Anthony Towns) 26d1a6ccd5fcc7abec737c0d8c67238561627d59 denialofservices_tests: check txorphanage's AddTx (Anthony Towns) 1041616d7eb66281bb4de51ffbc83df0923b2f7e txorphanage: Extract OrphanageAddTx (Anthony Towns) f294da727413210fda279afdc206a4dd12046d56 txorphanage: Extract GetOrphanTx (Anthony Towns) 83679ffc600305ec0926fd195ee31c11de2ed613 txorphanage: Extract HaveOrphanTx (Anthony Towns) ee135c8d5b39b0cb8b301a83e286285ab926dca7 txorphanage: Extract AddChildrenToWorkSet (Anthony Towns) 38a11c355acfc15134c682571b3d92f66b0e7c3c txorphanage: Add lock annotations (Anthony Towns) 81dd57e5b1ab1afa7e59468e30ef41bd34f0c8d7 txorphanage: Pass uint256 by reference instead of value (Anthony Towns) 9d5313df7eedad8562c822f5477747e924929fd3 move-only: Add txorphanage module (Anthony Towns) Pull request description: Splits orphan handling into its own module and reduces global usage. ACKs for top commit: jnewbery: utACK 5e50e2d1b9 amitiuttarwar: utACK 5e50e2d1b95e7ca7709a9671ab21f1164b8d0cb8 glozow: re ACK https://github.com/bitcoin/bitcoin/pull/21148/commits/5e50e2d1b95e7ca7709a9671ab21f1164b8d0cb8, comment updates laanwj: Code review ACK 5e50e2d1b95e7ca7709a9671ab21f1164b8d0cb8 Tree-SHA512: 92a959bb5dd414c96f78cb8dcaa68adb85faf16b8b843a2cbe0bb2aa08df13ad6bd9424d29b98f57a82ec29c942fbdbea3011883d00bf0b0feb643e295174e46
2021-03-04build: compile libnatpmp with -DNATPMP_STATICLIB on Windowsfanquake
This fixes linking issues and mirrors what we do with miniupnpc.
2021-03-01net: implement the necessary parts of the I2P SAM protocolVasil Dimov
Implement the following commands from the I2P SAM protocol: * HELLO: needed for all of the remaining ones * DEST GENERATE: to generate our private key and destination * NAMING LOOKUP: to convert .i2p addresses to destinations * SESSION CREATE: needed for STREAM CONNECT and STREAM ACCEPT * STREAM CONNECT: to make outgoing connections * STREAM ACCEPT: to accept incoming connections
2021-03-01util: extract {Read,Write}BinaryFile() to its own filesVasil Dimov
Extract `ReadBinaryFile()` and `WriteBinaryFile()` from `torcontrol.cpp` to its own `readwritefile.{h,cpp}` files, so that it can be reused from other modules.
2021-02-26move-only: Add txorphanage moduleAnthony Towns
This module captures orphan tracking code for tx relay. Can be reviewed with --color-moved=dimmed-zebra
2021-02-24Merge #18017: txmempool: split epoch logic into classWladimir J. van der Laan
fd6580e405699ccb051fd2a34525e48d3253673d [refactor] txmempool: split epoch logic into class (Anthony Towns) Pull request description: Splits the epoch logic introduced in #17925 into a separate class. Uses clang's thread safety annotations and encapsulates the data more strongly to reduce chances of bugs from API misuse. ACKs for top commit: jonatack: ACK fd6580e405699ccb051fd2a34525e48d3253673d using clang thread safety annotations looks like a very good idea, and the encapsulation this change adds should improve robustness (and possible unit test-ability) of the code. Verified that changing some of the locking duly provoked build-time warnings with Clang 9 on Debian and that small changes in the new `Epoch` class were covered by failing functional test assertions in `mempool_updatefromblock.py`, `mempool_resurrect.py`, and `mempool_reorg.py` hebasto: re-ACK fd6580e405699ccb051fd2a34525e48d3253673d, since my [previous](https://github.com/bitcoin/bitcoin/pull/18017#pullrequestreview-569619362) review: Tree-SHA512: 7004623faa02b56639aa05ab7a078320a6d8d54ec62d8022876221e33f350f47df51ddff056c0de5be798f8eb39b5c03c2d3f035698555d70abc218e950f2f8c
2021-02-23rpc: add external signer RPC filesSjors Provoost
2021-02-23wallet: add ExternalSignerScriptPubKeyManSjors Provoost
2021-02-23wallet: add -signer argument for external signer commandSjors Provoost
Create basic ExternalSigner class with contructor. A Signer(<cmd>) is added to CWallet on load if -signer=<cmd> is set.
2021-02-15Merge #20629: depends: Improve id string robustnessWladimir J. van der Laan
5200929bfe26c549d7da92c0adf8adf61e143416 depends: Include GUIX_ENVIRONMENT in id string (Carl Dong) 4c7d41858821e4fecf7cb0cec3fcad002365e6c9 depends: Improve id string robustness (Carl Dong) b3bdff42b5a7b4b956da700b187a7254daac54ae build: Proper quoting for var printing targets (Carl Dong) Pull request description: ``` Environment variables and search paths can drastically effect the operation of build tools. Include these in our id string to mitigate against false cache hits. ``` Note to builders: This will invalidate all depends output caches in `BASE_CACHE` ACKs for top commit: laanwj: re-ACK 5200929bfe26c549d7da92c0adf8adf61e143416 Tree-SHA512: e70c98da89cde90dc54bc3be89b925787cf94bbf246e27cc9345816b312073d78a02215448f731f21d8cf033c455234a2377ff1d66c00e1f3db69c9c9687d027
2021-02-12Merge #21064: refactor: use std::shared_mutex & remove Boost ThreadWladimir J. van der Laan
060a2a64d40d75fecb60b7d2b9946a67e46aa6fc ci: remove boost thread installation (fanquake) 06e1d7d81d5a56d136c6fc88f09a2b0654a164f9 build: don't build or use Boost Thread (fanquake) 7097add83c8596f81be9edd66971ffd2486357eb refactor: replace Boost shared_mutex with std shared_mutex in sigcache (fanquake) 8e55981ef834490c438436719f95cbaf888c4914 refactor: replace Boost shared_mutex with std shared_mutex in cuckoocache tests (fanquake) Pull request description: This replaces `boost::shared_mutex` and `boost::unique_lock` with [`std::shared_mutex`](https://en.cppreference.com/w/cpp/thread/shared_mutex) & [`std::unique_lock`](https://en.cppreference.com/w/cpp/thread/unique_lock). Even though [some concerns were raised](https://github.com/bitcoin/bitcoin/issues/16684#issuecomment-726214696) in #16684 with regard to `std::shared_mutex` being unsafe to use across some glibc versions, I still think this change is an improvement. As I mentioned in #21022, I also think trying to restrict standard library feature usage based on bugs in glibc is not only hard to do, but it's not currently clear exactly how we do that in practice (does it also extend to patching out use in our dependencies, should we be implementing more runtime checks for features we are using, when do we consider an affected glibc "old enough" not to worry about? etc). If you take a look through the [glibc bug tracker](https://sourceware.org/bugzilla/describecomponents.cgi?product=glibc) you'll no doubt find plenty of (active) bug reports for standard library code we already using. Obviously not to say we shouldn't try and avoid buggy code where possible. Two other points: [Cory mentioned in #21022](https://github.com/bitcoin/bitcoin/pull/21022#issuecomment-769274179): > It also seems reasonable to me to worry that boost hits the same underlying glibc bug, and we've just not happened to trigger the right conditions yet. Moving away from Boost to the standard library also removes the potential for differences related to Boosts configuration. Boost has multiple versions of `shared_mutex`, and what you end up using, and what it's backed by depends on: * The version of Boost. * The platform you're building for. * Which version of `BOOST_THREAD_VERSION` is defined: (2,3,4 or 5) default=2. (see [here](https://www.boost.org/doc/libs/1_70_0/doc/html/thread/build.html#thread.build.configuration) for some of the differences). * Is `BOOST_THREAD_V2_SHARED_MUTEX` defined? (not by default). If so, you might get the ["less performant, but more robust"](https://github.com/boostorg/thread/issues/230#issuecomment-475937761) version of `shared_mutex`. A lot of these factors are eliminated by our use of depends, but users will have varying configurations. It's also not inconceivable to think that a distro, or some package manager might start defining something like `BOOST_THREAD_VERSION=3`. Boost tried to change the default from 2 to 3 at one point. With this change, we no longer use Boost Thread, so this PR also removes it from depends, the build system, CI etc. Previous similar PRs were #19183 & #20922. The authors are included in the commits here. Also related to #21022 - pthread sanity checking. ACKs for top commit: laanwj: Code review ACK 060a2a64d40d75fecb60b7d2b9946a67e46aa6fc vasild: ACK 060a2a64d40d75fecb60b7d2b9946a67e46aa6fc Tree-SHA512: 572d14d8c9de20bc434511f20d3f431836393ff915b2fe9de5a47a02dca76805ad5c3fc4cceecb4cd43f3ba939a0508178c4e60e62abdbaaa6b3e8db20b75b03
2021-02-11Merge #20788: net: add RAII socket and use it instead of bare SOCKETWladimir J. van der Laan
615ba0eb96cf131364c1ceca9d3dedf006fa1e1c test: add Sock unit tests (Vasil Dimov) 7bd21ce1efc363b3e8ea1d51dd1410ccd66820cb style: rename hSocket to sock (Vasil Dimov) 04ae8469049e1f14585aabfb618ae522150240a7 net: use Sock in InterruptibleRecv() and Socks5() (Vasil Dimov) ba9d73268f9585d4b9254adcf54708f88222798b net: add RAII socket and use it instead of bare SOCKET (Vasil Dimov) dec9b5e850c6aad989e814aea5b630b36f55d580 net: move CloseSocket() from netbase to util/sock (Vasil Dimov) aa17a44551c03b00a47854438afe9f2f89b6ea74 net: move MillisToTimeval() from netbase to util/time (Vasil Dimov) Pull request description: Introduce a class to manage the lifetime of a socket - when the object that contains the socket goes out of scope, the underlying socket will be closed. In addition, the new `Sock` class has a `Send()`, `Recv()` and `Wait()` methods that can be overridden by unit tests to mock the socket operations. The `Wait()` method also hides the `#ifdef USE_POLL poll() #else select() #endif` technique from higher level code. ACKs for top commit: laanwj: Re-ACK 615ba0eb96cf131364c1ceca9d3dedf006fa1e1c jonatack: re-ACK 615ba0eb96cf131364c1ceca9d3dedf006fa1e1c Tree-SHA512: 3003e6bc0259295ca0265ccdeb1522ee25b4abe66d32e6ceaa51b55e0a999df7ddee765f86ce558a788c1953ee2009bfa149b09d494593f7d799c0d7d930bee8
2021-02-10net: move CloseSocket() from netbase to util/sockVasil Dimov
Move `CloseSocket()` (and `NetworkErrorString()` which it uses) from `netbase.{h,cpp}` to newly added `src/util/sock.{h,cpp}`. This is necessary in order to use `CloseSocket()` from a newly introduced Sock class (which will live in `src/util/sock.{h,cpp}`). `sock.{h,cpp}` cannot depend on netbase because netbase will depend on it.
2021-02-09[refactor] txmempool: split epoch logic into classAnthony Towns
2021-02-04Introduce GetUniquePath(base) helper method to replace ↵Kiminuo
boost::filesystem::unique_path() which is not available in std::filesystem.
2021-02-03build: Proper quoting for var printing targetsCarl Dong
Previously, if the value contained syntax that was meaningful to make, the printing would fail. Quoting properly avoids this.
2021-02-02build: don't build or use Boost Threadfanquake
2021-01-21build: Add var printing target to src/Makefile.amCarl Dong
See 181989f6c9427fc266dbdcc84cb60ac03e67cdb2 for more info. I missed this one last time.
2021-01-18Merge #19866: eBPF Linux tracepointsWladimir J. van der Laan
22eb7930a6ae021438aa0b8e750170534944f296 tracing: add tracing framework (William Casarin) 933ab8a720cb9b3341adec4109cffb6dc5b322a5 build: detect sys/sdt.h for eBPF tracing (William Casarin) Pull request description: Instead of writing ad-hoc logging everywhere (eg: #19509), we can take advantage of linux user static defined traces, aka. USDTs ( not the stablecoin :sweat_smile: ) The linux kernel can hook into these tracepoints at runtime, but otherwise they have little to no performance impact. Traces can pass data which can be printed externally via tools such as bpftrace. For example, here's one that prints incoming and outgoing network messages: # Examples ## Network Messages ``` #!/usr/bin/env bpftrace BEGIN { printf("bitcoin net msgs\n"); @start = nsecs; } usdt:./src/bitcoind:net:push_message { $ip = str(arg0); $peer_id = (int64)arg1; $command = str(arg2); $data_len = arg3; $data = buf(arg3,arg4); $t = (nsecs - @start) / 100000; printf("%zu outbound %s %s %zu %d %r\n", $t, $command, $ip, $peer_id, $data_len, $data); @outbound[$command]++; } usdt:./src/bitcoind:net:process_message { $ip = str(arg0); $peer_id = (int64)arg1; $command = str(arg2); $data_len = arg3; $data = buf(arg3,arg4); $t = (nsecs - @start) / 100000; printf("%zu inbound %s %s %zu %d %r\n", $t, $command, $ip, $peer_id, $data_len, $data); @inbound[$ip, $command]++; } ``` $ sudo bpftrace netmsg.bt output: https://jb55.com/s/b11312484b601fb3.txt if you look at the bottom of the output you can see a histogram of all the messages grouped by message type and IP. nice! ## IBD Benchmarking ``` #!/usr/bin/env bpftrace BEGIN { printf("IBD to 500,000 bench\n"); } usdt:./src/bitcoind:CChainState:ConnectBlock { $height = (uint32)arg0; if ($height == 1) { printf("block 1 found, starting benchmark\n"); @start = nsecs; } if ($height >= 500000) { @end = nsecs; @duration = @end - @start; exit(); } } END { printf("duration %d ms\n", @duration / 1000000) } ``` This one hooks into ConnectBlock and prints the IBD time to height 500,000 starting from the first call to ConnectBlock Userspace static tracepoints give lots of flexibility without invasive logging code. It's also more flexible than ad-hoc logging code, allowing you to instrument many different aspects of the system without having to enable per-subsystem logging. Other ideas: tracepoints for lock contention, threads, what else? Let me know what ya'll think and if this is worth adding to bitcoin. ## TODO - [ ] docs? - [x] Integrate systemtap-std-dev/libsystemtap into build (provides the <sys/sdt.h> header) - [x] ~dtrace macos support? (is this still a thing?)~ going to focus on linux for now ACKs for top commit: laanwj: Tested ACK 22eb7930a6ae021438aa0b8e750170534944f296 0xB10C: Tested ACK 22eb7930a6ae021438aa0b8e750170534944f296 Tree-SHA512: 69242242112b679c8a12a22b3bc50252c305894fb3055ae6e13d5f56221d858e58af1d698af55e23b69bdb7abedb5565ac6b45fa5144087b77a17acd04646a75
2021-01-13Merge #19935: Move SaltedHashers to separate file and add some new onesWladimir J. van der Laan
281fd1a4a032cded7f9ea9857e3e99fc793c714b Replace KeyIDHasher with SaltedSipHasher (Andrew Chow) 210b693db66e7c5b618014b5a287aee15af00045 Add generic SaltedSipHasher (Andrew Chow) 95e61c1cf2a91d041c8025306ba36f0ea2806894 Move Hashers to util/hasher.{cpp/h} (Andrew Chow) Pull request description: There are existing `SaltedOutPointHasher` and `SaltedTxidHasher` classes used for `std::unordered_map` and `std::unordered_set` that could be useful in other places in the codebase. So we these to their own `saltedhash.{cpp/h}` file. An existing `KeyIDHasher` is moved there too. Additionally, `ScriptIDHasher`, `SaltedPubkeyHasher`, and `SaltedScriptHasher` are added so that they can be used in future work. `KeyIDHasher` and `ScriptIDHasher` are not salted so that equality comparisons of maps and sets keyed by `CKeyID` and `CScriptID` will actually work. Split from #19602 (and a few other PRs/branches I have). ACKs for top commit: laanwj: Code review ACK 281fd1a4a032cded7f9ea9857e3e99fc793c714b jonatack: ACK 281fd1a4a032cded7f9ea9857e3e99fc793c714b, code review, debug build and ran bitcoind after rebasing to master @ dff0f6f753ea fjahr: utACK 281fd1a4a032cded7f9ea9857e3e99fc793c714b Tree-SHA512: bb03b231ccf3c9ecefc997b8da9c3770af4819f9be5b0a72997a103864e84046a2ac39b8eadf0dc9247bdccd53f86f433642e3a098882e6748341a9e7736271b
2021-01-12Merge #19937: signet mining utilityWladimir J. van der Laan
595a34dbea01954cb0372b0210d2fd64357a1762 contrib/signet: Document miner script in README.md (Anthony Towns) ff7dbdc08a11e999e7718b6ac7645ecceef81188 contrib/signet: Add script for generating a signet chain (Anthony Towns) 13762bcc9618138dd28b53c2031defdc9d762d26 Add bitcoin-util command line utility (Anthony Towns) 95d5d5e6257825bb385cee318d5681597f7f7646 rpc: allow getblocktemplate for test chains when unconnected or in IBD (Anthony Towns) 81c54dec20891f2627a49b2e3e785fdaf2a1e664 rpc: update getblocktemplate with signet rule, include signet_challenge (Anthony Towns) Pull request description: Adds `contrib/signet/miner` for mining signet blocks. Adds `bitcoin-util` cli utility, with the idea being it can provide bitcoin related functionality that does not rely on the ability to access a running node. Only subcommand currently is "grind" which takes a hex-encoded header and grinds its nonce until its nBits is satisfied. Updates `getblocktemplate` to include `signet_challenge` field, and makes `getblocktemplate` require the signet rule when invoked on the signet change. Removes connectivity and IBD checks from `getblocktemplate` when applied to a test chain (regtest, testnet, signet). ACKs for top commit: laanwj: code review ACK 595a34dbea01954cb0372b0210d2fd64357a1762 Tree-SHA512: 8d43297710fdc1edc58acd9b53e1bd1671e5724f7097b40ab73653715dc8becc70534c4496cbba9290f4dd6538a7a3d5830eb85f83391ea31a3bb5b9d3378cc3
2021-01-12Add bitcoin-util command line utilityAnthony Towns
2021-01-07Merge #18077: net: Add NAT-PMP port forwarding supportWladimir J. van der Laan
a191e23b8e7f0e19fc0359825eb7ca0d47966fa9 doc: Add release notes (Hennadii Stepanov) ae749d12ddbaf592fbdb65d98ca35a0ff5566992 doc: Add libnatpmp stuff (Hennadii Stepanov) e28f9be87a0f3c59a9184d602fe7947526df6a97 ci: Add libnatpmp-dev package to some builds (Hennadii Stepanov) 5a0185b6c9c838290103314916190a0330ed9a82 gui: Add NAT-PMP network option (Hennadii Stepanov) a39f7336a3b493d46a4486c4c94fdca1b3151370 net: Add -natpmp command line option (Hennadii Stepanov) 28acffd9d53ec437e908abb8c84497a4f41b91ed net: Add NAT-PMP to port mapping loop (Hennadii Stepanov) a8d9f275d0ca64797cc89627f8003b48b3efef63 net: Add libnatpmp support (Hennadii Stepanov) 58e8364dcdc4e57b0caac09f8402e6535301de9b gui: Apply port mapping changes on dialog exit (Hennadii Stepanov) cf151cc68c95a8943e43e3fa4061e176262779e7 scripted-diff: Rename UPnP stuff (Hennadii Stepanov) 4e91b1e24d96e0cdccdd2a3ed034413f3ba6bae6 net: Add flags for port mapping protocols (Hennadii Stepanov) 8b50d1b5bb29b7d1ea0245ba75a8df3144e312dc net: Keep trying to use UPnP when -upnp=1 (Hennadii Stepanov) 28e2961fd6a2a9101fc08fb748430989291aaf7e refactor: Replace magic number with named constant (Hennadii Stepanov) 02ccf69dd6b772423acb343d16ef2bdbb3e3da03 refactor: Move port mapping code to its own module (Hennadii Stepanov) Pull request description: Close #11902 This PR is an alternative to: - #12288 - #15717 To compile with NAT-PMP support on Ubuntu [`libnatpmp-dev`](https://packages.ubuntu.com/source/bionic/libnatpmp) should be available. Log excerpt: ``` 2020-02-05T20:12:28Z [mapport] NAT-PMP: public address = 95.164.65.194 2020-02-05T20:12:28Z [mapport] AddLocal(95.164.65.194:18333,3) 2020-02-05T20:12:28Z [mapport] NAT-PMP: port mapping successful. ``` See: [`libnatpmp`](https://miniupnp.tuxfamily.org/libnatpmp.html) --- Some follow-ups are out of this PR's scope: - mention NAT-PMP library in the version message - ~integrate NAT-PMP into the GUI~ (already [added](https://github.com/bitcoin/bitcoin/pull/18077#issuecomment-589405068)) ACKs for top commit: laanwj: Tested and code review ACK a191e23b8e7f0e19fc0359825eb7ca0d47966fa9 Tree-SHA512: 10e19267c21bf30f20ff1abfc882d526049f0e790b95e12f109dc2bed7c0aef45de03eaf967f4e667e7509be04f1873a5c508087393d947205f3aab2ad6d7cf1
2021-01-07Merge #19055: Add MuHash3072 implementationWladimir J. van der Laan
9815332d5158d69a94abeaf465a2c07bd8e43359 test: Change MuHash Python implementation to match cpp version again (Fabian Jahr) 01297fb3ca57e4b8cbc5a89fc7c6367de33b0bc6 fuzz: Add MuHash consistency fuzz test (Fabian Jahr) b111410914041b72961536c3e4037eba103a8085 test: Add MuHash3072 fuzz test (Fabian Jahr) c1225273857f9fa2e2276396e3f8b3ea48306df3 bench: Add Muhash benchmarks (Fabian Jahr) 7b1242229d1fcc9277238a3aefb3431061c82bfa test: Add MuHash3072 unit tests (Fabian Jahr) adc708c98dbf03b1735edc91f813a36580781a95 crypto: Add MuHash3072 implementation (Fabian Jahr) 0b4d290bf5b0a4d156c523431bf89aaa9ffe92e5 crypto: Add Num3072 implementation (Fabian Jahr) 589f958662a2dcaacdb9a66f1088c74828a39577 build: Check for 128 bit integer support (Fabian Jahr) Pull request description: This is the first split of #18000 which implements the Muhash algorithm and uses it to calculate the UTXO set hash in `gettxoutsetinfo`. ACKs for top commit: laanwj: Code review ACK 9815332d5158d69a94abeaf465a2c07bd8e43359 Tree-SHA512: 4bc090738f0e3d80b74bdd8122e24a8ce80121120fd37c7e4335a73e7ba4fcd7643f2a2d559e2eebf54b8e3a3bd5f12cfb27ba61ded135fda210a07a233eae45
2021-01-07net: Add libnatpmp supportHennadii Stepanov
2021-01-07refactor: Move port mapping code to its own moduleHennadii Stepanov
This commit does not change behavior.
2020-12-21crypto: Add MuHash3072 implementationFabian Jahr
Co-authored-by: Pieter Wuille <pieter.wuille@gmail.com>
2020-12-18Merge #20434: contrib: Parse ELF directly for symbol and security checksWladimir J. van der Laan
a0a771843fc39c3cc2574a51f009c3391e1808e9 contrib: Changes to checks for PowerPC64 (Luke Dashjr) 634f6ec4eb9997d7bd0f8209fad49a4171d42384 contrib: Parse ELF directly for symbol and security checks (Wladimir J. van der Laan) Pull request description: Instead of the ever-messier text parsing of the output of the readelf tool (which is clearly meant for human consumption not to be machine parseable), parse the ELF binaries directly. Add a small dependency-less ELF parser specific to the checks. This is slightly more secure, too, because it removes potential ambiguity due to misparsing and changes in the output format of `elfread`. It also allows for stricter and more specific ELF format checks in the future. This removes the build-time dependency for `readelf`. It passes the test-security-check for me locally, ~~though I haven't checked on all platforms~~. I've checked that this works on the cross-compile output for all ELF platforms supported by Bitcoin Core at the moment, as well as PPC64 LE and BE. Top commit has no ACKs. Tree-SHA512: 7f9241fec83ee512642fecf5afd90546964561efd8c8c0f99826dcf6660604a4db2b7255e1afb1e9bb0211fd06f5dbad18a6175dfc03e39761a40025118e7bfc
2020-12-16wallettool: Add dump commandAndrew Chow
Adds a new dump command to bitcoin-wallet which prints out all of the wallet's records in hex.
2020-12-07tracing: add tracing frameworkWilliam Casarin
Signed-off-by: William Casarin <jb55@jb55.com>
2020-12-01Merge #20494: refactor: Move node and wallet code out of src/interfacesMarcoFalke
629a9299b2a7241a3fa7d597cb34abcbe1af9255 Move WalletImpl from interfaces/wallet.cpp to wallet/interfaces.cpp (Russell Yanofsky) 2a26771d8161d30be1853a35acfee588cce03634 Move ChainImpl from interfaces/chain.cpp to node/interfaces.cpp (Russell Yanofsky) 12bd0fc9d70333c54c83ebb08c734272dbd330c2 Move NodeImpl from interfaces/node.cpp to node/interfaces.cpp (Russell Yanofsky) Pull request description: This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10). --- Move `NodeImpl` from `interfaces/node.cpp` to `node/interfaces.cpp` Move `ChainImpl` from `interfaces/chain.cpp` to `node/interfaces.cpp` Move `WalletImpl` from `interfaces/wallet.cpp` to `wallet/interfaces.cpp` No changes to any classes (can review with `git diff --color-moved=dimmed_zebra`) Motivation for this change is to move node and wallet code to respective directories where it might fit in better than `src/interfaces/`, but also to remove all unnecessary code from `src/interfaces/` to unblock #19160 review, which has been hung up partially because of code organization. Building on top of this PR, #19160 should now be able to organize interface implementations more understandably in `src/node/` `src/wallet/` `src/ipc/` and `src/init/` directories instead of having so much functionality all in `src/interfaces/` ACKs for top commit: promag: Code review ACK 629a9299b2a7241a3fa7d597cb34abcbe1af9255. MarcoFalke: review ACK 629a9299b2a7241a3fa7d597cb34abcbe1af9255 🔺 Tree-SHA512: 87c2b8fd51519bbd4e5ad3539a79debcf88c3bf021eb28c63f3f555186538b62a0c4cc1a3f07cfb4ff13aea8b0b2fdde505d81f22a5e5fd12a6e375b55a92ab8
2020-11-24Move WalletImpl from interfaces/wallet.cpp to wallet/interfaces.cppRussell Yanofsky
2020-11-24Move ChainImpl from interfaces/chain.cpp to node/interfaces.cppRussell Yanofsky
No changes to ChainImpl or any related classes (review with `git diff --color-moved=dimmed_zebra`)
2020-11-24Move NodeImpl from interfaces/node.cpp to node/interfaces.cppRussell Yanofsky
2020-11-23build: Avoid secp256k1.h include from systemNiklas Gögge
2020-11-22contrib: Parse ELF directly for symbol and security checksWladimir J. van der Laan
Instead of the ever-messier text parsing of the output of the readelf tool (which is clearly meant for human consumption not to be machine parseable), parse the ELF binaries directly. Add a small dependency-less ELF parser specific to the checks. This is slightly more secure, too, because it removes potential ambiguity due to misparsing and changes in the output format of `elfread`. It also allows for stricter and more specific ELF format checks in the future. This removes the build-time dependency for `readelf`. It passes the test-security-check for me locally, though I haven't checked on all platforms.
2020-11-18Do not compile BDB things when USE_BDB is definedAndrew Chow
2020-11-10Move Hashers to util/hasher.{cpp/h}Andrew Chow
Move the hashers that we use for hash tables to a common place. Moved hashers: - SaltedTxidHasher - SaltedOutpointHasher - FilterHeaderHasher - SignatureCacheHasher - BlockHasher
2020-10-20Make sqlite support optional (compile-time)Luke Dashjr
2020-10-15Merge #19077: wallet: Add sqlite as an alternative wallet database and use ↵Samuel Dobson
it for new descriptor wallets c4a29d0a90b821c443c10891d9326c534d15cf97 Update wallet_multiwallet.py for descriptor and sqlite wallets (Russell Yanofsky) 310b0fde04639b7446efd5c1d2701caa4b991b86 Run dumpwallet for legacy wallets only in wallet_backup.py (Andrew Chow) 6c6639ac9f6e1677da066cf809f9e3fa4d2e7c32 Include sqlite3 in documentation (Andrew Chow) f023b7cac0eb16d3c1bf40f1f7898b290de4cc73 wallet: Enforce sqlite serialized threading mode (Andrew Chow) 6173269866306058fcb1cc825b9eb681838678ca Set and check the sqlite user version (Andrew Chow) 9d3d2d263c331e3c77b8f0d01ecc9fea0407dd17 Use network magic as sqlite wallet application ID (Andrew Chow) 9af5de3798c49f86f27bb79396e075fb8c1b2381 Use SQLite for descriptor wallets (Andrew Chow) 9b78f3ce8ed1867c37f6b9fff98f74582d44b789 walletutil: Wallets can also be sqlite (Andrew Chow) ac38a87225be0f1103ff9629d63980550d2f372b Determine wallet file type based on file magic (Andrew Chow) 6045f77003f167bee9a85e2d53f8fc6ff2e297d8 Implement SQLiteDatabase::MakeBatch (Andrew Chow) 727e6b2a4ee5abb7f2dcbc9f7778291908dc28ad Implement SQLiteDatabase::Verify (Andrew Chow) b4df8fdb19fcded7e6d491ecf0b705cac0ec76a1 Implement SQLiteDatabase::Rewrite (Andrew Chow) 010e3659069e6f97dd7b24483f50ed71042b84b0 Implement SQLiteDatabase::TxnBegin, TxnCommit, and TxnAbort (Andrew Chow) ac5c1617e7f4273daf24c24da1f6bc5ef5ab2d2b Implement SQLiteDatabase::Backup (Andrew Chow) f6f9cd6a64842ef23777312f2465e826ca04b886 Implement SQLiteBatch::StartCursor, ReadAtCursor, and CloseCursor (Andrew Chow) bf90e033f4fe86cfb90492c7e0962278ea3a146d Implement SQLiteBatch::ReadKey, WriteKey, EraseKey, and HasKey (Andrew Chow) 7aa45620e2f2178145a2eca58ccbab3cecff08fb Add SetupSQLStatements (Andrew Chow) 6636a2608a4e5906ee8092d5731595542261e0ad Implement SQLiteBatch::Close (Andrew Chow) 93825352a36456283bf87e39b5888363ee242f21 Implement SQLiteDatabase::Close (Andrew Chow) a0de83372be83f59015cd3d61af2303b74fb64b5 Implement SQLiteDatabase::Open (Andrew Chow) 3bfa0fe1259280f8c32b41a798c9453b73f89b02 Initialize and Shutdown sqlite3 globals (Andrew Chow) 5a488b3d77326a0d957c1233493061da1b6ec207 Constructors, destructors, and relevant private fields for SQLiteDatabase/Batch (Andrew Chow) ca8b7e04ab89f99075b093fa248919fd10acbdf7 Implement SQLiteDatabaseVersion (Andrew Chow) 7577b6e1c88a1a7b45ecf5c7f1735bae6f5a82bf Add SQLiteDatabase and SQLiteBatch dummy classes (Andrew Chow) e87df8258090138d5c22ac46b8602b618620e8a1 Add sqlite to travis and depends (Andrew Chow) 54729f3f4e6765dfded590af5fb28c88331685f8 Add libsqlite3 (Andrew Chow) Pull request description: This PR adds a new class `SQLiteDatabase` which is a subclass of `WalletDatabase`. This provides access to a SQLite database that is used to store the wallet records. To keep compatibility with BDB and to complexity of the change down, we don't make use of many SQLite's features. We use it strictly as a key-value store. We create a table `main` which has two columns, `key` and `value` both with the type `blob`. For new descriptor wallets, we will create a `SQLiteDatabase` instead of a `BerkeleyDatabase`. There is no requirement that all SQLite wallets are descriptor wallets, nor is there a requirement that all descriptor wallets be SQLite wallets. This allows for existing descriptor wallets to work as well as keeping open the option to migrate existing wallets to SQLite. We keep the name `wallet.dat` for SQLite wallets. We are able to determine which database type to use by searching for specific magic bytes in the `wallet.dat` file. SQLite begins it's files with a null terminated string `SQLite format 3`. BDB has `0x00053162` at byte 12 (note that the byte order of this integer depends on the system endianness). So when we see that there is a `wallet.dat` file that we want to open, we check for the magic bytes to determine which database system to use. I decided to keep the `wallet.dat` naming to keep things like backup script to continue to function as they won't need to be modified to look for a different file name. It also simplifies a couple of things in the implementation and the tests as `wallet.dat` is something that is specifically being looked for. If we don't want this behavior, then I do have another branch which creates `wallet.sqlite` files instead, but I find that this direction is easier. ACKs for top commit: Sjors: re-utACK c4a29d0a90b821c443c10891d9326c534d15cf97 promag: Tested ACK c4a29d0a90b821c443c10891d9326c534d15cf97. fjahr: reACK c4a29d0a90b821c443c10891d9326c534d15cf97 S3RK: Re-review ACK c4a29d0a90b821c443c10891d9326c534d15cf97 meshcollider: re-utACK c4a29d0a90b821c443c10891d9326c534d15cf97 hebasto: re-ACK c4a29d0a90b821c443c10891d9326c534d15cf97, only rebased since my [previous](https://github.com/bitcoin/bitcoin/pull/19077#pullrequestreview-507743699) review, verified with `git range-diff master d18892dcc c4a29d0a9`. ryanofsky: Code review ACK c4a29d0a90b821c443c10891d9326c534d15cf97. I am honestly confused about reasons for locking into `wallet.dat` again when it's so easy now to use a clean format. I assume I'm just very dense, or there's some unstated reason, because the only thing that's been brought up are unrealistic compatibility scenarios (all require actively creating a wallet with non-default descriptor+sqlite option, then trying to using the descriptor+sqlite wallets with old software or scripts and ignoring the results) that we didn't pay attention to with previous PRs like #11687, which did not require any active interfaction. jonatack: ACK c4a29d0a90b821c443c10891d9326c534d15cf97, debug builds and test runs after rebase to latest master @ c2c4dbaebd9, some manual testing creating, using, unloading and reloading a few different new sqlite descriptor wallets over several node restarts/shutdowns. Tree-SHA512: 19145732e5001484947352d3175a660b5102bc6e833f227a55bd41b9b2f4d92737bbed7cead64b75b509decf9e1408cd81c185ab1fb4b90561aee427c4f9751c
generated by cgit v1.2.3 (git 2.26.2) at 2024-09-04 10:31:20 +0000