aboutsummaryrefslogtreecommitdiff
path: root/src
AgeCommit message (Collapse)Author
2024-02-20Merge bitcoin/bitcoin#29404: refactor: bitcoin-config.h includes cleanupfanquake
9d1dbbd4ceb8c04340927f5127195dc306adf3fc scripted-diff: Fix bitcoin_config_h includes (TheCharlatan) Pull request description: As mentioned in https://github.com/bitcoin/bitcoin/pull/26924#issuecomment-1403449932 and https://github.com/bitcoin/bitcoin/pull/29263#issuecomment-1922334399, it is currently not safe to remove `bitcoin-config.h` includes from headers because some unrelated file might be depending on it. See also #26972 for discussion. Solve this by including the file directly everywhere it's required, regardless of whether or not it's already included by another header. There should be no functional change here, but it will allow us to safely remove includes from headers in the future. ~I'm afraid it's a bit tedious to reproduce these commits, but it's reasonably straightforward:~ Edit: See note below ```bash # All commands executed from the src/ subdir. # Collect all tokens from bitcoin-config.h.in # Isolate the tokens and remove blank lines # Replace newlines with | and remove the last trailing one # Collect all files which use these tokens # Filter out subprojects (proper forwarding can be verified from Makefiles) # Filter out .rc files # Save to a text file git grep -E -l `grep undef config/bitcoin-config.h.in | cut -d" " -f2 | grep -v '^$' | tr '\n' '|' | sed 's/|$//'` | grep -v -e "^leveldb/" -e "^secp256k1/" -e "^crc32c/" -e "^minisketch/" -e "^Makefile" -e "\.rc$" > files-with-config-include.txt # Find all files from the above list which don't include bitcoin-config.h git grep -L -E "config/bitcoin-config.h" -- `cat files-with-config-include.txt` # Include them manually with the exception of some files in crypto: # crypto/sha256_arm_shani.cpp crypto/sha256_avx2.cpp crypto/sha256_sse41.cpp crypto/sha256_x86_shani.cpp # These are exceptions which don't use bitcoin-config.h, rather the Makefile.am adds these cppflags manually. # Commit changes. This should match the first commit of this PR. # Use the same search as above to find all files which DON'T use any config tokens git grep -E -L `grep undef config/bitcoin-config.h.in | cut -d" " -f2 | grep -v '^$' | tr '\n' '|' | sed 's/|$//'` | grep -v -e "^leveldb/" -e "^secp256k1/" -e "^crc32c/" -e "^minisketch/" -e "^Makefile" -e "\.rc$" > files-without-config-include.txt # Manually remove the includes and commit changes. This should match the second commit of this PR. ``` Edit: I'll keep this old description for posterity, but the manual approach has been replaced with a scripted diff from TheCharlatan ACKs for top commit: maflcko: ACK 9d1dbbd4ceb8c04340927f5127195dc306adf3f 🚪 TheCharlatan: ACK 9d1dbbd4ceb8c04340927f5127195dc306adf3fc hebasto: ACK 9d1dbbd4ceb8c04340927f5127195dc306adf3fc, I have reviewed the code and it looks OK. fanquake: ACK 9d1dbbd4ceb8c04340927f5127195dc306adf3fc Tree-SHA512: f11ddc4ae6a887f96b954a6b77f310558ddb271088a3fda3edc833669c4251b7f392515224bbb8e5f67eb2c799b4ffed3b07d96454e82ec635c686d0df545872
2024-02-20Merge bitcoin-core/gui#797: test: Recognize dialog object by nameHennadii Stepanov
4c9db9b5874acb5db2fb9bb1eb5d549aa17f9aa8 qt, test: Recognize dialog object by name (Hennadii Stepanov) Pull request description: Fixes https://github.com/bitcoin-core/gui/issues/796. ACKs for top commit: furszy: Code ACK 4c9db9b587 BrandonOdiwuor: ACK 4c9db9b5874acb5db2fb9bb1eb5d549aa17f9aa8 Tree-SHA512: bd54a95d3ef77bce189c2ce279c6b3b4045bdc749e115045bfd7beda73be5a553e145eb331f454cb50374c5a9e98e73794d72d43aa1887dc42bcc585ca17d10c
2024-02-20Merge bitcoin/bitcoin#26008: wallet: cache IsMine scriptPubKeys to improve ↵fanquake
performance of descriptor wallets e041ed9b755468d205ed48b29f6c4b9e9df8bc9f wallet: Retrieve ID from loaded DescSPKM directly (Ava Chow) 39640dd34e980e69d13664ddbc2a7612a1888ab4 wallet: Use scriptPubKeyCache in GetSolvingProvider (Ava Chow) b410f68791143800968f4c628beda1c9f898b4f6 wallet: Use scriptPubKey cache in GetScriptPubKeyMans (Ava Chow) edf4e73a163739a64eb9a54cd95413583a0e5c1f wallet: Use scriptPubKey cache in IsMine (Ava Chow) 37232332bd7253422ea92a8c9eeb36b12fc84b56 wallet: Cache scriptPubKeys for all DescriptorSPKMs (Ava Chow) 99a0cddbc04e8bfea3748a6cb4c0107392fab94f wallet: Introduce a callback called after TopUp completes (Ava Chow) b27682593266c99507c720855cb34f5f7d363dd2 bench: Add a benchmark for ismine (Ava Chow) Pull request description: Wallets that have a ton of non-ranged descriptors (such as a migrated non-HD wallet) perform fairly poorly due to looping through all of the wallet's `ScriptPubKeyMan`s. This is done in various places, such as `IsMine`, and helper functions for fetching a `ScriptPubKeyMan` and a `SolvingProvider`. This also has a bit of a performance impact on standard descriptor wallets, although less noticeable due to the small number of SPKMs. As these functions are based on doing `IsMine` for each `ScriptPubKeyMan`, we can improve this performance by caching `IsMine` scriptPubKeys for all descriptors and use that to determine which `ScriptPubKeyMan` to actually use for those things. This cache is used exclusively and we no longer iterate the SPKMs. Also added a benchmark for `IsMine`. ACKs for top commit: ryanofsky: Code review ACK e041ed9b755468d205ed48b29f6c4b9e9df8bc9f. Just suggested changes since last review josibake: ACK https://github.com/bitcoin/bitcoin/pull/26008/commits/e041ed9b755468d205ed48b29f6c4b9e9df8bc9f furszy: Code review ACK e041ed9b Tree-SHA512: 8e7081991a025e682e9dea838b4543b0d179832d1c47397fb9fe7a97fa01eb699c15a5d5a785634926844fc83a46e6ac07ef753119f39d84423220ef8a548894
2024-02-19Merge bitcoin/bitcoin#29434: rpc: Fixed signed integer overflow for large ↵Ava Chow
feerates dddd7be9bf038c25f3e53c5bd708fb9cf73d4493 doc: Clarify maxfeerate help (MarcoFalke) fa2a4fdef779b01e847def5985deafedc6dd3da8 rpc: Fixed signed integer overflow for large feerates (MarcoFalke) fade94d11a5b93113975c4b2f62a357a70d03191 rpc: Add ParseFeeRate helper (MarcoFalke) fa0ff6610944fdda716fb0134b78cb85a4c9c26d rpc: Implement RPCHelpMan::ArgValue<> for UniValue (MarcoFalke) Pull request description: Passing large BTC/kvB feerates to RPCs is problematic, because: * They are likely a typo. 1BTC/kvB (or larger) seems absurd. * They may cause signed integer overflow. * Anyone really wanting to pick such a large value can set `0` to disable the check. Fix all issues by rejecting anything more than 1BTC/kvB during parsing. ACKs for top commit: brunoerg: crACK dddd7be9bf038c25f3e53c5bd708fb9cf73d4493 achow101: ACK dddd7be9bf038c25f3e53c5bd708fb9cf73d4493 vasild: ACK dddd7be9bf038c25f3e53c5bd708fb9cf73d4493 tdb3: Code review ACK and basic test ACK for dddd7be9bf038c25f3e53c5bd708fb9cf73d4493. fjahr: utACK dddd7be9bf038c25f3e53c5bd708fb9cf73d4493 Tree-SHA512: 5dcce1f0abe059dc6b2ff56787e11081d73a45b4ddd6dcc2c1ea13709ebc13af5e7265e84fffb97ef32027b56b81955672a67ed7702e8fa30c2e849d67727bac
2024-02-19qt, test: Recognize dialog object by nameHennadii Stepanov
2024-02-16wallet: Retrieve ID from loaded DescSPKM directlyAva Chow
Instead of iterating m_spk_managers a DescriptorSPKM has been loaded in order to get it's ID to compare, have LoadDescriptorSPKM return a reference to the loaded DescriptorSPKM so it can be queried directly.
2024-02-16wallet: Use scriptPubKeyCache in GetSolvingProviderAva Chow
2024-02-16wallet: Use scriptPubKey cache in GetScriptPubKeyMansAva Chow
2024-02-16wallet: Use scriptPubKey cache in IsMineAva Chow
2024-02-16wallet: Cache scriptPubKeys for all DescriptorSPKMsAva Chow
Have CWallet maintain a cache of all known scriptPubKeys for its DescriptorSPKMs in order to improve performance of the functions that require searching for scriptPubKeys.
2024-02-16wallet: Introduce a callback called after TopUp completesAva Chow
After TopUp completes, the wallet containing each SPKM will want to know what new scriptPubKeys were generated. In order for all TopUp calls (including ones internal the the SPKM), we use a callback function in the WalletStorage interface.
2024-02-16bench: Add a benchmark for ismineAva Chow
2024-02-16Merge bitcoin/bitcoin#28037: rpc: Drop migratewallet experimental warningfanquake
f1684bb88a878eb3f54e945db39ea69b14256eef rpc: mention that migratewallet can take a while (Andrew Chow) 9ecff997e164e70c5958116b20ed441404ccd6f3 rpc: Drop migratewallet experimental warning (Andrew Chow) Pull request description: The migration process itself hasn't fundamentally changed since it was added, so I think it's reasonable to say that it is no longer experimental. ACKs for top commit: maflcko: lgtm ACK f1684bb88a878eb3f54e945db39ea69b14256eef josibake: ACK https://github.com/bitcoin/bitcoin/pull/28037/commits/f1684bb88a878eb3f54e945db39ea69b14256eef furszy: ACK https://github.com/bitcoin/bitcoin/commit/f1684bb88a878eb3f54e945db39ea69b14256eef ryanofsky: Code review ACK f1684bb88a878eb3f54e945db39ea69b14256eef willcl-ark: ACK f1684bb88a878eb3f54e945db39ea69b14256eef Tree-SHA512: 99b176cddbf3878c76bd4c80c030106200bf03139785e26dbae3341e1a675b623a13cd6dc7a0bb78344335bf859ae7548d97b2b58eb650c6e7b305d7cdc86e40
2024-02-15doc: Clarify maxfeerate helpMarcoFalke
2024-02-15rpc: Fixed signed integer overflow for large feeratesMarcoFalke
2024-02-15rpc: Add ParseFeeRate helperMarcoFalke
2024-02-14rpc: Implement RPCHelpMan::ArgValue<> for UniValueMarcoFalke
2024-02-13Merge bitcoin-core/gui#793: Update translation source file for v27.0 string ↵Hennadii Stepanov
freeze 3d1bb1a122a037e966c2fb2e2113f0440edb8d93 qt: Update translation source file for v27.0 string freeze (Hennadii Stepanov) Pull request description: This PR updates the `src/qt/locale/bitcoin_en.xlf` translation source file according to the [Release schedule for 27.0](https://github.com/bitcoin/bitcoin/issues/29028). Note for reviewers: it is expected to get a zero diff after running `make -C src translate` locally. ACKs for top commit: jarolrod: ACK 3d1bb1a122a037e966c2fb2e2113f0440edb8d93 Tree-SHA512: 9b6e5aa3aaabb918d0a6418559bc3eb14297abc48b99e8c6e6de770aa1478b8b28881f8965fd15fe23cf4aa377b88ba903e978c8b75681c4f11e428ca1588b96
2024-02-13scripted-diff: Fix bitcoin_config_h includesTheCharlatan
-BEGIN VERIFY SCRIPT- regex_string='^(?!//).*(AC_APPLE_UNIVERSAL_BUILD|BOOST_PROCESS_USE_STD_FS|CHAR_EQUALS_INT8|CLIENT_VERSION_BUILD|CLIENT_VERSION_IS_RELEASE|CLIENT_VERSION_MAJOR|CLIENT_VERSION_MINOR|COPYRIGHT_HOLDERS|COPYRIGHT_HOLDERS_FINAL|COPYRIGHT_HOLDERS_SUBSTITUTION|COPYRIGHT_YEAR|ENABLE_ARM_SHANI|ENABLE_AVX2|ENABLE_EXTERNAL_SIGNER|ENABLE_SSE41|ENABLE_TRACING|ENABLE_WALLET|ENABLE_X86_SHANI|ENABLE_ZMQ|HAVE_BOOST|HAVE_BUILTIN_CLZL|HAVE_BUILTIN_CLZLL|HAVE_BYTESWAP_H|HAVE_CLMUL|HAVE_CONSENSUS_LIB|HAVE_CXX20|HAVE_DECL_BE16TOH|HAVE_DECL_BE32TOH|HAVE_DECL_BE64TOH|HAVE_DECL_BSWAP_16|HAVE_DECL_BSWAP_32|HAVE_DECL_BSWAP_64|HAVE_DECL_FORK|HAVE_DECL_FREEIFADDRS|HAVE_DECL_GETIFADDRS|HAVE_DECL_HTOBE16|HAVE_DECL_HTOBE32|HAVE_DECL_HTOBE64|HAVE_DECL_HTOLE16|HAVE_DECL_HTOLE32|HAVE_DECL_HTOLE64|HAVE_DECL_LE16TOH|HAVE_DECL_LE32TOH|HAVE_DECL_LE64TOH|HAVE_DECL_PIPE2|HAVE_DECL_SETSID|HAVE_DECL_STRERROR_R|HAVE_DEFAULT_VISIBILITY_ATTRIBUTE|HAVE_DLFCN_H|HAVE_DLLEXPORT_ATTRIBUTE|HAVE_ENDIAN_H|HAVE_EVHTTP_CONNECTION_GET_PEER_CONST_CHAR|HAVE_FDATASYNC|HAVE_GETENTROPY_RAND|HAVE_GETRANDOM|HAVE_GMTIME_R|HAVE_INTTYPES_H|HAVE_LIBADVAPI32|HAVE_LIBCOMCTL32|HAVE_LIBCOMDLG32|HAVE_LIBGDI32|HAVE_LIBIPHLPAPI|HAVE_LIBKERNEL32|HAVE_LIBOLE32|HAVE_LIBOLEAUT32|HAVE_LIBSHELL32|HAVE_LIBSHLWAPI|HAVE_LIBUSER32|HAVE_LIBUUID|HAVE_LIBWINMM|HAVE_LIBWS2_32|HAVE_MALLOC_INFO|HAVE_MALLOPT_ARENA_MAX|HAVE_MINIUPNPC_MINIUPNPC_H|HAVE_MINIUPNPC_UPNPCOMMANDS_H|HAVE_MINIUPNPC_UPNPERRORS_H|HAVE_NATPMP_H|HAVE_O_CLOEXEC|HAVE_POSIX_FALLOCATE|HAVE_PTHREAD|HAVE_PTHREAD_PRIO_INHERIT|HAVE_STDINT_H|HAVE_STDIO_H|HAVE_STDLIB_H|HAVE_STRERROR_R|HAVE_STRINGS_H|HAVE_STRING_H|HAVE_STRONG_GETAUXVAL|HAVE_SYSCTL|HAVE_SYSCTL_ARND|HAVE_SYSTEM|HAVE_SYS_ENDIAN_H|HAVE_SYS_PRCTL_H|HAVE_SYS_RESOURCES_H|HAVE_SYS_SELECT_H|HAVE_SYS_STAT_H|HAVE_SYS_SYSCTL_H|HAVE_SYS_TYPES_H|HAVE_SYS_VMMETER_H|HAVE_THREAD_LOCAL|HAVE_TIMINGSAFE_BCMP|HAVE_UNISTD_H|HAVE_VM_VM_PARAM_H|LT_OBJDIR|PACKAGE_BUGREPORT|PACKAGE_NAME|PACKAGE_STRING|PACKAGE_TARNAME|PACKAGE_URL|PACKAGE_VERSION|PTHREAD_CREATE_JOINABLE|QT_QPA_PLATFORM_ANDROID|QT_QPA_PLATFORM_COCOA|QT_QPA_PLATFORM_MINIMAL|QT_QPA_PLATFORM_WINDOWS|QT_QPA_PLATFORM_XCB|QT_STATICPLUGIN|STDC_HEADERS|STRERROR_R_CHAR_P|USE_ASM|USE_BDB|USE_DBUS|USE_NATPMP|USE_QRCODE|USE_SQLITE|USE_UPNP|_FILE_OFFSET_BITS|_LARGE_FILES)' exclusion_files=":(exclude)src/minisketch :(exclude)src/crc32c :(exclude)src/secp256k1 :(exclude)src/crypto/sha256_arm_shani.cpp :(exclude)src/crypto/sha256_avx2.cpp :(exclude)src/crypto/sha256_sse41.cpp :(exclude)src/crypto/sha256_x86_shani.cpp" git grep --perl-regexp --files-with-matches "$regex_string" -- '*.cpp' $exclusion_files | xargs git grep -L "bitcoin-config.h" | while read -r file; do line_number=$(awk -v my_file="$file" '/\/\/ file COPYING or https?:\/\/www.opensource.org\/licenses\/mit-license.php\./ {line = NR} /^\/\// && NR == line + 1 {while(getline && /^\/\//) line = NR} END {print line+1}' "$file"); sed -i "${line_number}i\\\\n\#if defined(HAVE_CONFIG_H)\\n#include <config/bitcoin-config.h>\\n\#endif" "$file"; done; git grep --perl-regexp --files-with-matches "$regex_string" -- '*.h' $exclusion_files | xargs git grep -L "bitcoin-config.h" | while read -r file; do sed -i "/#define.*_H/a \\\\n\#if defined(HAVE_CONFIG_H)\\n#include <config/bitcoin-config.h>\\n\#endif" "$file"; done; for file in $(git grep --files-with-matches 'bitcoin-config.h' -- '*.cpp' '*.h' $exclusion_files); do if ! grep -q --perl-regexp "$regex_string" $file; then sed -i '/HAVE_CONFIG_H/{N;N;N;d;}' $file; fi; done; -END VERIFY SCRIPT- The first command creates a regular expression for matching all bitcoin-config.h symbols in the following form: ^(?!//).*(AC_APPLE_UNIVERSAL_BUILD|BOOST_PROCESS_USE_STD_FS|...|_LARGE_FILES). It was generated with: ./autogen.sh && printf '^(?!//).*(%s)' $(awk '/^#undef/ {print $2}' src/config/bitcoin-config.h.in | paste -sd "|" -) The second command holds a list of files and directories that should not be processed. These include subtree directories as well as some crypto files that already get their symbols through the makefile. The third command checks for missing bitcoin-config headers in .cpp files and adds the header if it is missing. The fourth command checks for missing bitcoin-config headers in .h files and adds the header if it is missing. The fifth command checks for unneeded bitcoin-config headers in sources files and removes the header if it is unneeded.
2024-02-13Merge bitcoin/bitcoin#29403: wallet: batch erase procedures and improve ↵Ava Chow
'EraseRecords' performance 77331aa2a198b708372a5c6cdf331faf7e2968ef wallet: simplify EraseRecords by using 'ErasePrefix' (furszy) 33757814ceb04102141d3fd5ef2f87abf0422310 wallet: bdb batch 'ErasePrefix', do not create txn internally (furszy) cf4d72a75e9603e947b3d47e1f3ac7c7f37bb401 wallet: db, introduce 'RunWithinTxn()' helper function (furszy) Pull request description: Seeks to optimize and simplify `WalletBatch::EraseRecords`. Currently, this process opens a cursor to iterate over the entire database, searching for records that match the type prefixes, to then call the `WalletBatch::Erase` function for each of the matching records. This PR rewrites this 40-line manual process into a single line; instead of performing all of those actions manually, we can simply utilize the `ErasePrefix()` functionality. The result is 06216b344dea6ad6c385fda0b37808ff9ae5273b. Moreover, it expands the test coverage for the `ErasePrefix` functionality and documents the db txn requirement for `BerkeleyBatch::ErasePrefix` . ACKs for top commit: achow101: reACK 77331aa2a198b708372a5c6cdf331faf7e2968ef josibake: code review ACK https://github.com/bitcoin/bitcoin/pull/29403/commits/77331aa2a198b708372a5c6cdf331faf7e2968ef Tree-SHA512: 9f78dda658677ff19b5979ba0efd11cf9fabf3d315feb79ed1160526f010fe843c41903fc18c0b092f78aa88bc874cf24edad8fc1ea6e96aabdc4fd1daf21ca5
2024-02-13Merge bitcoin/bitcoin#29413: fuzz: increase length of string used for ↵fanquake
`NetWhitelist{bind}Permissions::TryParse` 864e2e9097de8f1fda63137f803687dd5cc96c03 fuzz: increase length of string used for `NetWhitelist{bind}Permissions::TryParse` (brunoerg) Pull request description: The string `s` represents the value from `-whitelist`/`-whitebind` (e.g. "bloom,forcerelay,noban@1.2.3.4:32") and it is used in `NetWhitelistPermissions::TryParse` and `NetWhitebindPermissions::TryParse`. However, a max length of 32 is not enough to cover a lot of cases. Even disconsidering the permissions, 32 would not be enough to cover a lot of addresses. This PR fixes it. ACKs for top commit: maflcko: lgtm ACK 864e2e9097de8f1fda63137f803687dd5cc96c03 epiccurious: utACK 864e2e9097de8f1fda63137f803687dd5cc96c03. vasild: ACK 864e2e9097de8f1fda63137f803687dd5cc96c03 Tree-SHA512: 2b89031b9f2ea92d636f05fd167b1e5ac726742a7e7c1af8ddaeaf90236e659731aaa6b7c23f65ec16ce52ac1b9e68e7b16e23c59e355312d057e001976d172a
2024-02-13Merge bitcoin/bitcoin#29424: v3 followupsfanquake
6b161cb82a9766ef814f05e5b8f019f15d5ee14d [test] second child of a v3 tx can be replaced individually (glozow) 5c998a696c7a4ff2a91fe3d8c7177d2b806eb7ac [refactor] use MAX_PUBKEYS_PER_MULTISIG instead of magic numbers in test (glozow) a9346421db813ebb1233f6477fe924e2f7c562ad [test] PackageV3Checks with inheritance violation in mempool ancestor (glozow) 63b62e123e38cb92c2135e63eb1a5b760c11dd4e [doc] fix docs and comments from v3 (glozow) Pull request description: Addresses final comments from #28948: - thread at https://github.com/bitcoin/bitcoin/pull/28948#discussion_r1483245289, using https://github.com/ismaelsadeeq/bitcoin/commit/87fc7f0a8d8ad1fc2af60ee6cc678df5e6fb01dd with some modifications - https://github.com/bitcoin/bitcoin/pull/28948#discussion_r1483769698 - https://github.com/bitcoin/bitcoin/pull/28948#discussion_r1483776227 - https://github.com/bitcoin/bitcoin/pull/28948#discussion_r1484427635 - https://github.com/bitcoin/bitcoin/pull/28948#discussion_r1484467280 - https://github.com/bitcoin/bitcoin/pull/28948#discussion_r1484531064 - https://github.com/bitcoin/bitcoin/pull/28948#discussion_r1484992098 - https://github.com/bitcoin/bitcoin/pull/28948#discussion_r1484992336 - https://github.com/bitcoin/bitcoin/pull/28948#discussion_r1484994642 ACKs for top commit: instagibbs: ACK 6b161cb82a9766ef814f05e5b8f019f15d5ee14d sdaftuar: utACK 6b161cb82a9766ef814f05e5b8f019f15d5ee14d Tree-SHA512: 584fce7810f4d704ee6ab51fdc7d42bab342140cae3d076f89b5e1966dd1dd8293cb25b3121e41a4dcd65f9d4a735102b9ab2e90f98aa770b84e21f4d35d63d3
2024-02-13qt: Update translation source file for v27.0 string freezeHennadii Stepanov
The diff is produced by running `make -C src translate`.
2024-02-12wallet: simplify EraseRecords by using 'ErasePrefix'furszy
2024-02-12wallet: bdb batch 'ErasePrefix', do not create txn internallyfurszy
Transactions are intended to be started on upper layers rather than internally by the bdb batch object. This enables us to consolidate different write operations within a procedure in the same db txn, improving consistency due to the atomic property of the transaction, as well as its performance due to the reduction of disk write operations. Important Note: This approach also ensures that the BerkeleyBatch::ErasePrefix function behaves exactly as the SQLiteBatch::ErasePrefix function, which does not create a db txn internally. Furthermore, since the `BerkeleyBatch::ErasePrefix' implementation erases records one by one (by traversing the db), this change ensures that the function is always called within an active txn context. Without this measure, there's a potential risk to consistency; certain records may be removed while others could persist due to an internal failure during the procedure.
2024-02-12wallet: db, introduce 'RunWithinTxn()' helper functionfurszy
'RunWithinTxn()' provides a way to execute db operations within a transactional context. It avoids writing repetitive boilerplate code for starting and committing the database transaction.
2024-02-12Merge bitcoin/bitcoin#28987: wallet: simplify and batch zap wallet txes processAva Chow
9a3c5c8697659e34d0476103af942a4615818f4e scripted-diff: rename ZapSelectTx to RemoveTxs (furszy) 83b762845f5804f23b63526d403b2f327fe99632 wallet: batch and simplify ZapSelectTx process (furszy) 595d50a1032ad7ffa9945464c86aa57f16665e93 wallet: migration, remove extra NotifyTransactionChanged call (furszy) a2b071f9920c2f4893afcc43a152f593c03966bf wallet: ZapSelectTx, remove db rewrite code (furszy) Pull request description: Work decoupled from #28574. Brother of #28894. Includes two different, yet interconnected, performance and code improvements to the zap wallet transactions process. 1) As the goal of the `ZapSelectTx` function is to erase tx records that match any of the inputted hashes. There is no need to traverse the whole database record by record. We could just check if the tx exist, and remove it directly by calling `EraseTx()`. 2) Instead of performing single write operations per removed tx record, this PR batches them all within a single atomic db txn. Moreover, these changes will enable us to consolidate all individual write operations that take place during the wallet migration process into a single db txn in the future. ACKs for top commit: achow101: ACK 9a3c5c8697659e34d0476103af942a4615818f4e josibake: ACK https://github.com/bitcoin/bitcoin/pull/28987/commits/9a3c5c8697659e34d0476103af942a4615818f4e Tree-SHA512: fb2ecc48224c400ab3b1fbb32e174b5b13bf03794717727f80f01f55fb183883b067a68c0a127b2de8885564da15425d021a96541953bf38a72becc2e9929ccf
2024-02-12Merge bitcoin-core/gui#773: Check for private keys disabled before ↵Hennadii Stepanov
attempting unlock 517c7f9cba306292e12e166b9dbc6c0838f05b27 gui: Check for private keys disabled before attempting unlock (Andrew Chow) Pull request description: Before trying to unlock a wallet, first check if it has private keys disabled. If so, there is no need to unlock. Note that such wallets are not expected to occur in typical usage. However bugs in previous versions allowed such wallets to be created, and so we need to handle them. Fixes #772 For some additional context, see #631 ACKs for top commit: hebasto: ACK 517c7f9cba306292e12e166b9dbc6c0838f05b27, I have reviewed the code and it looks OK. BrandonOdiwuor: ACK 517c7f9cba306292e12e166b9dbc6c0838f05b27 Tree-SHA512: c92aa34344d04667b70b059d2aa0a1da999cb7239cd1413f3009781aa82379f309ff9808d7dc91d385e2c8afe2abda3564568e2091ef833b1536ebfcf80f7c3c
2024-02-12[test] PackageV3Checks with inheritance violation in mempool ancestorglozow
2024-02-12[doc] fix docs and comments from v3glozow
2024-02-12Merge bitcoin-core/gui#758: Update Node window title with the chain typeHennadii Stepanov
9d37886a3b6ce24f4a4a05193eb0d071655a8457 gui: Update Node window title with chain type (pablomartin4btc) Pull request description: It fixes #544. Enhance the Node window title by appending the chain type to it, except for the `mainnet`, mirroring the behavior in the main window. ![image](https://github.com/bitcoin-core/gui/assets/110166421/6b81675c-6e53-411f-9ea7-921e74cd2359) There was also some [interest](https://github.com/bitcoin-core/gui/issues/78#issuecomment-695755972) on this while discussing network switching. ACKs for top commit: MarnixCroes: tACK 9d37886a3b6ce24f4a4a05193eb0d071655a8457 hernanmarino: tACK 9d37886a3b6ce24f4a4a05193eb0d071655a8457 BrandonOdiwuor: tested ACK 9d37886a3b6ce24f4a4a05193eb0d071655a8457 alfonsoromanz: Tested ACK https://github.com/bitcoin-core/gui/pull/758/commits/9d37886a3b6ce24f4a4a05193eb0d071655a8457 kristapsk: ACK 9d37886a3b6ce24f4a4a05193eb0d071655a8457 hebasto: ACK 9d37886a3b6ce24f4a4a05193eb0d071655a8457, tested on Ubuntu 23.10. Tree-SHA512: 8c34c4586bd59b1c522662e8aa0726dccc8f12e020f7a6a1af5200a29e5817e1c51e0f467c7923041fc41535ea093c3e0dd787befbbcc84d6b9f7ff0d969db04
2024-02-12Merge bitcoin-core/gui#658: Intro: Never change the prune checkbox after the ↵Hennadii Stepanov
user has touched it bee0ffbecf4f95b65f4084893924a1ab250ca77c GUI/Intro: Never change the prune checkbox after the user has touched it (Luke Dashjr) 420a983e25d2f3ac3cc4d537bf223682be651604 Bugfix: GUI/Intro: Disable GUI prune option if -prune is set, regardless of set value (Luke Dashjr) Pull request description: Re-PR from https://github.com/bitcoin/bitcoin/pull/18729 Now includes a bugfix too (`-prune=2+` disabled the checkbox, but `-prune=0/1` did not; this behaviour is necessary since `-prune` overrides GUI settings) ACKs for top commit: hebasto: ACK bee0ffbecf4f95b65f4084893924a1ab250ca77c, both commits are improvements of the current behaviour. Tested on Ubuntu 23.10. Tree-SHA512: 8eb7d90af37deb30fe226179db3bc9df8ab59e4f3218c8e447ed31fc9ddc81ac1a1629da63347518587a56a4c8558b05cf7ec474024c5f5dfc6d49d6ff0eb0cc
2024-02-11Merge bitcoin-core/gui#780: Fix: Ensure 'Transaction View' remains disabled ↵Hennadii Stepanov
if no wallet is selected b2e531e70a88f5c9e1c055ae7341520a3128e15d qt: update widgets availability on wallet selection (pablomartin4btc) Pull request description: This PR addresses an issue where, with no wallet selected, ticking on "Settings -> Mask values" checkbox twice enables the transaction tab when the checkbox is unticked. <details> <summary>Current behavior display on master</summary> ![Peek 2023-12-06 19-18](https://github.com/bitcoin-core/gui/assets/110166421/6ca4eab6-5ef0-44c1-971c-89b8bc7f0283) </details> <details> <summary>Correction display from this branch</summary> ![Peek 2023-12-07 13-07](https://github.com/bitcoin-core/gui/assets/110166421/1c78f2aa-1cf7-4d63-b4ce-c034877b4832) </details> Note for maintaners: this PR should be backported to both 25.x and 26.x. --- Originally this PR was disabling the "Mask Values" checkbox when no wallet was selected but since a reviewer pointed out that a user might want to open a wallet already on "privacy mode" I rolled that change out. <details> <summary>Original correction display disabling "Mask Values" </summary> ![Peek 2023-12-06 19-11](https://github.com/bitcoin-core/gui/assets/110166421/66fdf023-998a-434d-a5bd-1a3d848fb751) </details> ACKs for top commit: alfonsoromanz: Tested ACK https://github.com/bitcoin-core/gui/pull/780/commits/b2e531e70a88f5c9e1c055ae7341520a3128e15d hebasto: ACK b2e531e70a88f5c9e1c055ae7341520a3128e15d, tested on Ubuntu 22.04. Tree-SHA512: 6be77ab4d5ec86267a9b0a289a4d8600bb67d279f7e0be65e47b608ec392fe705cf026e32f3c082d2f27449b697d1d9e6a1d110035900d7a804ba823c9f5dfd4
2024-02-11Merge bitcoin-core/gui#752: Modify command line help to show support for ↵Hennadii Stepanov
BIP21 URIs ede5014c445dcb40ddcfdede2c51236bbfe85f5e Modify command line help to show support for BIP21 URIs (Hernan Marino) Pull request description: While reviewing a different PR (see https://github.com/bitcoin-core/gui/pull/742 ) **hebasto** suggested that the help for bitcoin-qt should be updated to reflect the fact that bitcoin-qt supports an optional BIP21 URI parameter. Since this reflects actual behaviour of bitcoin-qt and is independent of whether or not the other PR gets merged, I created this simple PR to fix the help message. ACKs for top commit: kristapsk: utACK ede5014c445dcb40ddcfdede2c51236bbfe85f5e pablomartin4btc: lgtm, re ACK ede5014c445dcb40ddcfdede2c51236bbfe85f5e hebasto: ACK ede5014c445dcb40ddcfdede2c51236bbfe85f5e. Tree-SHA512: c456297c486bc5cc65e0e092e7ba9d51b0bd7a584d4fabca7f7ca1f8e58cbcc66e96226539c689ed0f5e7f40da220bbc4ea30b90e31e1aeeb8867a385a90209c
2024-02-09Merge bitcoin/bitcoin#28948: v3 transaction policy for anti-pinningAva Chow
29029df5c700e6940c712028303761d91ae15847 [doc] v3 signaling in mempool-replacements.md (glozow) e643ea795e4b6fea4a6bbb3d72870ee6a4c836b1 [fuzz] v3 transactions and sigop-adjusted vsize (glozow) 1fd16b5c62f54c7f4c60122acd65d852f63d1e8b [functional test] v3 transaction submission (glozow) 27c8786ba918a42c860e6a50eaee9fdf56d7c646 test framework: Add and use option for tx-version in MiniWallet methods (MarcoFalke) 9a1fea55b29fe025355b06b45e3d77d192acc635 [policy/validation] allow v3 transactions with certain restrictions (glozow) eb8d5a2e7d939dd3ee683486e98702079e0dfcc0 [policy] add v3 policy rules (glozow) 9a29d470fbb62bbb27d517efeafe46ff03c25f54 [rpc] return full string for package_msg and package-error (glozow) 158623b8e0726dff7eae4288138f1710e727db9c [refactor] change Workspace::m_conflicts and adjacent funcs/structs to use Txid (glozow) Pull request description: See #27463 for overall package relay tracking. Delving Bitcoin discussion thread: https://delvingbitcoin.org/t/v3-transaction-policy-for-anti-pinning/340 Delving Bitcoin discussion for LN usage: https://delvingbitcoin.org/t/lightning-transactions-with-v3-and-ephemeral-anchors/418 Rationale: - There are various pinning problems with RBF and our general ancestor/descendant limits. These policies help mitigate many pinning attacks and make package RBF feasible (see #28984 which implements package RBF on top of this). I would focus the most here on Rule 3 pinning. [1][2] - Switching to a cluster-based mempool (see #27677 and #28676) requires the removal of CPFP carve out, which applications depend on. V3 + package RBF + ephemeral anchors + 1-parent-1-child package relay provides an intermediate solution. V3 policy is for "Priority Transactions." [3][4] It allows users to opt in to more restrictive topological limits for shared transactions, in exchange for the more robust fee-bumping abilities that offers. Even though we don't have cluster limits, we are able to treat these transactions as having as having a maximum cluster size of 2. Immediate benefits: - You can presign a transaction with 0 fees (not just 1sat/vB!) and add a fee-bump later. - Rule 3 pinning is reduced by a significant amount, since the attacker can only attach a maximum of 1000vB to your shared transaction. This also enables some other cool things (again see #27463 for overall roadmap): - Ephemeral Anchors - Package RBF for these 1-parent-1-child packages. That means e.g. a commitment tx + child can replace another commitment tx using the child's fees. - We can transition to a "single anchor" universe without worrying about package limit pinning. So current users of CPFP carve out would have something else to use. - We can switch to a cluster-based mempool [5] (#27677 #28676), which removes CPFP carve out [6]. [1]: Original mailing list post and discussion about RBF pinning problems https://gist.github.com/glozow/25d9662c52453bd08b4b4b1d3783b9ff, https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-January/019817.html [2]: A FAQ is "we need this for cluster mempool, but is this still necessary afterwards?" There are some pinning issues that are fixed here and not fully fixed in cluster mempool, so we will still want this or something similar afterward. [3]: Mailing list post for v3 https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-September/020937.html [4]: Original PR #25038 also contains a lot of the discussion [5]: https://delvingbitcoin.org/t/an-overview-of-the-cluster-mempool-proposal/393/7 [6]: https://delvingbitcoin.org/t/an-overview-of-the-cluster-mempool-proposal/393#the-cpfp-carveout-rule-can-no-longer-be-supported-12 ACKs for top commit: sdaftuar: ACK 29029df5c700e6940c712028303761d91ae15847 achow101: ACK 29029df5c700e6940c712028303761d91ae15847 instagibbs: ACK 29029df5c700e6940c712028303761d91ae15847 modulo that Tree-SHA512: 9664b078890cfdca2a146439f8835c9d9ab483f43b30af8c7cd6962f09aa557fb1ce7689d5e130a2ec142235dbc8f21213881baa75241c5881660f9008d68450
2024-02-09Merge bitcoin/bitcoin#27877: wallet: Add CoinGrinder coin selection algorithmAva Chow
13161ecf032b7a850686e5942c12222c8f3d0d52 opt: Skip over barren combinations of tiny UTXOs (Murch) b7672c7cdd87acb105639f475744094b53cc9891 opt: Skip checking max_weight separately (Murch) 1edd2baa37a69ff69c2eaeceaad1028f1968cbab opt: Cut if last addition was minimal weight (Murch) 5248e2a60d243b3d5c77ecd9e4c335daca399a48 opt: Skip heavier UTXOs with same effective value (Murch) 9124c73742287b06dfe6e8a94be56ace25f07b2c opt: Tiebreak UTXOs by weight for CoinGrinder (Murch) 451be19dc10381122f705bbb2127b083f1d598c6 opt: Skip evaluation of equivalent input sets (Murch) 407b1e3432b77511900b77be84d5d7450352f462 opt: Track remaining effective_value in lookahead (Murch) 5f84f3cc043c5fb15072f5072fee752eaa01a2ec opt: Skip branches with worse weight (Murch) d68bc74fb2e3ae4ae775ab544fe5b4ab46025abb fuzz: Test optimality of CoinGrinder (Murch) 67df6c629a2e9878b06c1903e90b67087eaa3688 fuzz: Add CoinGrinder fuzz target (Murch) 1502231229dbc32c94e80a2fc2959275c5d246ef coinselection: Track whether CG completed (Murch) 7488acc64685ae16e20b78e7ad018137f27fe404 test: Add coin_grinder_tests (Murch) 6cc9a46cd0f4ed80d4523bbef1508142e0c80d27 coinselection: Add CoinGrinder algorithm (Murch) 89d09566431f57034d9a7df32547ceb13d79c62c opt: Tie-break UTXO sort by waste for BnB (Murch) aaee65823c6e620bef5cc96d8026567e64d822fe doc: Document max_weight on BnB (Murch) Pull request description: ***Please refer to the [topic on Delving Bitcoin](https://delvingbitcoin.org/t/gutterguard-and-coingrinder-simulation-results/279) discussing Gutter Guard/Coingrinder simulation results.*** Adds a coin selection algorithm that minimizes the weight of the input set while creating change. Motivations --- - At high feerates, using unnecessary inputs can significantly increase the fees - Users are upset when fees are relatively large compared to the amount sent - Some users struggle to maintain a sufficient count of UTXOs in their wallet Approach --- So far, Bitcoin Core has used a balanced approach to coin selection, where it will generate multiple input set candidates using various coin selection algorithms and pick the least wasteful among their results, but not explicitly minimize the input set weight. Under some circumstances, we _do_ want to minimize the weight of the input set. Sometimes changeless solutions require many or heavy inputs, and there is not always a changeless solution for Branch and Bound to find in the first place. This can cause expensive transactions unnecessarily. Given a wallet with sufficient funds, `CoinGrinder` will pick the minimal-waste input set for a transaction with a change output. The current implementation only runs `CoinGrinder` at feerates over 3×long-term-feerate-estimate (by default 30 ṩ/vB), which may be a decent compromise between our goal to reduce costs for the users, but still permit transactions at lower feerates to naturally reduce the wallet’s UTXO pool to curb bloat. Trade-offs --- Simulations for my thesis on coin selection ([see Section 6.3.2.1 [PDF]](https://murch.one/erhardt2016coinselection.pdf)) suggest that minimizing the input set for all transactions tends to grind a wallet’s UTXO pool to dust (pun intended): an approach selecting inputs per coin-age-priority (in effect similar to “largest first selection”) on average produced a UTXO pool with 15× the UTXO count as Bitcoin Core’s Knapsack-based Coin Selection then (in 2016). Therefore, I do not recommend running `CoinGrinder` under all circumstances, but only at extreme feerates or when we have another good reason to minimize the input set for other reasons. In the long-term, we should introduce additional metrics to score different input set candidates, e.g. on basis of their privacy and wallet health impact, to pick from all our coin selection results, but until then, we may want to limit use of `CoinGrinder` in other ways. ACKs for top commit: achow101: ACK 13161ecf032b7a850686e5942c12222c8f3d0d52 sr-gi: ACK [13161ec](https://github.com/bitcoin/bitcoin/pull/27877/commits/13161ecf032b7a850686e5942c12222c8f3d0d52) sipa: ACK 13161ecf032b7a850686e5942c12222c8f3d0d52 Tree-SHA512: 895b08b2ebfd0b71127949b7dba27146a6d10700bf8590402b14f261e7b937f4e2e1b24ca46de440c35f19349043ed2eba4159dc2aa3edae57721384186dae40
2024-02-09scripted-diff: rename ZapSelectTx to RemoveTxsfurszy
-BEGIN VERIFY SCRIPT- sed -i 's/ZapSelectTx/RemoveTxs/g' $(git grep -l 'ZapSelectTx' ./src/wallet) -END VERIFY SCRIPT-
2024-02-09wallet: batch and simplify ZapSelectTx processfurszy
The goal of the function is to erase the wallet transactions that match the inputted hashes. There is no need to traverse the database, reading record by record, to then perform single entry removals for each of them. To ensure consistency and improve performance, this change-set removes all tx records within a single atomic db batch operation, as well as it cleans up code, improves error handling and simplifies the transactions removal process entirely. This optimizes the removal of watch-only transactions during the wallet migration process and the 'removeprunedfunds' RPC command.
2024-02-09opt: Skip over barren combinations of tiny UTXOsMurch
Given a lot of small amount UTXOs it is possible that the lookahead indicates sufficient funds, but any combination of them would push us beyond the current best_weight. We can estimate a lower bound for the minimal necessary weight to reach target from the maximal amount and minimal weight in the tail of the UTXO pool: if adding a number of hypothetical UTXOs of this maximum amount and minimum weight would not be able to beat `best_weight`, we can SHIFT to the omission branch, and CUT if the last selected UTXO is not heavier than the minimum weight of the remainder.
2024-02-09opt: Skip checking max_weight separatelyMurch
Initialize `best_selection_weight` as `max_weight` allows us to skip the separate `max_weight` check on every loop.
2024-02-09opt: Cut if last addition was minimal weightMurch
In situations where we have UTXO groups of various weight, we can CUT rather than SHIFT when we exceeded the max_weight or the best selection’s weight while the last step was equal to the minimum weight in the lookahead.
2024-02-09opt: Skip heavier UTXOs with same effective valueMurch
When two successive UTXOs differ in weight but match in effective value, we can skip the second if the first is not selected, because all input sets we can generate by swapping out a lighter UTXOs with a heavier UTXO of matching effective value would be strictly worse.
2024-02-09opt: Tiebreak UTXOs by weight for CoinGrinderMurch
2024-02-09opt: Skip evaluation of equivalent input setsMurch
When two successive UTXOs match in effective value and weight, we can skip the second if the prior is not selected: adding it would create an equivalent input set to a previously evaluated. E.g. if we have three UTXOs with effective values {5, 3, 3} of the same weight each, we want to evaluate {5, _, _}, {5, 3, _}, {5, 3, 3}, {_, 3, _}, {_, 3, 3}, but skip {5, _, 3}, and {_, _, 3}, because the first 3 is not selected, and we therefore do not need to evaluate the second 3 at the same position in the input set. If we reach the end of the branch, we must SHIFT the previously selected UTXO group instead.
2024-02-09opt: Track remaining effective_value in lookaheadMurch
Introduces a dedicated data structure to track the total effective_value available in the remaining UTXOs at each index of the UTXO pool. In contrast to the approach in BnB, this allows us to immediately jump to a lower index instead of visiting every UTXO to add back their eff_value to the lookahead.
2024-02-09opt: Skip branches with worse weightMurch
Once we exceed the weight of the current best selection, we can always shift as adding more inputs can never yield a better solution.
2024-02-09fuzz: Test optimality of CoinGrinderMurch
Co-authored-by: Pieter Wuille <pieter@wuille.net>
2024-02-09fuzz: Add CoinGrinder fuzz targetMurch
2024-02-09coinselection: Track whether CG completedMurch
CoinGrinder may not be able to exhaustively search all potentially interesting combinations for large UTXO pools, so we keep track of whether the search was terminated by the iteration limit.
2024-02-09test: Add coin_grinder_testsMurch
generated by cgit v1.2.3 (git 2.26.2) at 2024-09-19 23:09:15 +0000