aboutsummaryrefslogtreecommitdiff
path: root/src/bench
AgeCommit message (Collapse)Author
2024-04-01Merge bitcoin/bitcoin#29648: Remove libbitcoinconsensusfanquake
80f8b92f4f2311b9e9a25361c9dd973244e6f95c remove libbitcoinconsensus (fanquake) Pull request description: This was deprecated in `v27.0`, for removal in `v28.0`. See discussion in PR #29189. ACKs for top commit: theuni: Concept ACK and light review ACK 80f8b92f4f2311b9e9a25361c9dd973244e6f95c. My only hesitation here is that (afaics?) there's now nothing keeping undesired features like threading or globals from working their way into the interpreter in future commits. m3dwards: Concept ACK https://github.com/bitcoin/bitcoin/pull/29648/commits/80f8b92f4f2311b9e9a25361c9dd973244e6f95c TheCharlatan: ACK 80f8b92f4f2311b9e9a25361c9dd973244e6f95c hebasto: ACK 80f8b92f4f2311b9e9a25361c9dd973244e6f95c, I have reviewed the code and it looks OK. Tree-SHA512: 17a62118aeb088f2695c892bb32794dfea3061e3cb7d9e8e9f1c06c3ff6f63a7587fa532e37edbb91fbc5a19b12c9a0f8e05fa9e8864aa07f92665375d847e80
2024-03-31bench: Disable WalletCreate* benchmarks when building with MSVCHennadii Stepanov
2024-03-20bench: add benchmark for checkblockindexMartin Zumsande
2024-03-18remove libbitcoinconsensusfanquake
This was deprecated in v27.0, for removal in v28.0. See discussion in PR #29189.
2024-03-12bench: basic block filter index initial syncfurszy
Introduce benchmark for the block filter index sync. And makes synchronous 'Sync()' mechanism accessible.
2024-03-11Merge bitcoin/bitcoin#29458: refactor: Preallocate result in TryParseHex to ↵Ava Chow
avoid resizing a19235c14b3dc02de30b5d769de29d1752c23dbd Preallocate result in `TryParseHex` to avoid resizing (Lőrinc) b7489ecb52c1f99facb7c81c5e46963394d0620d Add benchmark for TryParseHex (Lőrinc) Pull request description: This pull request introduces optimizations to the `TryParseHex` function, focusing primarily on the ideal case (valid hexadecimal input without spaces). A new benchmark, `HexParse` was introduced in a separate commit. The main optimization preallocates the result vector based on the input string's length. This aims to completely avoid costly dynamic reallocations when no spaces are present. ------------ Before: ``` | ns/base16 | base16/s | err% | total | benchmark |--------------------:|--------------------:|--------:|----------:|:---------- | 1.60 | 623,238,893.11 | 0.3% | 0.01 | `HexParse` | 1.65 | 606,747,566.34 | 0.6% | 0.01 | `HexParse` | 1.60 | 626,149,544.07 | 0.3% | 0.01 | `HexParse` ``` After: ``` | ns/base16 | base16/s | err% | total | benchmark |--------------------:|--------------------:|--------:|----------:|:---------- | 0.68 | 1,465,555,976.27 | 0.8% | 0.01 | `HexParse` | 0.68 | 1,472,962,920.18 | 0.3% | 0.01 | `HexParse` | 0.68 | 1,476,159,423.00 | 0.3% | 0.01 | `HexParse` ``` ACKs for top commit: achow101: ACK a19235c14b3dc02de30b5d769de29d1752c23dbd hebasto: ACK a19235c14b3dc02de30b5d769de29d1752c23dbd. andrewtoth: Re-ACK a19235c14b3dc02de30b5d769de29d1752c23dbd Empact: Re-ACK https://github.com/bitcoin/bitcoin/pull/29458/commits/a19235c14b3dc02de30b5d769de29d1752c23dbd Tree-SHA512: e09a59791104be3fd1026862ce98de9efafa1f949626fa01e3b7d58e6a2ef02a11f0de55ddba5c43230a53effd24e6d368c1e12848b17e8ce91d7908a59333f0
2024-03-11Merge bitcoin/bitcoin#26564: test: test_bitcoin: allow -testdatadir=<datadir>Ava Chow
d27e2d87b95b7982c05b4c88e463cc9626ab9f0a test: test_bitcoin: allow -testdatadir=<datadir> (Larry Ruane) Pull request description: This backward-compatible change would help with code review, testing, and debugging. When `test_bitcoin` runs, it creates a working or data directory within `/tmp/test_common_Bitcoin\ Core/`, named as a long random (hex) string. This small patch does three things: - If the (new) argument `-testdatadir=<datadir>` is given, use `<datadir>/test_temp/<test-name>/datadir` as the working directory - When the test starts, remove `<datadir>/test_temp/<test-name>/datadir` if it exists from an earlier run (currently, it's presumed not to exist due to the long random string) - Don't delete the working directory at the end of the test if a custom data directory is being used Example usage, which will remove, create, use `/somewhere/test_temp/getarg_tests/boolarg`, and leave it afterward: ``` $ test_bitcoin --run_test=getarg_tests/boolarg -- -testdatadir=/somewhere Running 1 test case... Test directory (will not be deleted): "/somewhere/test_temp/getarg_tests/boolarg/datadir" *** No errors detected $ ls -l /somewhere/test_temp/getarg_tests/boolarg/datadir total 8 drwxrwxr-x 2 larry larry 4096 Feb 22 10:28 blocks -rw-rw-r-- 1 larry larry 1273 Feb 22 10:28 debug.log ``` (A relative pathname also works.) This change affects only `test_bitcoin`; it could also be applied to `test_bitcoin-qt` but that's slightly more involved so I'm skipping that for now. The rationale for this change is that, when running the test using the debugger, it's often useful to watch `debug.log` as the test runs and inspect some of the other files (I've looked at the generated `blknnnn.dat` files for example). Currently, that requires figuring out where the test's working directory is since it changes on every test run. Tests can be run with `-printtoconsole=1` to show debug logging to the terminal, but it's nice to keep `debug.log` continuously open in an editor, for example. Even if not using a debugger, it's sometimes helpful to see `debug.log` and other artifacts after the test completes. Similar functionality is already possible with the functional tests using the `--tmpdir=` and `--nocleanup` arguments. ACKs for top commit: davidgumberg: ACK https://github.com/bitcoin/bitcoin/pull/26564/commits/d27e2d87b95b7982c05b4c88e463cc9626ab9f0a tdb3: re-ACK for d27e2d87b95b7982c05b4c88e463cc9626ab9f0a achow101: ACK d27e2d87b95b7982c05b4c88e463cc9626ab9f0a cbergqvist: ACK d27e2d87b95b7982c05b4c88e463cc9626ab9f0a! (Already did some testing with `fs::remove()` to make sure it was compatible with the `util::Lock/UnlockDirectory` implementation). marcofleon: ACK d27e2d87b95b7982c05b4c88e463cc9626ab9f0a. I ran all the tests with my previous open file limit and no errors were detected. Also ran some individual tests with no, relative, and absolute paths and everything looks good. furszy: ACK d27e2d8 Tree-SHA512: a8f535f34a48b6699cb440f97f5562ec643f3bfba4ea685768980b871fc8b6e1135f70fc05dbe19aa2c8bacb1ddeaff212d63473605a7422ff76332b3a6b1f68
2024-03-08Merge bitcoin/bitcoin#28960: kernel: Remove dependency on CSchedulerAva Chow
d5228efb5391b31a9a0673019e43e7fa2cd4ac07 kernel: Remove dependency on CScheduler (TheCharlatan) 06069b3913dda048f5d640a662b0852f86346ace scripted-diff: Rename MainSignals to ValidationSignals (TheCharlatan) 0d6d2b650d1017691f48c9109a6cd020ab46aa73 scripted-diff: Rename SingleThreadedSchedulerClient to SerialTaskRunner (TheCharlatan) 4abde2c4e3fd9b66394b79874583bdc2a9132c36 [refactor] Make MainSignals RAII styled (TheCharlatan) 84f5c135b8118cbe15b8bfb4db80d61237987f64 refactor: De-globalize g_signals (TheCharlatan) 473dd4b97ae40e43e1a1a97fdbeb40be4855e9bc [refactor] Prepare for g_signals de-globalization (TheCharlatan) 3fba3d5deec6d7bae33823b8da7682f9b03d9deb [refactor] Make signals optional in mempool and chainman (TheCharlatan) Pull request description: By defining a virtual interface class for the scheduler client, users of the kernel can now define their own event consuming infrastructure, without having to spawn threads or rely on the scheduler design. Removing `CScheduler` also allows removing the thread and exception modules from the kernel library. To make the `CMainSignals` class easier to use from a kernel library perspective, remove its global instantiation and adopt RAII practices. Renames `CMainSignals` to `ValidationSignals`, which more accurately describes its purpose and scope. Also make the `ValidationSignals` in the `ChainstateManager` and CTxMemPool` optional. This could be useful in the future for using or testing these classes without having to instantiate any form of signal handling. --- This PR is part of the [libbitcoinkernel project](https://github.com/bitcoin/bitcoin/issues/27587). It improves the kernel API and removes two modules from the kernel library. ACKs for top commit: maflcko: re-ACK d5228efb5391b31a9a0673019e43e7fa2cd4ac07 🌄 ryanofsky: Code review ACK d5228efb5391b31a9a0673019e43e7fa2cd4ac07. Just comment change since last review. vasild: ACK d5228efb5391b31a9a0673019e43e7fa2cd4ac07 furszy: diff ACK d5228ef Tree-SHA512: e93a5f10eb6182effb84bb981859a7ce750e466efd8171045d8d9e7fe46e4065631d9f6f533c5967c4d34c9bb7d7a67e9f4593bd4c5b30cd7b3bbad7be7b331b
2024-03-07test: test_bitcoin: allow -testdatadir=<datadir>Larry Ruane
Specifying this argument overrides the path location for test_bitcoin; it becomes <datadir>/test_common_Bitcoin Core/<testname>/datadir. Also, this directory isn't removed after the test completes. This can make it easier for developers to study the results of a test (see the state of the data directory after the test runs), and also (for example) have an editor open on debug.log to monitor it across multiple test runs instead of having to re-open a different pathname each time. Example usage (note the "--" is needed): test_bitcoin --run_test=getarg_tests/boolarg -- \ -testdatadir=/somewhere/mydatadir This will create (if necessary) and use the data directory: /somewhere/mydatadir/test_common_Bitcoin Core/getarg_tests/boolarg/datadir Co-authored-by: furszy <mfurszy@protonmail.com>
2024-02-28Add benchmark for TryParseHexLőrinc
Running `make && ./src/bench/bench_bitcoin -filter=HexParse` a few times results in: ``` | ns/base16 | base16/s | err% | total | benchmark |--------------------:|--------------------:|--------:|----------:|:---------- | 1.60 | 623,238,893.11 | 0.3% | 0.01 | `HexParse` | 1.65 | 606,747,566.34 | 0.6% | 0.01 | `HexParse` | 1.60 | 626,149,544.07 | 0.3% | 0.01 | `HexParse` ```
2024-02-20refactor: Add missing include for USE_BDB, USE_SQLITE to bench/wallet_ismine.cppMarcoFalke
It was included indirectly via src/wallet/test/util.h, however it is better to include what you use.
2024-02-20Merge bitcoin/bitcoin#29404: refactor: bitcoin-config.h includes cleanupfanquake
9d1dbbd4ceb8c04340927f5127195dc306adf3fc scripted-diff: Fix bitcoin_config_h includes (TheCharlatan) Pull request description: As mentioned in https://github.com/bitcoin/bitcoin/pull/26924#issuecomment-1403449932 and https://github.com/bitcoin/bitcoin/pull/29263#issuecomment-1922334399, it is currently not safe to remove `bitcoin-config.h` includes from headers because some unrelated file might be depending on it. See also #26972 for discussion. Solve this by including the file directly everywhere it's required, regardless of whether or not it's already included by another header. There should be no functional change here, but it will allow us to safely remove includes from headers in the future. ~I'm afraid it's a bit tedious to reproduce these commits, but it's reasonably straightforward:~ Edit: See note below ```bash # All commands executed from the src/ subdir. # Collect all tokens from bitcoin-config.h.in # Isolate the tokens and remove blank lines # Replace newlines with | and remove the last trailing one # Collect all files which use these tokens # Filter out subprojects (proper forwarding can be verified from Makefiles) # Filter out .rc files # Save to a text file git grep -E -l `grep undef config/bitcoin-config.h.in | cut -d" " -f2 | grep -v '^$' | tr '\n' '|' | sed 's/|$//'` | grep -v -e "^leveldb/" -e "^secp256k1/" -e "^crc32c/" -e "^minisketch/" -e "^Makefile" -e "\.rc$" > files-with-config-include.txt # Find all files from the above list which don't include bitcoin-config.h git grep -L -E "config/bitcoin-config.h" -- `cat files-with-config-include.txt` # Include them manually with the exception of some files in crypto: # crypto/sha256_arm_shani.cpp crypto/sha256_avx2.cpp crypto/sha256_sse41.cpp crypto/sha256_x86_shani.cpp # These are exceptions which don't use bitcoin-config.h, rather the Makefile.am adds these cppflags manually. # Commit changes. This should match the first commit of this PR. # Use the same search as above to find all files which DON'T use any config tokens git grep -E -L `grep undef config/bitcoin-config.h.in | cut -d" " -f2 | grep -v '^$' | tr '\n' '|' | sed 's/|$//'` | grep -v -e "^leveldb/" -e "^secp256k1/" -e "^crc32c/" -e "^minisketch/" -e "^Makefile" -e "\.rc$" > files-without-config-include.txt # Manually remove the includes and commit changes. This should match the second commit of this PR. ``` Edit: I'll keep this old description for posterity, but the manual approach has been replaced with a scripted diff from TheCharlatan ACKs for top commit: maflcko: ACK 9d1dbbd4ceb8c04340927f5127195dc306adf3f 🚪 TheCharlatan: ACK 9d1dbbd4ceb8c04340927f5127195dc306adf3fc hebasto: ACK 9d1dbbd4ceb8c04340927f5127195dc306adf3fc, I have reviewed the code and it looks OK. fanquake: ACK 9d1dbbd4ceb8c04340927f5127195dc306adf3fc Tree-SHA512: f11ddc4ae6a887f96b954a6b77f310558ddb271088a3fda3edc833669c4251b7f392515224bbb8e5f67eb2c799b4ffed3b07d96454e82ec635c686d0df545872
2024-02-16bench: Add a benchmark for ismineAva Chow
2024-02-15refactor: De-globalize g_signalsTheCharlatan
2024-02-13scripted-diff: Fix bitcoin_config_h includesTheCharlatan
-BEGIN VERIFY SCRIPT- regex_string='^(?!//).*(AC_APPLE_UNIVERSAL_BUILD|BOOST_PROCESS_USE_STD_FS|CHAR_EQUALS_INT8|CLIENT_VERSION_BUILD|CLIENT_VERSION_IS_RELEASE|CLIENT_VERSION_MAJOR|CLIENT_VERSION_MINOR|COPYRIGHT_HOLDERS|COPYRIGHT_HOLDERS_FINAL|COPYRIGHT_HOLDERS_SUBSTITUTION|COPYRIGHT_YEAR|ENABLE_ARM_SHANI|ENABLE_AVX2|ENABLE_EXTERNAL_SIGNER|ENABLE_SSE41|ENABLE_TRACING|ENABLE_WALLET|ENABLE_X86_SHANI|ENABLE_ZMQ|HAVE_BOOST|HAVE_BUILTIN_CLZL|HAVE_BUILTIN_CLZLL|HAVE_BYTESWAP_H|HAVE_CLMUL|HAVE_CONSENSUS_LIB|HAVE_CXX20|HAVE_DECL_BE16TOH|HAVE_DECL_BE32TOH|HAVE_DECL_BE64TOH|HAVE_DECL_BSWAP_16|HAVE_DECL_BSWAP_32|HAVE_DECL_BSWAP_64|HAVE_DECL_FORK|HAVE_DECL_FREEIFADDRS|HAVE_DECL_GETIFADDRS|HAVE_DECL_HTOBE16|HAVE_DECL_HTOBE32|HAVE_DECL_HTOBE64|HAVE_DECL_HTOLE16|HAVE_DECL_HTOLE32|HAVE_DECL_HTOLE64|HAVE_DECL_LE16TOH|HAVE_DECL_LE32TOH|HAVE_DECL_LE64TOH|HAVE_DECL_PIPE2|HAVE_DECL_SETSID|HAVE_DECL_STRERROR_R|HAVE_DEFAULT_VISIBILITY_ATTRIBUTE|HAVE_DLFCN_H|HAVE_DLLEXPORT_ATTRIBUTE|HAVE_ENDIAN_H|HAVE_EVHTTP_CONNECTION_GET_PEER_CONST_CHAR|HAVE_FDATASYNC|HAVE_GETENTROPY_RAND|HAVE_GETRANDOM|HAVE_GMTIME_R|HAVE_INTTYPES_H|HAVE_LIBADVAPI32|HAVE_LIBCOMCTL32|HAVE_LIBCOMDLG32|HAVE_LIBGDI32|HAVE_LIBIPHLPAPI|HAVE_LIBKERNEL32|HAVE_LIBOLE32|HAVE_LIBOLEAUT32|HAVE_LIBSHELL32|HAVE_LIBSHLWAPI|HAVE_LIBUSER32|HAVE_LIBUUID|HAVE_LIBWINMM|HAVE_LIBWS2_32|HAVE_MALLOC_INFO|HAVE_MALLOPT_ARENA_MAX|HAVE_MINIUPNPC_MINIUPNPC_H|HAVE_MINIUPNPC_UPNPCOMMANDS_H|HAVE_MINIUPNPC_UPNPERRORS_H|HAVE_NATPMP_H|HAVE_O_CLOEXEC|HAVE_POSIX_FALLOCATE|HAVE_PTHREAD|HAVE_PTHREAD_PRIO_INHERIT|HAVE_STDINT_H|HAVE_STDIO_H|HAVE_STDLIB_H|HAVE_STRERROR_R|HAVE_STRINGS_H|HAVE_STRING_H|HAVE_STRONG_GETAUXVAL|HAVE_SYSCTL|HAVE_SYSCTL_ARND|HAVE_SYSTEM|HAVE_SYS_ENDIAN_H|HAVE_SYS_PRCTL_H|HAVE_SYS_RESOURCES_H|HAVE_SYS_SELECT_H|HAVE_SYS_STAT_H|HAVE_SYS_SYSCTL_H|HAVE_SYS_TYPES_H|HAVE_SYS_VMMETER_H|HAVE_THREAD_LOCAL|HAVE_TIMINGSAFE_BCMP|HAVE_UNISTD_H|HAVE_VM_VM_PARAM_H|LT_OBJDIR|PACKAGE_BUGREPORT|PACKAGE_NAME|PACKAGE_STRING|PACKAGE_TARNAME|PACKAGE_URL|PACKAGE_VERSION|PTHREAD_CREATE_JOINABLE|QT_QPA_PLATFORM_ANDROID|QT_QPA_PLATFORM_COCOA|QT_QPA_PLATFORM_MINIMAL|QT_QPA_PLATFORM_WINDOWS|QT_QPA_PLATFORM_XCB|QT_STATICPLUGIN|STDC_HEADERS|STRERROR_R_CHAR_P|USE_ASM|USE_BDB|USE_DBUS|USE_NATPMP|USE_QRCODE|USE_SQLITE|USE_UPNP|_FILE_OFFSET_BITS|_LARGE_FILES)' exclusion_files=":(exclude)src/minisketch :(exclude)src/crc32c :(exclude)src/secp256k1 :(exclude)src/crypto/sha256_arm_shani.cpp :(exclude)src/crypto/sha256_avx2.cpp :(exclude)src/crypto/sha256_sse41.cpp :(exclude)src/crypto/sha256_x86_shani.cpp" git grep --perl-regexp --files-with-matches "$regex_string" -- '*.cpp' $exclusion_files | xargs git grep -L "bitcoin-config.h" | while read -r file; do line_number=$(awk -v my_file="$file" '/\/\/ file COPYING or https?:\/\/www.opensource.org\/licenses\/mit-license.php\./ {line = NR} /^\/\// && NR == line + 1 {while(getline && /^\/\//) line = NR} END {print line+1}' "$file"); sed -i "${line_number}i\\\\n\#if defined(HAVE_CONFIG_H)\\n#include <config/bitcoin-config.h>\\n\#endif" "$file"; done; git grep --perl-regexp --files-with-matches "$regex_string" -- '*.h' $exclusion_files | xargs git grep -L "bitcoin-config.h" | while read -r file; do sed -i "/#define.*_H/a \\\\n\#if defined(HAVE_CONFIG_H)\\n#include <config/bitcoin-config.h>\\n\#endif" "$file"; done; for file in $(git grep --files-with-matches 'bitcoin-config.h' -- '*.cpp' '*.h' $exclusion_files); do if ! grep -q --perl-regexp "$regex_string" $file; then sed -i '/HAVE_CONFIG_H/{N;N;N;d;}' $file; fi; done; -END VERIFY SCRIPT- The first command creates a regular expression for matching all bitcoin-config.h symbols in the following form: ^(?!//).*(AC_APPLE_UNIVERSAL_BUILD|BOOST_PROCESS_USE_STD_FS|...|_LARGE_FILES). It was generated with: ./autogen.sh && printf '^(?!//).*(%s)' $(awk '/^#undef/ {print $2}' src/config/bitcoin-config.h.in | paste -sd "|" -) The second command holds a list of files and directories that should not be processed. These include subtree directories as well as some crypto files that already get their symbols through the makefile. The third command checks for missing bitcoin-config headers in .cpp files and adds the header if it is missing. The fourth command checks for missing bitcoin-config headers in .h files and adds the header if it is missing. The fifth command checks for unneeded bitcoin-config headers in sources files and removes the header if it is unneeded.
2024-01-02Merge bitcoin/bitcoin#26684: bench: add readblock benchmarkAva Chow
1c4b9cbe906507295d8b7d52855de1441ad411dd bench: add readblock benchmark (Andrew Toth) Pull request description: Requested in https://github.com/bitcoin/bitcoin/pull/13151#issuecomment-385962450. See https://github.com/bitcoin/bitcoin/pull/26415 and https://github.com/bitcoin/bitcoin/pull/21319. Benchmarking shows a >50x increase in speed on both nvme and spinning disk. Benchmark results: | ns/op | op/s | err% | ins/op | cyc/op | IPC | bra/op | miss% | total | benchmark |--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:---------- | 5,377,375.00 | 185.96 | 0.2% | 60,125,513.00 | 11,633,676.00 | 5.168 | 3,588,800.00 | 0.4% | 0.09 | `ReadBlockFromDiskTest` | ns/op | op/s | err% | ins/op | cyc/op | IPC | bra/op | miss% | total | benchmark |--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:---------- | 89,945.58 | 11,117.83 | 0.7% | 12,743.90 | 64,530.33 | 0.197 | 2,595.20 | 0.2% | 0.01 | `ReadRawBlockFromDiskTest` ACKs for top commit: maflcko: lgtm ACK 1c4b9cbe906507295d8b7d52855de1441ad411dd achow101: ACK 1c4b9cbe906507295d8b7d52855de1441ad411dd TheCharlatan: ACK 1c4b9cbe906507295d8b7d52855de1441ad411dd Tree-SHA512: 71dbcd6c7e2be97eb3001e35d0a95ef8e0c9b10dc9193025c7f8e11a09017fa2fbf89489b686353cd88fb409fb729fe2c4a25c567d2988f64c9c164ab09fba9f
2023-12-23refactor: share and use `GenerateRandomKey` helperSebastian Falbesoner
Making the `GenerateRandomKey` helper available to other modules via key.{h.cpp} allows us to create random private keys directly at instantiation of CKey, in contrast to the two-step process of creating the instance and then having to call `MakeNewKey(...)`.
2023-12-14Merge bitcoin/bitcoin#29040: refactor: Remove pre-C++20 code, fs::path cleanupAva Chow
66667130416b86208e01a0eb5541a15ea805ac26 refactor: Rename fs::path::u8string() to fs::path::utf8string() (MarcoFalke) 856c88776f8486446602476a1c9e133ac0cff510 ArgsManager: return path by value from GetBlocksDirPath() (Vasil Dimov) fa3d9304e80c214c8b073f12a7f4b08c5a94af04 refactor: Remove pre-C++20 fs code (MarcoFalke) fa00098e1a493aa3cce20335d18e7f5f2fb7a4a8 Add tests for C++20 std::u8string (MarcoFalke) fa2bac08c22182e738a8cabf1b24a9dbf3b092d2 refactor: Avoid copy/move in fs.h (MarcoFalke) faea30227ba633da5ab257d0247853e0927244bb refactor: Use C++20 std::chrono::days (MarcoFalke) Pull request description: This: * Removes dead code. * Avoids unused copies in some places. * Adds copies in other places for safety. ACKs for top commit: achow101: ACK 66667130416b86208e01a0eb5541a15ea805ac26 ryanofsky: Code review ACK 66667130416b86208e01a0eb5541a15ea805ac26. Just documentation change since last review. stickies-v: re-ACK 66667130416b86208e01a0eb5541a15ea805ac26 Tree-SHA512: 6176e44f30b310d51632ec2d3827c3819905d0ddc6a4b57acfcb6cfa1f9735176da75ee8ed4a4abd1296cb0b83bee9374cc6f91ffac87c19b63c435eeadf3f46
2023-12-14refactor: Rename fs::path::u8string() to fs::path::utf8string()MarcoFalke
2023-12-12test: wallet, fix change position out of range errorfurszy
Since #25273, the behavior of 'inserting change at a random position' is instructed by passing std::nullopt instead of -1. Also, added missing documentation about the meaning of 'change_pos=std::nullopt' inside 'CWallet::CreateTransaction()'
2023-12-12Merge bitcoin/bitcoin#29055: tests, bench: Fix issue with ↵fanquake
CWallet::LoadWallet() being called in the wrong places bd7f5d33e3b9e01cd600afe1682b7afa935f68a3 wallet: Assert that the wallet is not initialized in LoadWallet (Andrew Chow) fb0b6ca4e5d981cf58bf23ae2993117f171608e8 tests, bench: Remove incorrect LoadWallet() calls (Andrew Chow) Pull request description: `CWallet::LoadWallet()` expects to be called after a `CWallet` is constructed, but before any of its member functions called. Doing so invalidates pointers which causes issues with some PRs and branches that I am working on. This was being used incorrectly in a few tests and benchmarks, resulting in segfaults. As a precaution for this kind of issue in the future, I've also added a few asserts to `LoadWallet()` so that developers will notice when it is used incorrectly. As similar issue was fixed in #27666 ACKs for top commit: S3RK: ACK bd7f5d33e3b9e01cd600afe1682b7afa935f68a3 furszy: ACK bd7f5d33 Tree-SHA512: 7664f12b8452994e7fc4d7d4f77697fb5f75edb0dba95ba99a4a23ec03d5b8e0ecbdcb7635547a0e8b4f89f708f98dcb5d039df0559e24b1ae411ed630e16e14
2023-12-12Merge bitcoin/bitcoin#29021: refactor: rpc: Pass CBlockIndex by reference ↵fanquake
instead of pointer fa5989d514d246e56977c528b2dd2abe6dc8efcc refactor: rpc: Pass CBlockIndex by reference instead of pointer (MarcoFalke) fa604eb6cfa7f70ce11c78c1060f0823884c745b refactor: Use reference instead of pointer in IsBlockPruned (MarcoFalke) Pull request description: Follow-up to https://github.com/bitcoin/bitcoin/pull/29003#issuecomment-1841435462 ACKs for top commit: TheCharlatan: ACK fa5989d514d246e56977c528b2dd2abe6dc8efcc pablomartin4btc: tACK fa5989d514d246e56977c528b2dd2abe6dc8efcc dergoegge: Code review ACK fa5989d514d246e56977c528b2dd2abe6dc8efcc Tree-SHA512: 7449de3e3bb435dcbf438df88df343bb70f6edc3228ee7c0078f912ffb415e951ba30f8ecad916765f8cf896f0d784fe30535c5cf997e303cf5af257ade69773
2023-12-11tests, bench: Remove incorrect LoadWallet() callsAndrew Chow
LoadWallet() must only be called immediately after a CWallet is constructed, or not at all. Doing so after any other CWallet member functions have been called may cause pointers and other objects setup by other those functions to become invalidated. Since these tests and benchmarks are using completely new wallets with mock databases, it's not necessary to call LoadWallet() anyways, so these can be dropped.
2023-12-08Merge bitcoin/bitcoin#28894: wallet: batch all individual spkms setup db ↵fanquake
writes in a single db txn f05302427386fe63f4929a7198652cb1e4ab3bcc wallet: batch external signer descriptor import (Sjors Provoost) 1f65241b733cd1e962c88909ae66816bc6451fd1 wallet: descriptors setup, batch db operations (furszy) 3eb769f15013873755e482707cad341bc1ce8a8c wallet: batch legacy spkm TopUp (furszy) 075aa44ceba41fa82bb3ce2295e2962e5fd0508e wallet: batch descriptor spkm TopUp (furszy) bb4554c81e0d819d74996f89cbb9c00476aedf8c bench: add benchmark for wallet creation procedure (furszy) Pull request description: Work decoupled from #28574. Instead of performing multiple single write operations per spkm setup call, this PR batches them all within a single atomic db txn. Speeding up the process and preventing the wallet from entering an inconsistent state if any of the intermediate transactions fail (which shouldn't happen but.. if it does, it is better to not store any spkm rather than storing them partially). To compare the changes, added benchmark in the first commit. ACKs for top commit: Sjors: re-utACK f05302427386fe63f4929a7198652cb1e4ab3bcc achow101: ACK f05302427386fe63f4929a7198652cb1e4ab3bcc BrandonOdiwuor: ACK f05302427386fe63f4929a7198652cb1e4ab3bcc theStack: Code-review ACK f05302427386fe63f4929a7198652cb1e4ab3bcc Tree-SHA512: aead8548473e17d4d53e8e7039bbaf5e8bf2fe83f33b33f81cdedefe8a31b7003ceb6d5379b1bad1ca2692e909492009a21284ec8338eede078df3d19046ab5a
2023-12-07refactor: rpc: Pass CBlockIndex by reference instead of pointerMarcoFalke
All functions assume that the pointer is never null, so pass by reference, to avoid accidental segfaults at runtime, or at least make them more obvious. Also, remove unused c-style casts in touched lines. Also, add CHECK_NONFATAL checks, to turn segfault crashes into an recoverable runtime error with debug information.
2023-11-30Merge bitcoin/bitcoin#26762: bugfix: Make `CCheckQueue` RAII-styled (attempt 2)Andrew Chow
5b3ea5fa2e7f6dc1c9161ed8b74c9be4bd1e92dd refactor: Move `{MAX,DEFAULT}_SCRIPTCHECK_THREADS` constants (Hennadii Stepanov) 6e17b3168072ab77ed7170ab81327c017877133a refactor: Make `CCheckQueue` non-copyable and non-movable explicitly (Hennadii Stepanov) 8111e74653dc5c93cb510672d99048c3f741d8dc refactor: Drop unneeded declaration (Hennadii Stepanov) 9cf89f7a5b81197e38f58b24be0793b28fe41477 refactor: Make `CCheckQueue` constructor start worker threads (Hennadii Stepanov) d03eaacbcfb276fb638db1b423113ff43bd7ec41 Make `CCheckQueue` destructor stop worker threads (Hennadii Stepanov) be4ff3060b7b43b496dfb5a2c02b114b2b717106 Move global `scriptcheckqueue` into `ChainstateManager` class (Hennadii Stepanov) Pull request description: This PR: - makes `CCheckQueue` RAII-styled - gets rid of the global `scriptcheckqueue` - fixes https://github.com/bitcoin/bitcoin/issues/25448 The previous attempt was in https://github.com/bitcoin/bitcoin/pull/18731. ACKs for top commit: martinus: ACK 5b3ea5fa2e7 achow101: ACK 5b3ea5fa2e7f6dc1c9161ed8b74c9be4bd1e92dd TheCharlatan: ACK 5b3ea5fa2e7f6dc1c9161ed8b74c9be4bd1e92dd Tree-SHA512: 45cca846e7ed107e3930149f0b616ddbaf2648d6cde381f815331b861b5d67ab39e154883ae174b8abb1dae485bc904318c50c51e5d6b46923d89de51c5eadb0
2023-11-30Remove unused CDataStreamMarcoFalke
2023-11-26bench: add readblock benchmarkAndrew Toth
2023-11-24Merge bitcoin/bitcoin#28922: Use Txid in COutpointfanquake
9e58c5bcd96e7ff2062274868814ccae0626589e Use Txid in COutpoint (dergoegge) Pull request description: This PR changes the type of the hash of a transaction outpoint from `uint256` to `Txid`. ACKs for top commit: Sjors: ACK 9e58c5bcd96e7ff2062274868814ccae0626589e stickies-v: ACK 9e58c5bcd96e7ff2062274868814ccae0626589e. A sizeable diff, but very straightforward changes. Didn't see anything controversial. Left a few nits, but nothing blocking, only if you have to retouch. TheCharlatan: ACK 9e58c5bcd96e7ff2062274868814ccae0626589e Tree-SHA512: 58f61ce1c58668f689513e62072a7775419c4d5af8f607669cd8cdc2e7be9645ba14af7f9e2d65da2670da3ec1ce7fc2a744037520caf799aba212fd1ac44b34
2023-11-22Merge bitcoin/bitcoin#28904: Drop CAutoFilefanquake
4eb2a9ea4b6262bec0bc7c20cb3e684ea75caf42 streams: Drop unused CAutoFile (Anthony Towns) cde9a4b137e93f1548dffac9b396ca0b472b6187 refactor: switch from CAutoFile to AutoFile (Anthony Towns) bbd4646a2ef514c31570ca9d1475fc9ddb35bdfd blockstorage: switch from CAutoFile to AutoFile (Anthony Towns) c72ddf04db95a94e91939d46d13ab6a46fefb4be streams: Remove unused CAutoFile::GetVersion (Anthony Towns) e63f64307929ad398a23ecfaabc3664270883155 streams: Base BufferedFile on AutoFile instead of CAutoFile (Anthony Towns) Pull request description: Continuing the move away from `GetVersion()`, replace uses of `CAutoFile` with `AutoFile`. ACKs for top commit: maflcko: re-ACK 4eb2a9ea4b6262bec0bc7c20cb3e684ea75caf42 🖼 TheCharlatan: ACK 4eb2a9ea4b6262bec0bc7c20cb3e684ea75caf42 stickies-v: ACK 4eb2a9ea4b6262bec0bc7c20cb3e684ea75caf42 Tree-SHA512: 1a68c42fdb725ca4bf573e22794fe7809fea764a5f97ecb33435add3c609d40f336038fb22ab1ea72567530efd39678278c9016f92ed04891afdb310631b4e82
2023-11-21Use Txid in COutpointdergoegge
2023-11-19pool: make sure PoolAllocator uses the correct alignmentMartin Leitner-Ankerl
This changes the PoolAllocator to default the alignment to the given type. This makes the code simpler, and most importantly fixes a bug on ARM 32bit that caused OOM: The class CTxOut has a member CAmount which is an int64_t and on ARM 32bit int64_t are 8 byte aligned which is larger than the pointer alignment of 4 bytes. So for CCoinsMap to be able to use the pool, we need to use the alignment of the member instead of just alignof(void*).
2023-11-18refactor: switch from CAutoFile to AutoFileAnthony Towns
2023-11-17Merge bitcoin/bitcoin#28878: Remove version field from GetSerializeSizefanquake
83986f464c59a6517f790a960a72574e167f3f72 Include version.h in fewer places (Anthony Towns) c7b61fd61b199cbefda660c9d394bb4035a49528 Convert some CDataStream to DataStream (Anthony Towns) 1410d300df7e57a895f2697d9849a2201021c973 serialize: Drop useless version param from GetSerializeSize() (Anthony Towns) bf574a75016123309b894da895ab1c7a81731933 serialize: drop GetSerializeSizeMany (Anthony Towns) efa9eb6d7c8012fe4ed85699d81c8fe5dd18da1e serialize: Drop nVersion from [C]SizeComputer (Anthony Towns) Pull request description: Drops the version field from `GetSerializeSize()`, simplifying the code in various places. Also drop `GetSerializeSizeMany()` (as just removing the version parameter could result in silent bugs) and remove unnecessary instances of `#include <version.h>`. ACKs for top commit: maflcko: ACK 83986f464c59a6517f790a960a72574e167f3f72 📒 theuni: ACK 83986f464c59a6517f790a960a72574e167f3f72. Tree-SHA512: 36617b6dfbb1b4b0afbf673e905525fc6d623d3f568d3f86e3b9d4f69820db97d099e83a88007bfff881f731ddca6755ebf1549e8d8a7762437dfadbf434c62e
2023-11-16bench: add benchmark for wallet creation procedurefurszy
2023-11-16Merge bitcoin/bitcoin#28877: bench: Update nanobench to 4.3.11fanquake
fe434a469534766f18d7560d968deed37193835f bench: Update nanobench to 4.3.11 (TheCharlatan) Pull request description: The newest version fixes the false positive `* Turbo is enabled, CPU frequency will fluctuate` warning on AMD CPUs. The file was directly taken from the release page: https://github.com/martinus/nanobench/releases/tag/v4.3.11. Other changes from the release notes: * Check for failures in parseFile(), perf events tweaks by tommi-cujo in https://github.com/martinus/nanobench/pull/84 * Workaround missing noexcept for std::string move assignment by tommi-cujo in https://github.com/martinus/nanobench/pull/87 * removed the link by martinus in https://github.com/martinus/nanobench/pull/89 * Lots of minor cleanups by martinus in https://github.com/martinus/nanobench/pull/85 * Add linter for version & clang-format. Updated version by martinus in https://github.com/martinus/nanobench/pull/90 ACKs for top commit: fanquake: ACK fe434a469534766f18d7560d968deed37193835f - have not tested. Tree-SHA512: a8f15e1db1d993673e4b295a3bab22e67ee3c9f3c0bcbef28974fe9ff37dbb741967a526088d5b148c8d25c9d57cd3b844238100c17b23038638787461805678
2023-11-16Convert some CDataStream to DataStreamAnthony Towns
2023-11-14bench: Update nanobench to 4.3.11TheCharlatan
2023-11-14Use ParamsWrapper for witness serializationAnthony Towns
2023-11-02Merge bitcoin/bitcoin#28530: tests, bug fix: DisconnectedBlockTransactions ↵glozow
rewrite followups 9b3da70bd06b45482e7211aa95637a72bd115553 [test] DisconnectedBlockTransactions::DynamicMemoryUsage (glozow) b2d04479647af64ad7cf5ebfb6175251b2f6b72e bugfix: correct DisconnectedBlockTransactions memory usage (stickies-v) f4254e209801d6a790b5f0c251c0b32154a4e3cc assume duplicate transactions are not added to `iters_by_txid` (ismaelsadeeq) 29eb219c1247993378fce06c8f71aab20736c237 move only: move implementation code to disconnected_transactions.cpp (ismaelsadeeq) 81dfeddea70ae5feeaf79062585c2ff9f33c0ca3 refactor: update `MAX_DISCONNECTED_TX_POOL` from kb to bytes (ismaelsadeeq) Pull request description: This PR is a follow-up to fix review comments and a bugfix from #28385 The PR - Updated `DisconnectedBlockTransactions`'s `MAX_DISCONNECTED_TX_POOL` from kb to bytes. - Moved `DisconnectedBlockTransactions` implementation code to `kernel/disconnected_transactions.cpp`. - `AddTransactionsFromBlock` now assume duplicate transactions are not passed by asserting after inserting each transaction to `iters_by_txid`. - Included a Bug fix: In the current master we are underestimating the memory usage of `DisconnectedBlockTransactions`. * When adding and subtracting `cachedInnerUsage` we call `RecursiveDynamicUsage` with `CTransaction` which invokes this [`RecursiveDynamicUsage(const CTransaction& tx)`](https://github.com/bitcoin/bitcoin/blob/6e721c923c87abdb8d99674093d08d8c17bc52c2/src/core_memusage.h#L32) version of `RecursiveDynamicUsage`, the output of that call only account for the memory usage of the inputs and outputs of the `CTransaction`, this omits the memory usage of the `CTransaction` object and the control block. * This PR fixes this bug by calling `RecursiveDynamicUsage` with `CTransactionRef` when adding and subtracting `cachedInnerUsage` which invokes [`RecursiveDynamicUsage(const std::shared_ptr<X>& p)`](https://github.com/bitcoin/bitcoin/blob/6e721c923c87abdb8d99674093d08d8c17bc52c2/src/core_memusage.h#L67) version of `RecursiveDynamicUsage` the output of the calculation accounts for the` CTransaction` object, the control blocks, inputs and outputs memory usage. * see [comment ](https://github.com/bitcoin/bitcoin/pull/28385#discussion_r1322948452) - Added test for DisconnectedBlockTransactions memory limit. ACKs for top commit: stickies-v: ACK 9b3da70bd06b45482e7211aa95637a72bd115553 - nice work! BrandonOdiwuor: re ACK 9b3da70bd06b45482e7211aa95637a72bd115553 glozow: ACK 9b3da70bd06b45482e7211aa95637a72bd115553 Tree-SHA512: 69b9595d09f4d0209038f97081d790cea92ccf63efb94e9e372749979fcbe527f7f17a8e454720cedd12021be0c8e11cf99874625d3dafd9ec602b12dbeb4098
2023-10-25bugfix: Mark CNoDestination and PubKeyDestination constructor explicitMarcoFalke
This should fix the bug reported in https://github.com/bitcoin/bitcoin/pull/28246#discussion_r1371640502, which caused the GUI to not detect the destination type of recipients, thus picking the wrong change destination type. Also, add missing lifetimebound attribute to a getter method.
2023-10-13refactor: update `MAX_DISCONNECTED_TX_POOL` from kb to bytesismaelsadeeq
2023-10-12tidy: modernize-use-emplaceMarcoFalke
2023-10-04Merge bitcoin/bitcoin#27598: bench: Add SHA256 implementation specific ↵fanquake
benchmarks ce6df7df9bab2405cfe7d6e382f5682cf30de476 bench: Add SHA256 implementation specific benchmarks (Hennadii Stepanov) 5f72417176cfffece9a5aa11e97d5a6599c51e7a Add ability to specify SHA256 implementation for benchmark purposes (Hennadii Stepanov) Pull request description: On the master branch, only the best available `SHA256` implementation is being benchmarked. This PR makes `bench_bitcoin` benchmark all `SHA256` implementations that are available on the system. For example: - on Linux: ``` $ ./src/bench/bench_bitcoin -filter=SHA.* Using the 'x86_shani(1way,2way)' SHA256 implementation | ns/byte | byte/s | err% | total | benchmark |--------------------:|--------------------:|--------:|----------:|:---------- | 1.00 | 1,002,545,462.93 | 0.4% | 0.01 | `SHA1` | 2.91 | 344,117,991.18 | 0.1% | 0.03 | `SHA256 using the 'standard' SHA256 implementation` | 2.21 | 453,081,794.40 | 0.1% | 0.02 | `SHA256 using the 'sse4(1way),sse41(4way)' SHA256 implementation` | 2.21 | 453,396,506.58 | 0.1% | 0.02 | `SHA256 using the 'sse4(1way),sse41(4way),avx2(8way)' SHA256 implementation` | 0.53 | 1,870,520,687.49 | 0.1% | 0.01 | `SHA256 using the 'x86_shani(1way,2way)' SHA256 implementation` | 7.90 | 126,627,134.33 | 0.0% | 0.01 | `SHA256D64_1024 using the 'standard' SHA256 implementation` | 3.94 | 253,850,206.07 | 0.0% | 0.01 | `SHA256D64_1024 using the 'sse4(1way),sse41(4way)' SHA256 implementation` | 1.40 | 716,247,553.38 | 0.4% | 0.01 | `SHA256D64_1024 using the 'sse4(1way),sse41(4way),avx2(8way)' SHA256 implementation` | 1.26 | 792,706,270.13 | 0.9% | 0.01 | `SHA256D64_1024 using the 'x86_shani(1way,2way)' SHA256 implementation` | 6.75 | 148,172,097.64 | 0.2% | 0.01 | `SHA256_32b using the 'standard' SHA256 implementation` | 4.90 | 204,156,289.96 | 0.1% | 0.01 | `SHA256_32b using the 'sse4(1way),sse41(4way)' SHA256 implementation` | 4.90 | 204,101,274.22 | 0.1% | 0.01 | `SHA256_32b using the 'sse4(1way),sse41(4way),avx2(8way)' SHA256 implementation` | 1.70 | 589,052,595.35 | 0.4% | 0.01 | `SHA256_32b using the 'x86_shani(1way,2way)' SHA256 implementation` | 2.21 | 453,441,736.14 | 1.0% | 0.02 | `SHA3_256_1M` | 1.92 | 521,807,101.48 | 1.0% | 0.02 | `SHA512` ``` - on macOS (M1): ``` % ./src/bench/bench_bitcoin -filter=SHA.\* Using the 'arm_shani(1way,2way)' SHA256 implementation | ns/byte | byte/s | err% | total | benchmark |--------------------:|--------------------:|--------:|----------:|:---------- | 1.36 | 737,644,274.00 | 0.6% | 0.02 | `SHA1` | 3.08 | 324,556,777.15 | 0.2% | 0.03 | `SHA256 using the 'standard' SHA256 implementation` | 0.45 | 2,198,104,135.18 | 0.3% | 0.01 | `SHA256 using the 'arm_shani(1way,2way)' SHA256 implementation` | 8.84 | 113,131,299.18 | 0.0% | 0.01 | `SHA256D64_1024 using the 'standard' SHA256 implementation` | 0.94 | 1,059,406,239.36 | 0.0% | 0.01 | `SHA256D64_1024 using the 'arm_shani(1way,2way)' SHA256 implementation` | 6.17 | 162,050,659.51 | 0.2% | 0.01 | `SHA256_32b using the 'standard' SHA256 implementation` | 1.15 | 866,637,155.98 | 0.0% | 0.01 | `SHA256_32b using the 'arm_shani(1way,2way)' SHA256 implementation` | 1.69 | 592,636,491.59 | 0.2% | 0.02 | `SHA3_256_1M` | 1.89 | 528,785,775.66 | 0.0% | 0.02 | `SHA512` ``` Found it useful, while working on https://github.com/bitcoin/bitcoin/pull/24773. ACKs for top commit: martinus: ACK ce6df7df9bab2405cfe7d6e382f5682cf30de476. I would have created a helper function in the test to avoid the code duplication for each test, but that's just me nitpicking. Here are results from my Ryzen 7950X, with `./src/bench/bench_bitcoin -filter="SHA256.*" -min-time=1000`: MarcoFalke: review ACK ce6df7df9bab2405cfe7d6e382f5682cf30de476 🏵 sipa: ACK ce6df7df9bab2405cfe7d6e382f5682cf30de476 Tree-SHA512: e3de50e11b9a3a0d1e05583786041d4dc9afa2022e2115d75d6d1f63b11f62f6336f093001e53a631431d558c4dae29c596755c9e2d6aa78c382270116cc1f7f
2023-10-03refactor: Make `CCheckQueue` constructor start worker threadsHennadii Stepanov
2023-10-03Make `CCheckQueue` destructor stop worker threadsHennadii Stepanov
2023-10-02Merge bitcoin/bitcoin#27596: assumeutxo (2)Andrew Chow
edbed31066e3674ba52b8c093ab235625527f383 chainparams: add signet assumeutxo param at height 160_000 (Sjors Provoost) b8cafe38713cbf10d15459042f7f911bcc1b1e4e chainparams: add testnet assumeutxo param at height 2_500_000 (Sjors Provoost) 99839bbfa7110c7abf22e587ae2f72c9c57d3c85 doc: add note about confusing HaveTxsDownloaded name (James O'Beirne) 7ee46a755f1d57ce9d51975d3b54dc9ac3d08d52 contrib: add script to demo/test assumeutxo (James O'Beirne) 42cae39356fd20d521aaf99aff1ed85856f3c9f3 test: add feature_assumeutxo functional test (James O'Beirne) 0f64bac6030334d798ae205cd7af4bf248feddd9 rpc: add getchainstates (James O'Beirne) bb0585779472962f40d9cdd9c6532132850d371c refuse to activate a UTXO snapshot if mempool not empty (James O'Beirne) ce585a9a158476b0ad3296477b922e79f308e795 rpc: add loadtxoutset (James O'Beirne) 62ac519e718eb7a31dca1102a96ba219fbc7f95d validation: do not activate snapshot if behind active chain (James O'Beirne) 9511fb3616b7bbe1d0d2f54a45ea0a650ba0367b validation: assumeutxo: swap m_mempool on snapshot activation (James O'Beirne) 7fcd21544a333ffdf1910b65c573579860be6a36 blockstorage: segment normal/assumedvalid blockfiles (James O'Beirne) 4c3b8ca35c2e4a441264749bb312df2bd054b5b8 validation: populate nChainTx value for assumedvalid chainstates (James O'Beirne) 49ef778158c43859946a592e11ec34fe1b93a5b6 test: adjust chainstate tests to use recognized snapshot base (James O'Beirne) 1019c399825b0d512c1fd751c376d46fed4992b9 validation: pruning for multiple chainstates (James O'Beirne) 373cf91531b84bfdd06fdf8abf4dca228029ce6b validation: indexing changes for assumeutxo (James O'Beirne) 1fffdd76a1bca908f55d73b64983655b14cf7432 net_processing: validationinterface: ignore some events for bg chain (James O'Beirne) fbe0a7d7ca680358237b6c2369b3fd2b43221113 wallet: validationinterface: only handle active chain notifications (James O'Beirne) f073917a9e7ba423643dcae0339776470b628f65 validationinterface: only send zmq notifications for active (James O'Beirne) 4d8f4dcb450d31e4847804e62bf91545b949fa14 validation: pass ChainstateRole for validationinterface calls (James O'Beirne) 1e59acdf17309f567c370885f0cf02605e2baa58 validation: only call UpdatedBlockTip for active chainstate (James O'Beirne) c6af23c5179cc383f8e6c275373af8d11e6a989f validation: add ChainstateRole (James O'Beirne) 9f2318c76cc6986d48e13831cf5bd8dab194fdf4 validation: MaybeRebalanceCaches when chain leaves IBD (James O'Beirne) 434495a8c1496ca23fe35b84499f3daf668d76b8 chainparams: add blockhash to AssumeutxoData (James O'Beirne) c711ca186f8d8a28810be0beedcb615ddcf93163 assumeutxo: remove snapshot during -reindex{-chainstate} (James O'Beirne) c93ef43e4fd4fbc1263cdc9e98ae5856830fe89e bugfix: correct is_snapshot_cs in VerifyDB (James O'Beirne) b73d3bbd23220857bf17cbb6401275bf58013b72 net_processing: Request assumeutxo background chain blocks (Suhas Daftuar) Pull request description: - Background and FAQ: https://github.com/jamesob/assumeutxo-docs/tree/2019-04-proposal/proposal - Prior progress/project: https://github.com/bitcoin/bitcoin/projects/11 - Replaces https://github.com/bitcoin/bitcoin/pull/15606, which was closed due to Github slowness. Original description and commentary can be found there. --- This changeset finishes the first phase of the assumeutxo project. It makes UTXO snapshots loadable via RPC (`loadtxoutset`) and adds `assumeutxo` parameters to chainparams. It contains all the remaining changes necessary to both use an assumedvalid snapshot chainstate and do a full validation sync in the background. This may look like a lot to review, but note that - ~200 lines are a (non-essential) demo shell script - Many lines are functional test, documentation, and relatively dilute RPC code. So it shouldn't be as burdensome to review as the linecount might suggest. - **P2P**: minor changes are made to `init.cpp` and `net_processing.cpp` to make simultaneous IBD across multiple chainstates work. - **Pruning**: implement correct pruning behavior when using a background chainstate - **Blockfile separation**: to prevent "fragmentation" in blockfile storage, have background chainstates use separate blockfiles from active snapshot chainstates to avoid interleaving heights and impairing pruning. - **Indexing**: some `CValidationInterface` events are given with an additional parameter, ChainstateRole, and all indexers ignore events from ChainstateRole::ASSUMEDVALID so that indexation only happens sequentially. - Have `-reindex` properly wipe snapshot chainstates. - **RPC**: introduce RPC commands `loadtxoutset` and (hidden) `getchainstates`. - **Release docs & first assumeutxo commitment**: add notes and a particular assumeutxo hash value for first AU-enabled release. - This will complete the project and allow use of UTXO snapshots for faster node bootstrap. The next phase, if it were to be pursued, would be coming up with a way to distribute the UTXO snapshots over the P2P network. --- ### UTXO snapshots Create your own with `./contrib/devtools/utxo_snapshot.sh`, e.g. ```shell ./contrib/devtools/utxo_snapshot.sh 788000 utxo.dat ./src/bitcoin-cli -datadir=$(pwd)/testdata`) ``` or use the pre-generated ones listed below. - Testnet: **2'500'000** (Sjors): - torrent: `magnet:?xt=urn:btih:511e09f4bf853aefab00de5c070b1e031f0ecbe9&dn=utxo-testnet-2500000.dat&tr=udp%3A%2F%2Ftracker.bitcoin.sprovoost.nl%3A6969` - sha256: `79db4b025448cc0ac388d8589a28eab02de53055d181e34eb47391717aa16388` - Signet: **160'000** (Sjors): - torrent: `magnet:?xt=urn:btih:9da986cb27b3980ea7fd06b21e199b148d486880&dn=utxo-signet-160000.dat&tr=udp%3A%2F%2Ftracker.bitcoin.sprovoost.nl%3A6969` - sha256: `eeeca845385ba91e84ef58c09d38f98f246a24feadaad57fe1e5874f3f92ef8c` - Mainnet: **800'000** (Sjors): - Note: this needs the following commit cherry-picked in: https://github.com/Sjors/bitcoin/commit/24deb2022b822f22fba9fcbee201e37a83225eb2 - torrent: `magnet:?xt=urn:btih:50ee955bef37f5ec3e5b0df4cf0288af3d715a2e&dn=utxo-800000.dat&tr=udp%3A%2F%2Ftracker.bitcoin.sprovoost.nl%3A6969` ### Testing #### For fun (~5min) If you want to do a quick test, you can run `./contrib/devtools/test_utxo_snapshots.sh` and follow the instructions. This is mostly obviated by the functional tests, though. #### For real (longer) If you'd like to experience a real usage of assumeutxo, you can do that too. I've cut a new snapshot at height 788'000 (http://img.jameso.be/utxo-788000.dat - but you can do it yourself with `./contrib/devtools/utxo_snapshot.sh` if you want). Download that, and then create a datadir for testing: ```sh $ cd ~/src/bitcoin # or whatever # get the snapshot $ curl http://img.jameso.be/utxo-788000.dat > utxo-788000.dat # you'll want to do this if you like copy/pasting $ export AU_DATADIR=/home/${USER}/au-test # or wherever $ mkdir ${AU_DATADIR} $ vim ${AU_DATADIR}/bitcoin.conf dbcache=8000 # or, you know, something high blockfilterindex=1 coinstatsindex=1 prune=3000 logthreadnames=1 ``` Obtain this branch, build it, and then start bitcoind: ```sh $ git remote add jamesob https://github.com/jamesob/bitcoin $ git fetch jamesob assumeutxo $ git checkout jamesob/assumeutxo $ ./configure $conf_args && make # (whatever you like to do here) # start 'er up and watch the logs $ ./src/bitcoind -datadir=${AU_DATADIR} ``` Then, in some other window, load the snapshot ```sh $ ./src/bitcoin-cli -datadir=${AU_DATADIR} loadtxoutset $(pwd)/utxo-788000.dat ``` You'll see some log messages about headers retrieval and waiting to see the snapshot in the headers chain. Once you get the full headers chain, you'll spend a decent amount of time (~10min) loading the snapshot, checking it, and flushing it to disk. After all that happens, you should be syncing to tip in pretty short order, and you'll see the occasional `[background validation]` log message go by. In yet another window, you can check out chainstate status with ```sh $ ./src/bitcoin-cli -datadir=${AU_DATADIR} getchainstates ``` as well as usual favorites like `getblockchaininfo`. ACKs for top commit: achow101: ACK edbed31066e3674ba52b8c093ab235625527f383 Tree-SHA512: 6086fb9a38dc7df85fedc76b30084dd8154617a2a91e89a84fb41326d34ef8e7d7ea593107afba01369093bf8cc91770621d98f0ea42a5b3b99db868d2f14dc2
2023-10-02bench: drop NO_THREAD_SAFETY_ANALYSIS from disconnected_txsfanquake
2023-09-30validation: pass ChainstateRole for validationinterface callsJames O'Beirne
This allows consumers to decide how to handle events from background or assumedvalid chainstates.
2023-09-26Merge bitcoin/bitcoin#28483: refactor: Return CAutoFile from ↵fanquake
BlockManager::Open*File() fa56c421be04af846f479c30749b17e6663ab418 Return CAutoFile from BlockManager::Open*File() (MarcoFalke) 9999b89cd37fb2a23c5ebcd91d9cb31d69375933 Make BufferedFile to be a CAutoFile wrapper (MarcoFalke) fa389d902fbf5fac19fba8c5d0c5e0a25f15ca63 refactor: Drop unused fclose() from BufferedFile (MarcoFalke) Pull request description: This is required for https://github.com/bitcoin/bitcoin/pull/28052, but makes sense on its own, because offloading logic to `CAutoFile` instead of re-implementing it allows to delete code and complexity. ACKs for top commit: TheCharlatan: Re-ACK fa56c421be04af846f479c30749b17e6663ab418 willcl-ark: tACK fa56c421be Tree-SHA512: fe4638f3a6bd3f9d968cfb9ae3259c9d6cd278fe2912cbc90289851311c8c781099db4c160e775960975c4739098d9af801a8d2d12603f371f8edfe134d8f85a
generated by cgit v1.2.3 (git 2.26.2) at 2024-09-12 21:01:51 +0000