aboutsummaryrefslogtreecommitdiff
path: root/src/Makefile.bench.include
AgeCommit message (Collapse)Author
2020-06-13Replace current benchmarking framework with nanobenchMartin Ankerl
This replaces the current benchmarking framework with nanobench [1], an MIT licensed single-header benchmarking library, of which I am the autor. This has in my opinion several advantages, especially on Linux: * fast: Running all benchmarks takes ~6 seconds instead of 4m13s on an Intel i7-8700 CPU @ 3.20GHz. * accurate: I ran e.g. the benchmark for SipHash_32b 10 times and calculate standard deviation / mean = coefficient of variation: * 0.57% CV for old benchmarking framework * 0.20% CV for nanobench So the benchmark results with nanobench seem to vary less than with the old framework. * It automatically determines runtime based on clock precision, no need to specify number of evaluations. * measure instructions, cycles, branches, instructions per cycle, branch misses (only Linux, when performance counters are available) * output in markdown table format. * Warn about unstable environment (frequency scaling, turbo, ...) * For better profiling, it is possible to set the environment variable NANOBENCH_ENDLESS to force endless running of a particular benchmark without the need to recompile. This makes it to e.g. run "perf top" and look at hotspots. Here is an example copy & pasted from the terminal output: | ns/byte | byte/s | err% | ins/byte | cyc/byte | IPC | bra/byte | miss% | total | benchmark |--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:---------- | 2.52 | 396,529,415.94 | 0.6% | 25.42 | 8.02 | 3.169 | 0.06 | 0.0% | 0.03 | `bench/crypto_hash.cpp RIPEMD160` | 1.87 | 535,161,444.83 | 0.3% | 21.36 | 5.95 | 3.589 | 0.06 | 0.0% | 0.02 | `bench/crypto_hash.cpp SHA1` | 3.22 | 310,344,174.79 | 1.1% | 36.80 | 10.22 | 3.601 | 0.09 | 0.0% | 0.04 | `bench/crypto_hash.cpp SHA256` | 2.01 | 496,375,796.23 | 0.0% | 18.72 | 6.43 | 2.911 | 0.01 | 1.0% | 0.00 | `bench/crypto_hash.cpp SHA256D64_1024` | 7.23 | 138,263,519.35 | 0.1% | 82.66 | 23.11 | 3.577 | 1.63 | 0.1% | 0.00 | `bench/crypto_hash.cpp SHA256_32b` | 3.04 | 328,780,166.40 | 0.3% | 35.82 | 9.69 | 3.696 | 0.03 | 0.0% | 0.03 | `bench/crypto_hash.cpp SHA512` [1] https://github.com/martinus/nanobench * Adds support for asymptotes This adds support to calculate asymptotic complexity of a benchmark. This is similar to #17375, but currently only one asymptote is supported, and I have added support in the benchmark `ComplexMemPool` as an example. Usage is e.g. like this: ``` ./bench_bitcoin -filter=ComplexMemPool -asymptote=25,50,100,200,400,600,800 ``` This runs the benchmark `ComplexMemPool` several times but with different complexityN settings. The benchmark can extract that number and use it accordingly. Here, it's used for `childTxs`. The output is this: | complexityN | ns/op | op/s | err% | ins/op | cyc/op | IPC | total | benchmark |------------:|--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|----------:|:---------- | 25 | 1,064,241.00 | 939.64 | 1.4% | 3,960,279.00 | 2,829,708.00 | 1.400 | 0.01 | `ComplexMemPool` | 50 | 1,579,530.00 | 633.10 | 1.0% | 6,231,810.00 | 4,412,674.00 | 1.412 | 0.02 | `ComplexMemPool` | 100 | 4,022,774.00 | 248.58 | 0.6% | 16,544,406.00 | 11,889,535.00 | 1.392 | 0.04 | `ComplexMemPool` | 200 | 15,390,986.00 | 64.97 | 0.2% | 63,904,254.00 | 47,731,705.00 | 1.339 | 0.17 | `ComplexMemPool` | 400 | 69,394,711.00 | 14.41 | 0.1% | 272,602,461.00 | 219,014,691.00 | 1.245 | 0.76 | `ComplexMemPool` | 600 | 168,977,165.00 | 5.92 | 0.1% | 639,108,082.00 | 535,316,887.00 | 1.194 | 1.86 | `ComplexMemPool` | 800 | 310,109,077.00 | 3.22 | 0.1% |1,149,134,246.00 | 984,620,812.00 | 1.167 | 3.41 | `ComplexMemPool` | coefficient | err% | complexity |--------------:|-------:|------------ | 4.78486e-07 | 4.5% | O(n^2) | 6.38557e-10 | 21.7% | O(n^3) | 3.42338e-05 | 38.0% | O(n log n) | 0.000313914 | 46.9% | O(n) | 0.0129823 | 114.4% | O(log n) | 0.0815055 | 133.8% | O(1) The best fitting curve is O(n^2), so the algorithm seems to scale quadratic with `childTxs` in the range 25 to 800.
2020-04-29Add Hash Padding MicrobenchmarksJeremy Rubin
2020-04-21bench: add CAddrMan benchmarksVasil Dimov
The added benchmarks exercise the public methods Add(), GetAddr(), Select() and Good().
2020-03-26Merge #18429: build: remove double LIBBITCOIN_SERVER from bench-Makefilefanquake
1f97b69ba27deb645bf5dd229d0cb97d2baf8f49 build: remove double LIBBITCOIN_SERVER from bench-Makefile (Harris) Pull request description: This PR removes the redundant **LIBBITCOIN_SERVER** linking from bench's Makefile. This PR is similar to https://github.com/bitcoin/bitcoin/pull/17910 Originally, this PR was part of https://github.com/bitcoin/bitcoin/pull/18377, which later got replaced by a better one https://github.com/bitcoin/bitcoin/pull/18397 written by **hebasto**. ACKs for top commit: Empact: Code Review ACK https://github.com/bitcoin/bitcoin/pull/18429/commits/1f97b69ba27deb645bf5dd229d0cb97d2baf8f49 theStack: ACK https://github.com/bitcoin/bitcoin/pull/18429/commits/1f97b69ba27deb645bf5dd229d0cb97d2baf8f49 hebasto: ACK 1f97b69ba27deb645bf5dd229d0cb97d2baf8f49 Tree-SHA512: e43035262361d4458a7dcfc920445540f19301387814cde1be0539c936fc20da0dcbe49e5ea25385e6d36d9639515b7a4171228223da568d93427e9c32810945
2020-03-25build: remove double LIBBITCOIN_SERVER from bench-MakefileHarris
2020-03-21build: Fix typo in EVENT_CFLAGS variableHennadii Stepanov
Co-authored-by: Harris <brakmic@gmail.com>
2019-11-25[test] move wallet helper functions into test libraryMartin Zumsande
2019-11-21build: Create test utility library from src/test/util/Harris
2019-11-18build: remove OpenSSL detection and libsfanquake
2019-11-06scripted-diff: test: Move setup_common to test libraryMarcoFalke
-BEGIN VERIFY SCRIPT- # Move files for f in $(git ls-files src/test/lib/); do git mv $f src/test/util/; done git mv src/test/setup_common.cpp src/test/util/ git mv src/test/setup_common.h src/test/util/ # Replace Windows paths sed -i -e 's|\\setup_common|\\util\\setup_common|g' $(git grep -l '\\setup_common') sed -i -e 's|src\\test\\lib\\|src\\test\\util\\|g' build_msvc/test_bitcoin/test_bitcoin.vcxproj # Everything else sed -i -e 's|/setup_common|/util/setup_common|g' $(git grep -l 'setup_common') sed -i -e 's|test/lib/|test/util/|g' $(git grep -l 'test/lib/') # Fix include guard sed -i -e 's|BITCOIN_TEST_SETUP_COMMON_H|BITCOIN_TEST_UTIL_SETUP_COMMON_H|g' ./src/test/util/setup_common.h sed -i -e 's|BITCOIN_TEST_LIB_|BITCOIN_TEST_UTIL_|g' $(git grep -l 'BITCOIN_TEST_LIB_') -END VERIFY SCRIPT-
2019-11-01Merge #17292: Add new mempool benchmarks for a complex poolMarcoFalke
b0c774b48a3198584e61429ef053844bdde2f9ae Add new mempool benchmarks for a complex pool (Jeremy Rubin) Pull request description: This PR is related to #17268. It adds a mempool stress test which makes a really big complicated tx graph, and then, similar to mempool_eviction test, trims the size. The test setup is to make 100 original transactions with Rand(10)+2 outputs each. Then, 800 times: we create a new transaction with Rand(10) + 1 parents that are randomly sampled from all existing transactions (with unspent outputs). From each such parent, we then select Rand(remaining outputs) +1 50% of the time, or 1 outputs 50% of the time. Then, we trim the size to 3/4. Then we trim it to just a single transaction. This creates, hopefully, a big bundle of transactions with lots of complex structure, that should really put a strain on the mempool graph algorithms. This ends up testing both the descendant and ancestor tracking. I don't love that the test is "unstable". That is, in order to compare this test to another, you really can't modify any of the internal state because it will have a different order of invocations of the deterministic randomness. However, it certainly suffices for comparing branches. Top commit has no ACKs. Tree-SHA512: cabe96b849b9885878e20eec558915e921d49e6ed1e4b011b22ca191b4c99aa28930a8b963784c9adf78cc8b034a655513f7a0da865e280a1214ae15ebb1d574
2019-10-28Add new mempool benchmarks for a complex poolJeremy Rubin
2019-10-23refactor: test/bench: dedup Build{Crediting,Spending}Transaction()Sebastian Falbesoner
prototypes used in src/test/script_tests.cpp: - CMutableTransaction BuildCreditingTransaction(const CScript& scriptPubKey, int nValue = 0); - CMutableTransaction BuildSpendingTransaction(const CScript& scriptSig, const CScriptWitness& scriptWitness, const CTransaction& txCredit); prototypes used in bench/verify_script.cpp: - CMutableTransaction BuildCreditingTransaction(const CScript& scriptPubKey); - CMutableTransaction BuildSpendingTransaction(const CScript& scriptSig, const CMutableTransaction& txCredit); The more generic versions from the script tests are moved into a new file pair transaction_utils.cpp/h and the calls are adapted accordingly in the verify_script benchmark (passing the nValue of 1 explicitely for BuildCreditingTransaction(), passing empty scriptWitness explicitely and converting txCredit parameter to CTransaction in BuildSpendingTransaction()).
2019-07-11Merge #15649: Add ChaCha20Poly1305@Bitcoin AEADWladimir J. van der Laan
bb326add9f38f2a8e5ce5ee29d98ce08038200d8 Add ChaCha20Poly1305@Bitcoin AEAD benchmark (Jonas Schnelli) 99aea045d688059caf89c0e485fa427bd28eddd8 Add ChaCha20Poly1305@Bitcoin tests (Jonas Schnelli) af5d1b5f4a7b56628a76af21284c258d845894f0 Add ChaCha20Poly1305@Bitcoin AEAD implementation (Jonas Schnelli) Pull request description: This adds a new AEAD (authenticated encryption with additional data) construct optimised for small messages (like used in Bitcoins p2p network). Includes: #15519, #15512 (please review those first). The construct is specified here. https://gist.github.com/jonasschnelli/c530ea8421b8d0e80c51486325587c52#ChaCha20Poly1305Bitcoin_Cipher_Suite This aims for being used in v2 peer-to-peer messages. ACKs for top commit: laanwj: code review ACK bb326add9f38f2a8e5ce5ee29d98ce08038200d8 Tree-SHA512: 15bcb86c510fce7abb7a73536ff2ae89893b24646bf108c6cf18f064d672dbbbea8b1dd0868849fdac0c6854e498f1345d01dab56d1c92031afd728302234686
2019-07-05bench: Benchmark blockToJSONKirill Fomichev
2019-07-03Add ChaCha20Poly1305@Bitcoin AEAD benchmarkJonas Schnelli
2019-07-02bench: Move generated data to a dedicated translation unitJoão Barbosa
2019-05-23util: Add type safe GetTimeMarcoFalke
2019-05-10Merge #15512: Add ChaCha20 encryption option (XOR)Jonas Schnelli
2dfe27517 Add ChaCha20 bench (Jonas Schnelli) 2bc2b8b49 Add ChaCha20 encryption option (XOR) (Jonas Schnelli) Pull request description: The current ChaCha20 implementation does not support message encryption (it can only output the keystream which is sufficient for the RNG). This PR adds the actual XORing of the `plaintext` with the `keystream` in order to return the desired `ciphertext`. Required for v2 message transport protocol. ACKs for commit 2dfe27: jnewbery: Looks good. utACK 2dfe2751713c814aea53b5a7563eb74ad1baea00. jnewbery: utACK 2dfe2751713c814aea53b5a7563eb74ad1baea00 sipa: utACK 2dfe2751713c814aea53b5a7563eb74ad1baea00 ryanofsky: utACK 2dfe2751713c814aea53b5a7563eb74ad1baea00. Changes since last review are just renaming the Crypt method, adding comments, and simplifying the benchmark. Tree-SHA512: 84bb234da2ca9fdc44bc29a786d9dd215520f81245270c1aef801ef66b6091b7793e2eb38ad6dbb084925245065c5dce9e5582f2d0fa220ab3e182d43412d5b5
2019-05-03Add ChaCha20 benchJonas Schnelli
2019-04-15bench: Add wallet_balance benchmarksMarcoFalke
2019-04-11scripted-diff: Rename test_bitcoin to test/setup_commonMarcoFalke
-BEGIN VERIFY SCRIPT- sed -i --regexp-extended -e 's/test_bitcoin\.(h|cpp)/setup_common.\1/g' $(git grep -l test_bitcoin) git mv ./src/test/test_bitcoin.h ./src/test/setup_common.h git mv ./src/test/test_bitcoin.cpp ./src/test/setup_common.cpp sed -i -e 's/BITCOIN_TEST_TEST_BITCOIN_H/BITCOIN_TEST_SETUP_COMMON_H/g' ./src/test/setup_common.h -END VERIFY SCRIPT-
2019-04-10test: Use test_bitcoin setup in bench, Add test utilsMarcoFalke
2019-03-27Merge #15519: Add Poly1305 implementationWladimir J. van der Laan
e9d5e975612e828ec44f9247b4c5c08f0268d360 Poly1305: tolerate the intentional unsigned wraparound in poly1305.cpp (Jonas Schnelli) b34bf302f26c7cede47cc20b3bdfb613c51ab67e Add Poly1305 bench (Jonas Schnelli) 03be7f48fad10aa8da3291c28a185ed750193c7b Add Poly1305 implementation (Jonas Schnelli) Pull request description: This adds a currently unused Poly1305 implementation including test vectors from RFC7539. Required for BIP151 (and related to #15512). Tree-SHA512: f8c1ad2f686b980a7498ca50c517e2348ac7b1fe550565156f6c2b20faf764978e4fa6b5b1c3777a16e7a12e2eca3fb57a59be9c788b00d4358ee80f2959edb1
2019-03-26Add Poly1305 benchJonas Schnelli
2019-03-06Merge #15473: bench: Benchmark MempoolToJSONMarcoFalke
fa38535130 bench: Benchmark MempoolToJSON (MarcoFalke) fa5dc3534b rpc: Pass mempool into MempoolToJSON (MarcoFalke) Pull request description: This is used in production (e.g. https://jochen-hoenicke.de/queue/#0,24h), so add a benchmark to avoid making it even slower. Related: * "getrawmempool true RPC call is O(n^2)" #14765 Tree-SHA512: da09d2e54ee261af8671152f97f863cf1acd7a6adc6578e94046b1ec9e647a670c67499760ef765254f65522dfdf773c3c8729006fa2d63ccb6d53166bafc425
2019-02-25bench: Benchmark MempoolToJSONMarcoFalke
2019-02-22Remove use of uiInterface.LoadWallet in wallet codeRussell Yanofsky
This also changes the uiInterface.LoadWallet signal argument type from shared_ptr<CWallet> to unique_ptr<interfaces::Wallet> because CWallet is an internal wallet class that shouldn't be used in non-wallet code (and also can't be passed across process boundaries). This commit does not change behavior.
2019-02-22Remove uses of g_connman in wallet codeRussell Yanofsky
This commit does not change behavior.
2018-11-25Add Benchmark to test input de-duplication worst caseJeremy Rubin
Fix nits replace utiltime?
2018-09-13build: Remove libssl from LDADD unless guiMarcoFalke
2018-08-25bench: Benchmark GCS filter creation and matching.Jim Posen
2018-07-11Merge #13586: refactor: add benchmarks to bech32::Encode/DecodeWladimir J. van der Laan
189cf35f3e6d2cc9ed08eb23dd0ea36be28b6c11 Add simple bech32 benchmarks (Karl-Johan Alm) Pull request description: This PR adds benchmarks to `Encode()`/`Decode()`. The benchmark commit is duplicated in #13632. Tree-SHA512: 102a193e4af58c9cb23c66d3dc7e174aa6328edab0ed74f92deb7804db5c3d0601807b3e25a5472b5c72d6113cde0dbc9976315644671a8f14ecf349967dbaaa
2018-07-03Add simple bech32 benchmarksKarl-Johan Alm
2018-06-25Break circular dependency: init -> * -> init by extracting shutdown.hBen Woosley
Most includers just wanted to react to pending shutdown. This isolates access to `fRequestShutdown` and limits access to the shutdown api functions, including the new `AbortShutdown` for setting it to `false`. Note I originally called `AbortShutdown` `CancelShutdown` but that name was already taken by winuser.h https://travis-ci.org/bitcoin/bitcoin/jobs/386913329 This change also triggered a build error in bench. Fixing it required moving LIBBITCOIN_SERVER after LIBBITCOIN_WALLET in bench_bench_bitcoin_LDADD To make server definitions in src/net.cpp available to wallet methods in src/wallet/wallet.cpp. Specifically, solving: libbitcoin_wallet.a(libbitcoin_wallet_a-wallet.o): In function `CWalletTx::RelayWalletTransaction(CConnman*)': wallet.cpp:(.text+0x3f0e): undefined reference to `CConnman::NodeFullyConnected(CNode const*)' collect2: error: ld returned 1 exit status https://travis-ci.org/bitcoin/bitcoin/jobs/392133581 Need for remaining init.h includes confirmed via a thorough search with a more specific regex: \bInterrupt\(\)|\bShutdown\(\)|\bInitLogging\(\)|\bInitParameterInteraction\(\)|\bAppInitBasicSetup\(\)|\bAppInitParameterInteraction\(\)|\bAppInitSanityChecks\(\)|\bAppInitLockDataDirectory\(\)|\bAppInitMain\(\)|\bSetupServerArgs\(\)|\bLicenseInfo\(\)|g_wallet_init_interface|init.h
2018-06-24Merge #13219: bench: Add block assemble benchmarkWladimir J. van der Laan
fa0fc1bc7edc7f1dd419db6776dcf89749cc1a00 bench: Add block assemble benchmark (MarcoFalke) Pull request description: Tree-SHA512: 614da28020b76154f4cb17752df226466d2d7da7a81f31812e10ca5b94987c7f5abab62b2f45a3a958663bd8cbf2191f3f23e497215196ad7a775f0ea8fb6116
2018-06-12Merge #13312: docs: Add a note about the source code filename naming conventionMarcoFalke
e56771365b Do not use uppercase characters in source code filenames (practicalswift) 419a1983ca docs: Add a note about the source code filename naming convention (practicalswift) Pull request description: Add a note about the source code filename naming convention. Tree-SHA512: 8d329bd9e19bcd26e74b0862fb0bc2369b46095dbd3e69d34859908632763abd7c3d00ccc44ee059772ad4bae4460c2bcc1c0e22fd9d8876d57e5fcd346cea4b
2018-06-06crypto: cleanup sha256 buildCory Fields
Rather than appending all possible cpu variants to all targets, create a convenience variable that encompasses all.
2018-05-298-way AVX2 implementation for double SHA256 on 64-byte inputsPieter Wuille
2018-05-294-way SSE4.1 implementation for double SHA256 on 64-byte inputsPieter Wuille
2018-05-23Do not use uppercase characters in source code filenamespracticalswift
2018-05-13bench: Add block assemble benchmarkMarcoFalke
2018-05-08Benchmark Merkle root computationPieter Wuille
2018-04-18benchmark: Removed bench/perf.cppThomas Snider
2018-02-27Add new prevector benchmarks.Evan Klitzke
This prepares for a series of two additional commits which optimize prevector performance.
2018-02-16Create getaddressinfo RPC and deprecate parts of validateaddressAndrew Chow
Moves the parts of validateaddress which require the wallet into getaddressinfo which is part of the wallet RPCs. Mark those parts of validateaddress which require the wallet as deprecated. Validateaddress will call getaddressinfo for the data that both share for right now. Moves IsMine functions to libbitcoin_common and then links libbitcoin_wallet before libbitcoin_common in order to prevent linker errors since IsMine is no longer used in libbitcoin_server.
2017-10-04build: Make "make clean" remove all files created when running "make check"practicalswift
More specifically: remove also obj/build.h and bench/data/block413567.raw.h. Before this patch: ``` $ diff -rq bitcoin-before-make/ bitcoin-after-make-and-make-clean/ | grep -E "^Only in bitcoin-after-make-and-make-clean/" | grep -v dirstamp Only in bitcoin-after-make-and-make-clean/src/bench/data: block413567.raw.h Only in bitcoin-after-make-and-make-clean/src/obj: build.h $ ``` After this patch: ``` $ diff -rq bitcoin-before-make/ bitcoin-after-make-and-make-clean/ | grep -E "^Only in bitcoin-after-make-and-make-clean/" | grep -v dirstamp $ ```
2017-06-09leveldb: enable runtime-detected crc32 instructionsCory Fields
2017-03-14Merge #9505: Prevector Quick DestructWladimir J. van der Laan
45a5aaf Only call clear on prevector if it isn't trivially destructible and don't loop in clear (Jeremy Rubin) aaa02e7 Add prevector destructor benchmark (Jeremy Rubin) Tree-SHA512: 52bc8163b65b71310252f2d578349d0ddc364a6c23795c5e06e101f5449f04c96cbdca41c0cffb1974b984b8e33006471137d92b8dd4a81a98e922610a94132a
2017-01-10Add prevector destructor benchmarkJeremy Rubin
generated by cgit v1.2.3 (git 2.26.2) at 2024-11-09 16:08:46 +0000