aboutsummaryrefslogtreecommitdiff
path: root/src/test
AgeCommit message (Collapse)Author
3 daysMerge bitcoin/bitcoin#30644: fuzz: Faster utxo_snapshot fuzz targetAva Chow
fa899fb7aa8a14acecadd8936ad5824fa0f697ff fuzz: Speed up utxo_snapshot fuzz target (MarcoFalke) fa386642b4dfd88f74488c288c7886494d69f4ed fuzz: Speed up utxo_snapshot by lazy re-init (MarcoFalke) fa645c7a861ffa83a53a459263b6a620defe31f9 fuzz: Remove unused DataStream object (MarcoFalke) fae8c73d9e4eba4603447bb52b6e3e760fbf15f8 test: Disallow fee_estimator construction in ChainTestingSetup (MarcoFalke) Pull request description: Two commits to speed up unit and fuzz tests. Can be tested by running the fuzz target and looking at the time it took, or by looking at the flamegraph. For example: ``` FUZZ=utxo_snapshot perf record -g --call-graph dwarf ./src/test/fuzz/fuzz -runs=100 hotspot ./perf.data ACKs for top commit: TheCharlatan: Re-ACK fa899fb7aa8a14acecadd8936ad5824fa0f697ff marcofleon: Re ACK fa899fb7aa8a14acecadd8936ad5824fa0f697ff brunoerg: ACK fa899fb7aa8a14acecadd8936ad5824fa0f697ff Tree-SHA512: d3a771bb12d7ef491eee61ca47325dd1cea5c20b6ad42554babf13ec98d03bef8e7786159d077e59cc7ab8112495037b0f6e55edae65b871c7cf1708687cf717
4 daysfuzz: Speed up utxo_snapshot fuzz targetMarcoFalke
This speeds up the fuzz target, which allows "valid" inputs. It does not affect the "INVALID" fuzz target.
8 daysfuzz: Speed up utxo_snapshot by lazy re-initMarcoFalke
The re-init is expensive, so skip it if there is no need. Also, add an even faster fuzz target utxo_snapshot_invalid, which does not need any re-init at all.
9 daysfuzz: Remove unused DataStream objectMarcoFalke
11 daysMerge bitcoin/bitcoin#30617: net: Clarify that m_addr_local is only set onceglozow
fa6fe432075df5e0eceb1ccd85038159cc820ccc net: Clarify that m_addr_local is only set once (MarcoFalke) Pull request description: The function is supposed to be only called once when the version msg arrives (a single time). Calling it twice would be an internal logic bug. However, the `LogError` in this function has many issues: * If the error happens in tests, as is the case for the buggy fuzz test, it will go unnoticed * It is dead code, unless a bug is introduced to execute it Fix all issues by using `Assume(!m_addr_local.IsValid())` instead. Idea taken from https://github.com/bitcoin/bitcoin/pull/30364#discussion_r1680530382 ACKs for top commit: achow101: ACK fa6fe432075df5e0eceb1ccd85038159cc820ccc mzumsande: utACK fa6fe432075df5e0eceb1ccd85038159cc820ccc glozow: ACK fa6fe432075df5e0eceb1ccd85038159cc820ccc Tree-SHA512: 8c1e8c524768f4f36cc50110ae54ee423e057a963ff78f736f3bf92df1ce5af28e3e0149153780897944e1d5c22ddbca9dac9865d9f4d44afffa152bc8559405
11 daystest: Disallow fee_estimator construction in ChainTestingSetupMarcoFalke
It is expensive to construct, and only one test uses it. Fix both issues by disallowing the construction and moving it to the single test that uses it.
12 daysMerge bitcoin/bitcoin#28209: fuzz: a target for the block index databaseAva Chow
86b38529d5014612c3e7bb59fdc4dad3bff2aa64 qa: a fuzz target for the block index database (Antoine Poinsot) Pull request description: This introduces a small fuzz target for `CBlockTreeDB` which asserts a few invariants by using an in-memory LevelDb. ACKs for top commit: achow101: ACK 86b38529d5014612c3e7bb59fdc4dad3bff2aa64 TheCharlatan: Re-ACK 86b38529d5014612c3e7bb59fdc4dad3bff2aa64 maflcko: review ACK 86b38529d5014612c3e7bb59fdc4dad3bff2aa64 🥒 brunoerg: utACK 86b38529d5014612c3e7bb59fdc4dad3bff2aa64 Tree-SHA512: ab75b4ae1c7e0a4b15f8a6ceffdf509fbc79833e6ea073ecef68558d53b83663d1b30362aaa2d77c22b8890a572f5b1d4b1c5abbca483c8c8f9b1fb5b276a59a
12 daysMerge bitcoin/bitcoin#30616: test: remove `ExtractDestination` false ↵glozow
assertion for `ANCHOR` script a4f2b185732649eeea4a042cebd90d0e0e12cc92 [test]: remove `ExtractDestination` false assertion for `ANCHOR` script (ismaelsadeeq) Pull request description: This PR fixes #30615 `ExtractDestination` returns `true` when `TxoutType` is `ANCHOR` see https://github.com/bitcoin/bitcoin/issues/30615#issuecomment-2277538703 ACKs for top commit: maflcko: review ACK a4f2b185732649eeea4a042cebd90d0e0e12cc92 instagibbs: ACK a4f2b185732649eeea4a042cebd90d0e0e12cc92 theStack: utACK a4f2b185732649eeea4a042cebd90d0e0e12cc92 BrandonOdiwuor: Code Review ACK a4f2b185732649eeea4a042cebd90d0e0e12cc92 glozow: ACK a4f2b185732649eeea4a042cebd90d0e0e12cc92 Tree-SHA512: 6120494fe888acf26b252d4aadc01dc256e294ea5e4c954fd9b4694be27dc35cf0e33e3b3bcb012fb4abe1cab0b1d0d515db226fa771e791e0fe7efbcbd8834d
2024-08-09Merge bitcoin/bitcoin#30598: assumeutxo: Drop block height from metadataAva Chow
00618e8745192d209c23e3ae873c077e58168957 assumeutxo: Drop block height from metadata (Fabian Jahr) Pull request description: Fixes https://github.com/bitcoin/bitcoin/issues/30514 which has more context and shows how the issue can be reproduced. Since the value in question is removed, there is no test to add to reproduce anything. This is an alternative approach to #30516 with much of the [code being suggested there](https://github.com/bitcoin/bitcoin/pull/30516#discussion_r1689146902). ACKs for top commit: maflcko: re-ACK 00618e8745192d209c23e3ae873c077e58168957 🎌 achow101: ACK 00618e8745192d209c23e3ae873c077e58168957 theStack: Code-review ACK 00618e8745192d209c23e3ae873c077e58168957 ismaelsadeeq: Re-ACK 00618e8745192d209c23e3ae873c077e58168957 mzumsande: ACK 00618e8745192d209c23e3ae873c077e58168957 Tree-SHA512: db9575247bae838ad7742a27a216faaf55bb11e022f9afdd05752bb09bbf9614717d0ad64304ff5722a16bf41d8dea888af544e4ae26dcaa528c1add0269a4a8
2024-08-09net: Clarify that m_addr_local is only set onceMarcoFalke
Includes a rename from addrLocal to m_addr_local to match the name of its corresponding Mutex.
2024-08-09[test]: remove `ExtractDestination` false assertion for `ANCHOR` scriptismaelsadeeq
2024-08-09Merge bitcoin/bitcoin#28687: C++20 std::views::reversemerge-script
2925bd537cbd8c70594e23f6c4298b7101f7f73d refactor: use c++20 std::views::reverse instead of reverse_iterator.h (stickies-v) Pull request description: C++20 introduces [`std::ranges::views::reverse`](https://en.cppreference.com/w/cpp/ranges/reverse_view), which allows us to drop our own `reverse_iterator.h` implementation and also makes it easier to chain views (even though I think we currently don't use this). ACKs for top commit: achow101: ACK 2925bd537cbd8c70594e23f6c4298b7101f7f73d maflcko: ACK 2925bd537cbd8c70594e23f6c4298b7101f7f73d 🎷 Tree-SHA512: 567666ec44af5d1beb7a271836bcc89c4c577abc77f522fcc18bc6d4de516ae9b0df766d0bfa6dd217569e6878331c2aee1d9815620860375e3510dad7fed476
2024-08-08assumeutxo: Drop block height from metadataFabian Jahr
The Snapshot format version is updated to 2 to indicate this change. Co-authored-by: MarcoFalke <*~=`'#}+{/-|&$^_@721217.xyz>
2024-08-07Merge bitcoin/bitcoin#28280: Don't empty dbcache on prune flushes: >30% ↵Ava Chow
faster IBD 589db872e116779ab9cae693171ac8a8c02d9923 validation: don't erase coins cache on prune flushes (Andrew Toth) 0e8918755f725b6269ed2be5a0b46f1611233515 Add linked-list test to CCoinsViewCache::SanityCheck (Pieter Wuille) 05cf4e18758618bb493d26014d3a9c89bf28d898 coins: move Sync logic to CoinsViewCacheCursor (Andrew Toth) 7825b8b9aeceb4ff607650cdc9c49e5de9c7719f coins: pass linked list of flagged entries to BatchWrite (Andrew Toth) a14edada8a051e280af6fedd5130be40247e2d7a test: add cache entry linked list tests (Andrew Toth) 24ce37cb867b95e86d9fd4e50858d64ee8a59abf coins: track flagged cache entries in linked list (Andrew Toth) 58b7ed156d5993b69375bb455b03bd8b17f64fa4 coins: call ClearFlags in CCoinsCacheEntry destructor (Andrew Toth) 8bd3959feaa0e71585bc5e0976f584fb06ee6d14 refactor: require self and sentinel parameters for AddFlags (Andrew Toth) 75f36d241d2a344c5c4ce2c80250bdde91b3295e refactor: add CoinsCachePair alias (Andrew Toth) f08faeade2f99ae1de6f3c481697541cc16186c7 refactor: move flags to private uint8_t and rename to m_flags (Andrew Toth) 4e4fb4cbabcc10e723534f5f80f3e3cf09f6ca50 refactor: disallow setting flags in CCoinsCacheEntry constructors (Andrew Toth) 8737c0cefa6ec49a4d17d9bef9e5e1a7990af1ac refactor: encapsulate flags setting with AddFlags and ClearFlags (Andrew Toth) 9715d3bf1e4013476539f1523a6b54d2055c32c6 refactor: encapsulate flags get access for all other checks (Andrew Toth) df34a94e57659c31873c26c86a8de5a032400061 refactor: encapsulate flags access for dirty and fresh checks (Andrew Toth) Pull request description: Since https://github.com/bitcoin/bitcoin/pull/17487 we no longer need to clear the coins cache when syncing to disk. A warm coins cache significantly speeds up block connection, and only needs to be fully flushed when nearing the `dbcache` limit. For frequent pruning flushes there's no need to empty the cache and kill connect block speed. However, simply using `Sync` in place of `Flush` actually slows down a pruned full IBD with a high `dbcache` value. This is because as the cache grows, sync takes longer since every coin in the cache is scanned to check if it's dirty. For frequent prune flushes and a large cache this constant scanning starts to really slow IBD down, and just emptying the cache on every prune becomes faster. To fix this, we can add two pointers to each cache entry and construct a doubly linked list of dirty entries. We can then only iterate through all dirty entries on each `Sync`, and simply clear the pointers after. With this approach a full IBD with `dbcache=16384` and `prune=550` was 32% faster than master. For default `dbcache=450` speedup was ~9%. All benchmarks were run with `stopatheight=800000`. | | prune | dbcache | time | max RSS | speedup | |-----------:|----------:|------------:|--------:|-------------:|--------------:| | master | 550 | 16384 | 8:52:57 | 2,417,464k | - | | branch | 550 | 16384 | 6:01:00 | 16,216,736k | 32% | | branch | 550 | 450 | 8:05:08 | 2,818,072k | 8.8% | | master | 10000 | 5000 | 8:19:59 | 2,962,752k | - | | branch | 10000 | 5000| 5:56:39 | 6,179,764k | 28.8% | | master | 0 | 16384 | 4:51:53 | 14,726,408k | - | | branch | 0 | 16384 | 4:43:11 | 16,526,348k | 2.7% | | master | 0 | 450 | 7:08:07 | 3,005,892k | - | | branch | 0 | 450 | 6:57:24 | 3,013,556k |2.6%| While the 2 pointers add memory to each cache entry, it did not slow down IBD. For non-pruned IBD results were similar for this branch and master. When I performed the initial IBD, the full UTXO set could be held in memory when using the max `dbcache` value. For non-pruned IBD with max `dbcache` to tip ended up using 12% more memory, but it was also 2.7% faster somehow. For smaller `dbcache` values the `dbcache` limit is respected so does not consume more memory, and the potentially more frequent flushes were not significant enough to cause any slowdown. For reviewers, the commits in order do the following: First 4 commits encapsulate all accesses to `flags` on cache entries, and then the 5th makes `flags` private. Commits `refactor: add CoinsCachePair alias` to `coins: call ClearFlags in CCoinsCacheEntry destructor` create the linked list head nodes and cache entry self references and pass them into `AddFlags`. Commit `coins: track flagged cache entries in linked list` actually adds the entries into a linked list when they are flagged DIRTY or FRESH and removes them from the linked list when they are destroyed or the flags are cleared manually. However, the linked list is not yet used anywhere. Commit `test: add cache entry linked list tests` adds unit tests for the linked list. Commit `coins: pass linked list of flagged entries to BatchWrite` uses the linked list to iterate through DIRTY entries instead of using the entire coins cache. Commit `validation: don't erase coins cache on prune flushes` uses `Sync` instead of `Flush` for pruning flushes, so the cache is no longer cleared. Inspired by [this comment](https://github.com/bitcoin/bitcoin/pull/15265#issuecomment-457720636). Fixes https://github.com/bitcoin/bitcoin/issues/11315. ACKs for top commit: paplorinc: ACK 589db872e116779ab9cae693171ac8a8c02d9923 sipa: reACK 589db872e116779ab9cae693171ac8a8c02d9923 achow101: ACK 589db872e116779ab9cae693171ac8a8c02d9923 mzumsande: re-ACK 589db872e116779ab9cae693171ac8a8c02d9923 Tree-SHA512: 23b2bc01c83edacb5b39aa60bb0b766de9a74ce17f0c59bf13b97b4328a7b758ad9aff6581c3ca88e2973f7658380651530d497444f48d6e22ea0bfc51cc921d
2024-08-07Merge bitcoin/bitcoin#29775: Testnet4 including PoW difficulty adjustment fixAva Chow
6bfa26048dbafb91e9ca63ea8d3960271e798098 testnet: Add timewarp attack prevention for Testnet4 (Fabian Jahr) 0100907ca168c53e8fe044bdda396f308825162c testnet: Add Testnet4 difficulty adjustment rules fix (Fabian Jahr) 74a04f9e7ad6a16988149cc3438b9ce13c91cdb9 testnet: Introduce Testnet4 (Fabian Jahr) Pull request description: To supplement the [ongoing conceptual discussion about a testnet reset](https://groups.google.com/g/bitcoindev/c/9bL00vRj7OU/m/9yCPo3uUBwAJ) I have drafted a move to v4 including a fix to the difficulty adjustment mechanism, which was part of the motivation that started the discussion. Conceptual considerations: - The conceptual discussion about doing a testnet4 or softforking the fix into testnet3 is outside of the scope of this PR and I would ask reviewers to contribute their opinions on this on the ML instead. However, I am happy to adapt this PR to a softfork change on testnet3 if there is consensus for that instead. - The difficulty adjustment fix suggested here touches the `CalculateNextWorkRequired` function and uses the same logic used in `GetNextWorkRequired` to find the last previous block that was not mined with difficulty 1 under the exceptionf. An alternative fix briefly mentioned on the mailing list by Jameson Lopp would be to "restrict the special testnet minimum difficulty rule so that it can't be triggered on the block right before a difficulty retarget". That would also fix the issue but I find my suggestion here a bit more elegant. ACKs for top commit: jsarenik: tACK 6bfa26048dba achow101: ACK 6bfa26048dbafb91e9ca63ea8d3960271e798098 murchandamus: tACK 6bfa26048dbafb91e9ca63ea8d3960271e798098 Tree-SHA512: 0b8b69a621406a944da5be551b863d065358ba94d85dd3b80d83c412660e230ee93b27316081fbee9b4851cc4ff8585db64c7dfa26cb5148ac835663f2712c3d
2024-08-06Merge bitcoin/bitcoin#30051: crypto, refactor: add new KeyPair classRyan Ofsky
ec973dd19719541dbcd6f3a6facf6f5dd7cf439c refactor: remove un-tested early returns (josibake) 72a5822d43d47431b2838ebfcb1f2e21210f5ccb tests: add tests for KeyPair (josibake) cebb08b121ce8c4c5e68bd043b8668c106e31169 refactor: move SignSchnorr to KeyPair (josibake) c39fd39ba868253b5118db2e1ac1461d29f0b4ce crypto: add KeyPair wrapper class (josibake) 5d507a0091da1b6c013b00b6c76e19dd4d3b93a7 tests: add key tweak smoke test (josibake) f14900b6e4eac26ae5f1c0badfa176d895851c97 bench: add benchmark for signing with a taptweak (josibake) Pull request description: Broken out from #28201 --- The wallet returns an untweaked internal key for taproot outputs. If the output commits to a tree of scripts, this key needs to be tweaked with the merkle root. Even if the output does not commit to a tree of scripts, BIP341/342 recommend commiting to a hash of the public key. Previously, this logic for applying the taptweak was implemented in the `CKey::SignSchnorr` method. This PR moves introduces a KeyPair class which wraps a `secp256k1_keypair` type and refactors SignSchnorr to use this new KeyPair. The KeyPair class is created with an optional merkle_root argument and the logic from BIP341 is applied depending on the state of the merkle_root argument. The motivation for this refactor is to be able to use the tap tweak logic outside of signing, e.g. in silent payments when retrieving the private key (see #28201). Outside of silent payments, since we almost always convert a `CKey` to a `secp256k1_keypair` when doing anything with taproot keys, it seems generally useful to have a way to model this type in our code base. ACKs for top commit: paplorinc: ACK ec973dd19719541dbcd6f3a6facf6f5dd7cf439c - will happily reack if you decide to apply @ismaelsadeeq's suggestions ismaelsadeeq: Code review ACK ec973dd19719541dbcd6f3a6facf6f5dd7cf439c itornaza: trACK ec973dd19719541dbcd6f3a6facf6f5dd7cf439c theStack: Code-review ACK ec973dd19719541dbcd6f3a6facf6f5dd7cf439c Tree-SHA512: 34947e3eac39bd959807fa21b6045191fc80113bd650f6f08606e4bcd89aa17d6afd48dd034f6741ac4ff304b104fa8c1c1898e297467edcf262d5f97425da7b
2024-08-06Merge bitcoin/bitcoin#30577: miniscript: Use `ToIntegral` instead of ↵Ava Chow
`ParseInt64` 6714276d72244c2e2dbe9617f1341ba7fc06c0cc miniscript: Use `ToIntegral` instead of `ParseInt64` (brunoerg) Pull request description: Currently, miniscript code uses `ParseInt64` function for `after`, `older`, `multi` and `thresh` fragments. It means that a leading `+` or whitespace, among other things, are accepted into the fragments. However, these cases are not useful and cause Bitcoin Core to behave differently compared to other miniscript implementations (see https://github.com/brunoerg/bitcoinfuzz/issues/34). This PR fixes it. ACKs for top commit: achow101: ACK 6714276d72244c2e2dbe9617f1341ba7fc06c0cc tdb3: cr ACK 6714276d72244c2e2dbe9617f1341ba7fc06c0cc danielabrozzoni: tACK 6714276d72244c2e2dbe9617f1341ba7fc06c0cc darosior: utACK 6714276d72244c2e2dbe9617f1341ba7fc06c0cc Tree-SHA512: d9eeb93f380f346d636513eeaf26865285e7b0907b8ed258fe1e02153a9eb69d484c82180eb1c78b0ed77ad5f0e5b244be6672c2f890b1d9fddc9e844bee6dde
2024-08-06Merge bitcoin/bitcoin#30212: rename TransactionError:ALREADY_IN_CHAINRyan Ofsky
e9de0a76b99fd4708295e1178f6c079db92e7f36 doc: release note for 30212 (willcl-ark) 87b188052528e97729a85d9701864eaff1ea5ec6 rpc: clarify ALREADY_IN_CHAIN rpc errors (willcl-ark) Pull request description: Closes: #19363 Renaming this error improves clarity around the returned error both internally and externally when a transactions' outputs are already found in the utxo set (`TransactionError::ALREADY_IN_CHAIN -> TransactionError::ALREADY_IN_UTXO_SET`) ACKs for top commit: tdb3: ACK e9de0a76b99fd4708295e1178f6c079db92e7f36 ismaelsadeeq: ACK e9de0a76b99fd4708295e1178f6c079db92e7f36 ryanofsky: Code review ACK e9de0a76b99fd4708295e1178f6c079db92e7f36. Tree-SHA512: 7d2617200909790340951fe56a241448f9ce511900777cb2a712e8b9c0778a27d1f912b460f82335844224f1abb4322bc898ca076440959edade55c082a09237
2024-08-06fuzz: replace hardcoded numbers for bech32 limitsjosibake
Use bech32::CharLimit::BECH32 and bech32::CHECKSUM_SIZE instead of hardcoded values. This is a follow-up fix for #34007 (where this file was missed)
2024-08-05coins: pass linked list of flagged entries to BatchWriteAndrew Toth
BatchWrite now iterates through the linked list of flagged entries instead of the entire coinsCache map. Co-Authored-By: l0rinc <pap.lorinc@gmail.com>
2024-08-05test: add cache entry linked list testsAndrew Toth
Co-Authored-By: l0rinc <pap.lorinc@gmail.com>
2024-08-06testnet: Introduce Testnet4Fabian Jahr
2024-08-06refactor: use c++20 std::views::reverse instead of reverse_iterator.hstickies-v
Use std::ranges::views::reverse instead of the implementation in reverse_iterator.h, and remove it as it is no longer used.
2024-08-05Merge bitcoin/bitcoin#28052: blockstorage: XOR blocksdir *.dat filesAva Chow
fa895c72832f9555b52d5bb1dba1093f73de3136 mingw: Document mode wbx workaround (MarcoFalke) fa359255fe6b4de5f26784bfc147dbfb58bef116 Add -blocksxor boolean option (MarcoFalke) fa7f7ac040a9467c307b20e77dc47c87d7377ded Return XOR AutoFile from BlockManager::Open*File() (MarcoFalke) Pull request description: Currently the *.dat files in the blocksdir store the data received from remote peers as-is. This may be problematic when a program other than Bitcoin Core tries to interpret them by accident. For example, an anti-virus program or other program may scan them and move them into quarantine, or delete them, or corrupt them. This may cause Bitcoin Core to fail a reorg, or fail to reply to block requests (via P2P, RPC, REST, ...). Fix this, similar to https://github.com/bitcoin/bitcoin/pull/6650, by rolling a random XOR pattern over the dat files when writing or reading them. Obviously this can only protect against programs that accidentally and unintentionally are trying to mess with the dat files. Any program that intentionally wants to mess with the dat files can still trivially do so. The XOR pattern is only applied when the blocksdir is freshly created, and there is an option to disable it (on creation), so that people can disable it, if needed. ACKs for top commit: achow101: ACK fa895c72832f9555b52d5bb1dba1093f73de3136 TheCharlatan: Re-ACK fa895c72832f9555b52d5bb1dba1093f73de3136 hodlinator: ACK fa895c72832f9555b52d5bb1dba1093f73de3136 Tree-SHA512: c92a6a717da83bc33a9b8671a779eeefde2c63b192362ba1d71e6535ee31d08e2802b74acc908345197de9daac6930e4771595ee25b09acd5a67f7ea34854720
2024-08-05Merge bitcoin/bitcoin#30082: test: expand LimitOrphan and EraseForPeer coverageAva Chow
172c1ad026cc38c6f52679e74c14579ecc77c48e test: expand LimitOrphan and EraseForPeer coverage (Greg Sanders) 28dbe218feef51cbc28051273334dd73ba4500c0 refactor: move orphanage constants to header file (Greg Sanders) Pull request description: Inspired by refactorings in #30000 as the coverage appeared a bit sparse. Added some minimal border value testing, timeouts, and tightened existing assertions. ACKs for top commit: achow101: ACK 172c1ad026cc38c6f52679e74c14579ecc77c48e rkrux: reACK [172c1ad](https://github.com/bitcoin/bitcoin/pull/30082/commits/172c1ad026cc38c6f52679e74c14579ecc77c48e) glozow: reACK 172c1ad026cc38c6f52679e74c14579ecc77c48e Tree-SHA512: e8fa9b1de6a8617612bbe9b132c9c0c9b5a651ec94fd8c91042a34a8c91c5f9fa7ec4175b47e2b97d1320d452c23775be671a9970613533e68e81937539a7d70
2024-08-05rpc: clarify ALREADY_IN_CHAIN rpc errorswillcl-ark
When using `sendrawtransaction` the ALREADY_IN_CHAIN error help string may be confusing. Rename TransactionError::ALREADY_IN_CHAIN to TransactionError::ALREADY_IN_UTXO_SET and update the rpc help string. Remove backwards compatibility alias as no longer required.
2024-08-05scripted-diff: Replace uint256S("str") -> uint256{"str"}Hodlinator
-BEGIN VERIFY SCRIPT- sed -i --regexp-extended -e 's/\buint256S\("(0x)?([^"]{64})"\)/uint256{"\2"}/g' $(git grep -l uint256S) -END VERIFY SCRIPT-
2024-08-05refactor: Hand-replace some uint256S -> uint256Hodlinator
chainparams.cpp - workaround for MSVC bug triggering C7595 - Calling consteval constructors in initializer lists fails, but works on GCC (13.2.0) & Clang (17.0.6).
2024-08-05refactor: Add consteval uint256(hex_str)Hodlinator
Complements uint256::FromHex() nicely in that it naturally does all error checking at compile time and so doesn't need to return an std::optional. Will be used in the following 2 commits to replace many calls to uint256S(). uint256S() calls taking C-string literals are littered throughout the codebase and executed at runtime to perform parsing unless a given optimizer was surprisingly efficient. While this may not be a hot spot, it's better hygiene in C++20 to store the parsed data blob directly in the binary, without any parsing at runtime.
2024-08-05miniscript: Use `ToIntegral` instead of `ParseInt64`brunoerg
2024-08-05Merge bitcoin/bitcoin#29656: chainparams: Change nChainTx type to uint64_tglozow
bf0efb4fc72d3c49a2c498c944e55466dfa046dc scripted-diff: Modernize naming of nChainTx and nTxCount (Fabian Jahr) 72e5d1be1f4491565249d43e836ee42cfd858866 test: Add basic check for nChainTx type (Fabian Jahr) dc2938e9799d79696d1db2438ef33d90542d984b chainparams: Change nChainTx to uint64_t (Fabian Jahr) Pull request description: This picks up the work from #29331 and closes #29258. This simply changes the type and addresses the comments from #29331 by changing the type in all relevant places and removing unnecessary casts. This also adds an extremely simple unit test. Additionally this modernizes the name of `nChainTx` which helps reviewers check all use of the symbol and can make silent merge conflicts. ACKs for top commit: maflcko: only rebase in scripted-diff, re-ACK bf0efb4fc72d3c49a2c498c944e55466dfa046dc 🔈 glozow: reACK bf0efb4fc72 via range-diff Tree-SHA512: ee4020926d0800236fe655d0c7b127215ab36b553b04d5f91494f4b7fac6e1cfe7ee298b07c0983db5a3f4786932acaa54f5fd2ccd45f2fcdcfa13427358dc3b
2024-08-05Merge bitcoin/bitcoin#30285: cluster mempool: merging & postprocessing of ↵glozow
linearizations bbcee5a0d67db46526ba29a1a4a7c590d303de03 clusterlin: improve rechunking in LinearizationChunking (optimization) (Pieter Wuille) 04d7a04ea426dd0a69b61e3b887867b0277d84d1 clusterlin: add MergeLinearizations function + fuzz test + benchmark (Pieter Wuille) 4f8958d7563ae2d0d359ec1e6885f8cb5e40a5e0 clusterlin: add PostLinearize + benchmarks + fuzz tests (Pieter Wuille) 0e2812d2938b933debffba5b873637fa1d348b81 clusterlin: add algorithms for connectedness/connected components (Pieter Wuille) 0e52728a2d6ccafcfecfefbb5a0432a9881d8e0d clusterlin: rename Intersect -> IntersectPrefixes (Pieter Wuille) Pull request description: Part of cluster mempool: #30289 Depends on #30126, and was split off from it. #28676 depends on this. This adds the algorithms for merging & postprocessing linearizations. The `PostLinearize(depgraph, linearization)` function performs an in-place improvement of `linearization`, using two iterations of the [Linearization post-processing](https://delvingbitcoin.org/t/linearization-post-processing-o-n-2-fancy-chunking/201/8) algorithm. The first running from back to front, the second from front to back. The `MergeLinearizations(depgraph, linearization1, linearization2)` function computes a new linearization for the provided cluster, given two existing linearizations for that cluster, which is at least as good as both inputs. The algorithm is described at a high level in [merging incomparable linearizations](https://delvingbitcoin.org/t/merging-incomparable-linearizations/209). For background and references, see [Introduction to cluster linearization](https://delvingbitcoin.org/t/introduction-to-cluster-linearization/1032). ACKs for top commit: sdaftuar: ACK bbcee5a0d67db46526ba29a1a4a7c590d303de03 glozow: code review ACK bbcee5a0d67 instagibbs: ACK https://github.com/bitcoin/bitcoin/pull/30285/commits/bbcee5a0d67db46526ba29a1a4a7c590d303de03 Tree-SHA512: d2b5a3f132d1ef22ddf9c56421ab8b397efe45b3c4c705548dda56f5b39fe4b8f57a0d2a4c65b338462d80bb5b9b84a9a39efa1b4f390420a8005ce31817774e
2024-08-04Merge bitcoin/bitcoin#30526: doc: Correct uint256 hex string endiannessRyan Ofsky
73e3fa10b4dd63b7767d6b6f270df66971067341 doc + test: Correct uint256 hex string endianness (Hodlinator) Pull request description: This PR is a follow-up to #30436. Only changes test-code and modifies/adds comments. Byte order of hex string representation was wrongfully documented as little-endian, but are in fact closer to "big-endian" (endianness is a memory-order concept rather than a numeric concept). `[arith_]uint256` both store their data in arrays with little-endian byte order (`arith_uint256` has host byte order within each `uint32_t` element). **uint256_tests.cpp** - Avoid using variable from the left side of the condition in the right side. Credits to @maflcko: https://github.com/bitcoin/bitcoin/pull/30436#discussion_r1688273553 **setup_common.cpp** - Skip needless ArithToUint256-conversion. Credits to @stickies-v: https://github.com/bitcoin/bitcoin/pull/30436#discussion_r1688621638 --- <details> <summary> ## Logical reasoning for endianness </summary> 1. Comparing an `arith_uint256` (`base_uint<256>`) to a `uint64_t` compares the beginning of the array, and verifies the remaining elements are zero. ```C++ template <unsigned int BITS> bool base_uint<BITS>::EqualTo(uint64_t b) const { for (int i = WIDTH - 1; i >= 2; i--) { if (pn[i]) return false; } if (pn[1] != (b >> 32)) return false; if (pn[0] != (b & 0xfffffffful)) return false; return true; } ``` ...that is consistent with little endian ordering of the array. 2. They have the same endianness (but `arith_*` has host-ordering of each `uint32_t` element): ```C++ arith_uint256 UintToArith256(const uint256 &a) { arith_uint256 b; for(int x=0; x<b.WIDTH; ++x) b.pn[x] = ReadLE32(a.begin() + x*4); return b; } ``` ### String conversions The reversal of order which happens when converting hex-strings <=> uint256 means strings are actually closer to big-endian, see the end of `base_blob<BITS>::SetHexDeprecated`: ```C++ unsigned char* p1 = m_data.data(); unsigned char* pend = p1 + WIDTH; while (digits > 0 && p1 < pend) { *p1 = ::HexDigit(trimmed[--digits]); if (digits > 0) { *p1 |= ((unsigned char)::HexDigit(trimmed[--digits]) << 4); p1++; } } ``` Same reversal here: ```C++ template <unsigned int BITS> std::string base_blob<BITS>::GetHex() const { uint8_t m_data_rev[WIDTH]; for (int i = 0; i < WIDTH; ++i) { m_data_rev[i] = m_data[WIDTH - 1 - i]; } return HexStr(m_data_rev); } ``` It now makes sense to me that `SetHexDeprecated`, upon receiving a shorter hex string that requires zero-padding, would pad as if the missing hex chars where towards the end of the little-endian byte array, as they are the most significant bytes. "Big-endian" string representation is also consistent with the case where `SetHexDeprecated` receives too many hex digits and discards the leftmost ones, as a form of integer narrowing takes place. ### How I got it wrong in #30436 Previously I used the less than (`<`) comparison to prove endianness, but for `uint256` it uses `memcmp` and thereby gives priority to the *lower* bytes at the beginning of the array. ```C++ constexpr int Compare(const base_blob& other) const { return std::memcmp(m_data.data(), other.m_data.data(), WIDTH); } ``` `arith_uint256` is different in that it begins by comparing the bytes from the end, as it is using little endian representation, where the bytes toward the end are more significant. ```C++ template <unsigned int BITS> int base_uint<BITS>::CompareTo(const base_uint<BITS>& b) const { for (int i = WIDTH - 1; i >= 0; i--) { if (pn[i] < b.pn[i]) return -1; if (pn[i] > b.pn[i]) return 1; } return 0; } ``` (The commit documents that `base_blob::Compare()` is doing lexicographic ordering unlike the `arith_*`-variant which is doing numeric ordering). </details> ACKs for top commit: paplorinc: ACK 73e3fa10b4dd63b7767d6b6f270df66971067341 ryanofsky: Code review ACK 73e3fa10b4dd63b7767d6b6f270df66971067341 Tree-SHA512: 121630c37ab01aa7f7097f10322ab37da3cbc0696a6bbdbf2bbd6db180dc5938c7ed91003aaa2df7cf4a4106f973f5118ba541b5e077cf3588aa641bbd528f4e
2024-08-04scripted-diff: Modernize naming of nChainTx and nTxCountFabian Jahr
-BEGIN VERIFY SCRIPT- sed -i 's/nChainTx/m_chain_tx_count/g' $(git grep -l 'nChainTx' ./src) sed -i 's/nTxCount/tx_count/g' $(git grep -l 'nTxCount' ./src) -END VERIFY SCRIPT-
2024-08-04test: Add basic check for nChainTx typeFabian Jahr
2024-08-04tests: add tests for KeyPairjosibake
Reuse existing BIP340 tests, as there should be no behavior change between the two
2024-08-03doc + test: Correct uint256 hex string endiannessHodlinator
Follow-up to #30436. uint256 string representation was wrongfully documented as little-endian due to them being reversed by GetHex() etc, and base_blob::Compare() giving most significance to the beginning of the internal array. They are closer to "big-endian", but this commit tries to be even more precise than that. uint256_tests.cpp - Avoid using variable from the left side of the condition in the right side. setup_common.cpp - Skip needless ArithToUint256-conversion.
2024-08-03tests: add key tweak smoke testjosibake
Sanity check that using CKey/CPubKey directly vs using secp256k1_keypair objects returns the same results for BIP341 key tweaking. Co-authored-by: l0rinc <pap.lorinc@gmail.com>
2024-08-02Merge bitcoin/bitcoin#30352: policy: Add PayToAnchor(P2A), `OP_1 <0x4e73>` ↵glozow
as a standard output script for spending 75648cea5a9032b3d388cbebacb94d908e08924e test: add P2A ProduceSignature coverage (Greg Sanders) 7998ce6b20fba62c022228355907b612ba6692e1 Add release note for P2A output feature (Greg Sanders) 71c9b02a04742eeecab14aae4697b1a3eb51ff7f test: add P2A coverage for decodescript (Greg Sanders) 1349e9ec1558484f2912a2444c410170fcec8745 test: Add anchor mempool acceptance test (Greg Sanders) 9d892099378b2ad5f52220403bdeae43c61d6955 policy: stop 3rd party wtxid malleability of anchor spend (Greg Sanders) b60aaf8b239978947d2b0e3f56e7d8a4092d7570 policy: make anchor spend standard (Greg Sanders) 455fca86cfada1823aa28615b5683f9dc73dbb9a policy: Add OP_1 <0x4e73> as a standard output type (Greg Sanders) Pull request description: This is a sub-feature taken out of the original proposal for ephemeral anchors #30239 This PR makes *spending* of `OP_1 <0x4e73>` (i.e. `bc1pfeessrawgf`) standard. Creation of this output type is already standard. Any future witness output types are considered relay-standard to create, but not to spend. This preserves upgrade hooks, such as a completely new output type for a softfork such as BIP341. It also gives us a bit of room to use a new output type for policy uses. This particular sized witness program has no other known use-cases (https://bitcoin.stackexchange.com/a/110664/17078), s it affords insufficient cryptographic security for a secure commitment to data, such as a script or a public key. This makes this type of output "keyless", or unauthenticated. As a witness program, the `scriptSig` of the input MUST be blank, by BIP141. This helps ensure txid-stability of the spending transaction, which may be required for smart contracting wallets. If we do not use segwit, a miner can simply insert an `OP_NOP` in the `scriptSig` without effecting the result of program execution. An additional relay restriction is to disallow non-empty witness data, which an adversary may use to penalize the "honest" transactor when RBF'ing the transaction due to the incremental fee requirement of RBF rules. The intended use-case for this output type is to "anchor" the transaction with a spending child to bring exogenous CPFP fees into the transaction package, encouraging the inclusion of the package in a block. The minimal size of creation and spending of this output makes it an attractive contrast to outputs like `p2sh(OP_TRUE)` and `p2wsh(OP_TRUE)` which are significantly larger in vbyte terms. Combined with TRUC transactions which limits the size of child transactions significantly, this is an attractive option for presigned transactions that need to be fee-bumped after the fact. ACKs for top commit: sdaftuar: utACK 75648cea5a9032b3d388cbebacb94d908e08924e theStack: re-ACK 75648cea5a9032b3d388cbebacb94d908e08924e ismaelsadeeq: re-ACK 75648cea5a9032b3d388cbebacb94d908e08924e via [diff](https://github.com/bitcoin/bitcoin/compare/e7ce6dc070c0319cbb868d41cadd836b2e6ca9db..75648cea5a9032b3d388cbebacb94d908e08924e) glozow: ACK 75648cea5a9032b3d388cbebacb94d908e08924e tdb3: ACK 75648cea5a9032b3d388cbebacb94d908e08924e Tree-SHA512: d529de23d20857e6cdb40fa611d0446b49989eaafed06c28280e8fd1897f1ed8d89a4eabbec1bbf8df3d319910066c3dbbba5a70a87ff0b2967d5205db32ad1e
2024-08-02refactor: require self and sentinel parameters for AddFlagsAndrew Toth
No behavior change. Prepares for adding the CoinsCachePairs to a linked list when flagged.
2024-08-01refactor: move flags to private uint8_t and rename to m_flagsAndrew Toth
No behavior change. This prepares to add CCoinsCacheEntrys to a doubly linked list when a flag is added.
2024-08-01refactor: encapsulate flags setting with AddFlags and ClearFlagsAndrew Toth
No behavior change. This prepares moving the cache entry flags field to private access.
2024-08-01refactor: encapsulate flags get access for all other checksAndrew Toth
No behavior change. This prepares moving the cache entry flags field to private access.
2024-08-01refactor: encapsulate flags access for dirty and fresh checksAndrew Toth
No behavior change. This prepares moving the cache entry flags field to private access. Co-Authored-By: l0rinc <pap.lorinc@gmail.com>
2024-08-01clusterlin: add MergeLinearizations function + fuzz test + benchmarkPieter Wuille
2024-08-01clusterlin: add PostLinearize + benchmarks + fuzz testsPieter Wuille
2024-08-01clusterlin: add algorithms for connectedness/connected componentsPieter Wuille
Add utility functions to DepGraph for finding connected components.
2024-08-01clusterlin: rename Intersect -> IntersectPrefixesPieter Wuille
This makes it clearer what the function does.
2024-08-01Merge bitcoin/bitcoin#30532: refactor: remove deprecated TxidFromString() in ↵glozow
favour of transaction_identifier::FromHex() f553e6d86fe114e90585ea6d4b8e345a209cae5d refactor: remove TxidFromString (stickies-v) 285ab50ace4c04209f331ccaf121152b977cc490 test: replace WtxidFromString with Wtxid::FromHex (stickies-v) 9a0b2a69c4d836d7382f6fe7703de40288fc7421 fuzz: increase FromHex() coverage (stickies-v) 526a87ba6b4f20592ad3c090b82e83ecff2107fc test: add uint256::FromHex unittest coverage (stickies-v) Pull request description: Since https://github.com/bitcoin/bitcoin/pull/30482/commits/fab6ddbee64e50d5e2f499aebca35b5911896ec4, `TxidFromString()` has been deprecated because it is less robust than the `transaction_identifier::FromHex()` introduced in [the same PR](https://github.com/bitcoin/bitcoin/pull/30482). Specifically, it tries to recover from length-mismatches, recover from untrimmed whitespace, 0x-prefix and garbage at the end, instead of simply requiring exactly 64 hex-only characters. In this PR, `TxidFromString` is removed completely to clean up the code and prevent further unsafe usage. Unit and fuzz test coverage on `uint256::FromHex()` and functions that wrap it is increased. Note: `TxidFromSring` allowed users to prefix strings with "0x", this is no longer allowed for `transaction_identifier::FromHex()`, so a helper function for input validation may prove helpful in the future _(this overlaps with the `uint256::uint256S()` vs `uint256::FromHex()` future cleanup)_. It is not relevant to this PR, though, besides the fact that this unused (except for in tests) functionality is removed. The only users of `TxidFromString` are: - `test`, where it is straightforward to drop in the new `FromHex()` methods without much further concern - `qt` coincontrol. There is no need for input validation here, but txids are not guaranteed to be 64 characters. This is already handled by the existing code, so again, using `FromHex()` here seems quite straightforward. Addresses @maflcko's suggestion: https://github.com/bitcoin/bitcoin/pull/30482#discussion_r1691826934 Also removes `WtxidFromString()`, which is a test-only helper function. ### Testing GUI changes To test the GUI coincontrol affected lines, `regtest` is probably the easiest way to quickly get some test coins, you can use e.g. ``` alias cli="./src/bitcoin-cli -regtest" cli createwallet "coincontrol" # generate 10 spendable outputs on 1 address cli generatetoaddress 10 $(cli -rpcwallet=coincontrol getnewaddress) # generate 10 spendable outputs on another address cli generatetoaddress 10 $(cli -rpcwallet=coincontrol getnewaddress) # make previous outputs spendable cli generatetoaddress 100 $(cli -rpcwallet=coincontrol getnewaddress) ``` ACKs for top commit: maflcko: re-ACK f553e6d86fe114e90585ea6d4b8e345a209cae5d 🔻 hodlinator: ACK f553e6d86fe114e90585ea6d4b8e345a209cae5d paplorinc: ACK f553e6d86fe114e90585ea6d4b8e345a209cae5d TheCharlatan: Nice, ACK f553e6d86fe114e90585ea6d4b8e345a209cae5d Tree-SHA512: c1c7e6ea4cbf05cf660ba178ffc4f35f0328f7aa6ad81872e2462fb91a6a22e4681ff64b3d0202a5a9abcb650c939561585cd309164a69ab6081c0765ee271ef
2024-07-31refactor: remove TxidFromStringstickies-v
TxidFromString was deprecated due to missing 64-character length-check and hex-check, replace it with the more robust Txid::FromHex.
generated by cgit v1.2.3 (git 2.26.2) at 2024-08-24 09:54:41 +0000