aboutsummaryrefslogtreecommitdiff
path: root/src/validation.cpp
AgeCommit message (Collapse)Author
2021-07-03prefer to use txindex if available for GetTransactionJameson Lopp
Fixes #22382
2021-06-29Merge bitcoin/bitcoin#21789: refactor: Remove ::Params() global from CChainStatefanquake
fa0d9211ef87a682573aaae932c0c440acbcb8a8 refactor: Remove chainparams arg from CChainState member functions (MarcoFalke) fa389471251f043ec25e7b01e59b37d3b921ce54 refactor: Remove ::Params() global from inside CChainState member functions (MarcoFalke) Pull request description: The `::Params()` global is verbose and confusing. Also it makes tests a bit harder to write because they'd have to mock a global. Fix all issues by simply using a member variable that points to the right params. (Can be reviewed with `--word-diff-regex=.`) ACKs for top commit: jnewbery: ACK fa0d9211ef87a682573aaae932c0c440acbcb8a8 kiminuo: utACK fa0d9211 theStack: ACK fa0d9211ef87a682573aaae932c0c440acbcb8a8 🍉 Tree-SHA512: 44676b19c9ed471ccb536331d3029bad192d7d50f394fd7b8527ec431452aeec8c4494164b9cf8e16e0123c4463b16be864366c6b599370032c17262625a0356
2021-06-28Merge bitcoin/bitcoin#22146: Reject invalid coin height and output index ↵MarcoFalke
when loading assumeutxo fa9ebedec3f982bb5bb459ea33d74c94d9b5cec4 Reject invalid coin height and output index when loading assumeutxo (MarcoFalke) Pull request description: It should be impossible to have a coin at a height higher than the height of the snapshot block, so reject those early to avoid integer wraparounds and hash collisions later on. Same for the outpoint index. Both issues were found by fuzzing: * The height issue by OSS-Fuzz: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=34793 * The outpoint issue by my fuzz server: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=34793#c2 ACKs for top commit: practicalswift: cr ACK fa9ebedec3f982bb5bb459ea33d74c94d9b5cec4: patch looks correct jamesob: crACK https://github.com/bitcoin/bitcoin/pull/22146/commits/fa9ebedec3f982bb5bb459ea33d74c94d9b5cec4 theStack: Code review ACK fa9ebedec3f982bb5bb459ea33d74c94d9b5cec4 benthecarman: crACK fa9ebedec3f982bb5bb459ea33d74c94d9b5cec4 Tree-SHA512: dae7caee4b3862b23ebdf2acb7edec4baf75b0dbf1409b370b1a73aa6b632b317ebfac596dcbaf4edfb1301b513f45465ea75328962460f35e2af0d7e547c9ac
2021-06-13refactor: Remove chainparams arg from CChainState member functionsMarcoFalke
Passing this is confusing and redundant with the m_params member.
2021-06-13refactor: Remove ::Params() global from inside CChainState member functionsMarcoFalke
It is confusing and verbose to repeatedly access the global when a member variable can simply refer to it.
2021-06-13Merge bitcoin/bitcoin#22102: Remove `Warning:` from warning message printed ↵MarcoFalke
for unknown new rules 6d7e46ce23217da53ff52f535879c393c02fa2b2 Remove `Warning:` (Prayank) Pull request description: Reason: I noticed that `Warning` is printed 2 times in `-getinfo` while reviewing https://github.com/bitcoin/bitcoin/pull/21832#issuecomment-851004943 Same string is used for GUI, log and stderr. If we need to add `Warning:` in GUI or other place we can always prepend to this string. CLI: ``` Warnings: Unknown new rules activated (versionbit 28) ``` GUI: ![image](https://user-images.githubusercontent.com/13405205/120110401-e36ab180-c18a-11eb-8031-4d52287dc263.png) ACKs for top commit: MarcoFalke: review ACK 6d7e46ce23217da53ff52f535879c393c02fa2b2 Tree-SHA512: 25760cf6d850f6c2216d651fa46574d2d21a9d58f4577ecb58f9566ea8cd07bfcd6679bb8a1f53575e441b02d295306f492c0c99a48c909e60e5d977ec1861f6
2021-06-10validation: Farewell, global Chainstate!Carl Dong
2021-06-10scripted-diff: tree-wide: Remove all review-only assertionsCarl Dong
-BEGIN VERIFY SCRIPT- find_regex='((assert|CHECK_NONFATAL)\(std::addressof|TODO: REVIEW-ONLY)' \ && git grep -l -E "$find_regex" -- . \ | xargs sed -i -E "/${find_regex}/d" -END VERIFY SCRIPT-
2021-06-10Merge bitcoin/bitcoin#22084: package testmempoolaccept followupsfanquake
ee862d6efb4c3c01e55f0d5d7a82cce75323cf40 MOVEONLY: context-free package policies (glozow) 5cac95cd15da04b83afa1d31a43be9f5b30a1827 disallow_mempool_conflicts -> allow_bip125_replacement and check earlier (glozow) e8ecc621be6afd3252c0f8147e42c3b4918f7f46 [refactor] comment/naming improvements (glozow) 7d91442461776e2ef240d7885f768b624de341a7 [rpc] reserve space in txns (glozow) 6c5f19d9c4d267c54f4dbc4f9d65370ff1e0625b [package] static_assert max package size >= max tx size (glozow) Pull request description: various followups from #20833 ACKs for top commit: jnewbery: utACK ee862d6efb4c3c01e55f0d5d7a82cce75323cf40 ariard: Code Review ACK ee862d6 Tree-SHA512: 96ecb41f7bbced84d4253070f5274b7267607bfe4033e2bb0d2f55ec778cc41e811130b6321131e0418b5835894e510a4be8a0f822bc9d68d9224418359ac837
2021-06-09Reject invalid coin height and output index when loading assumeutxoMarcoFalke
2021-06-08Merge bitcoin/bitcoin#21946: Document and test lack of inherited signaling ↵MarcoFalke
in RBF policy 2eb0eeda39cab997f9a5f817f7c12e7fffeaf35a validation: document lack of inherited signaling in RBF policy (Antoine Riard) 906b6d9da6a6b2e6a5f1d9046b3b9c2c7e490c99 test: Extend feature_rbf.py with no inherited signaling (Antoine Riard) Pull request description: Contrary to BIP125 or other full-node implementation (e.g btcd), Bitcoin Core's mempool policy doesn't implement inherited signaling. This PR documents our mempool behavior on this and add a test demonstrating the case. ACKs for top commit: jonatack: ACK 2eb0eeda39cab997f9a5f817f7c12e7fffeaf35a benthecarman: ACK 2eb0eeda39cab997f9a5f817f7c12e7fffeaf35a Tree-SHA512: d41453d3b49bae3c1eb532a968f43bc047084913bd285929d4d9cba142777ff2be38163d912e28dfc635f4ecf446de68effad799c6e71be52f81e83410c712fb
2021-06-07Remove `Warning:`Prayank
+ Remove `Warning:` from warning message printed for unknown new rules + Change warning message in test Author: Prayank <prayank@tutanota.de>
2021-06-04Merge bitcoin/bitcoin#22121: doc: Various validation doc fixupsfanquake
fa4245d88409091a3a6115a96a200b70be663725 doc: Various validation doc fixups (MarcoFalke) Pull request description: ACKs for top commit: michaelfolkson: Re-ACK fa4245d88409091a3a6115a96a200b70be663725 jnewbery: ACK fa4245d884 Tree-SHA512: fa1086b09941247a4ffcbc1d7d27dc77a17a3ae093a5146dbb703db9ff4ba5d73ea77bd5b7747af79ea8a7dfe2c4c56a7e19ac5aac3417090e9ae127836022ae
2021-06-03[doc] explain why CheckBlock() is called before AcceptBlock()Sjors Provoost
Co-authored-by: Suhas Daftuar <sdaftuar@gmail.com>
2021-06-03doc: Various validation doc fixupsMarcoFalke
* Rename RewindBlockIndex -> NeedsRedownload (follow-up to commit d831e711cab83c70bf2ded62fe33f484844e73dd) * Fix typos * Inline comments about faking chain data to avoid duplicating them
2021-06-02MOVEONLY: context-free package policiesglozow
Co-authored-by: ariard <antoine.riard@gmail.com>
2021-06-02disallow_mempool_conflicts -> allow_bip125_replacement and check earlierglozow
2021-06-02[refactor] comment/naming improvementsglozow
2021-05-31validation: change ProcessNewBlock() to take a CBlock referencefanquake
Update ProcessNewBlock arguments to newer style.
2021-05-27Merge bitcoin/bitcoin#20833: rpc/validation: enable packages through ↵W. J. van der Laan
testmempoolaccept 13650fe2e527bf0cf5d977bf5f3f1563b853ecdc [policy] detect unsorted packages (glozow) 9ef643e21b44f99f4bce54077788d0ad4d81f7cd [doc] add release note for package testmempoolaccept (glozow) c4259f4b7ee23ef6e0ec82c5d5b9dfa9cadd5bed [test] functional test for packages in RPCs (glozow) 9ede34a6f20378e86c5289ebd20dd394a5915123 [rpc] allow multiple txns in testmempoolaccept (glozow) ae8e6df709ff3d52b8e9918e09cacb64f83ae379 [policy] limit package sizes (glozow) c9e1a26d1f17c8b98632b7796ffa8f8788b5a83c [fuzz] add ProcessNewPackage call in tx_pool fuzzer (glozow) 363e3d916cc036488783bb4bdcfdd3665aecf711 [test] unit tests for ProcessNewPackage (glozow) cd9a11ac96c01e200d0086b2f011f4a614f5a705 [test] make submit optional in CreateValidMempoolTransaction (glozow) 2ef187941db439c5b3e529f08b6ab153ff061fc5 [validation] package validation for test accepts (glozow) 578148ded62828a9820398165c41670f4dbb523d [validation] explicit Success/Failure ctors for MempoolAcceptResult (glozow) b88d77aec5e7bef5305a668d15031351c0548b4d [policy] Define packages (glozow) 249f43f3cc52b0ffdf2c47aad95ba9d195f6a45e [refactor] add option to disable RBF (glozow) 897e348f5987eadd8559981a973c045c471b3ad8 [coins/mempool] extend CCoinsViewMemPool to track temporary coins (glozow) 42cf8b25df07c45562b7210e0e15c3fd5edb2c11 [validation] make CheckSequenceLocks context-free (glozow) Pull request description: This PR enables validation dry-runs of packages through the `testmempoolaccept` RPC. The expectation is that the results returned from `testmempoolaccept` are what you'd get from test-then-submitting each transaction individually, in that order (this means the package is expected to be sorted in topological order, for now at least). The validation is also atomic: in the case of failure, it immediately halts and may return "unfinished" `MempoolAcceptResult`s for transactions that weren't fully validated. The API for 1 transaction stays the same. **Motivation:** - This allows you to test validity for transaction chains (e.g. with multiple spending paths and where you don't want to broadcast yet); closes #18480. - It's also a first step towards package validation in a minimally invasive way. - The RPC commit happens to close #21074 by clarifying the "allowed" key. There are a few added restrictions on the packages, mostly to simplify the logic for areas that aren't critical to main package use cases: - No package can have conflicts, i.e. none of them can spend the same inputs, even if it would be a valid BIP125 replacement. - The package cannot conflict with the mempool, i.e. RBF is disabled. - The total count of the package cannot exceed 25 (the default descendant count limit), and total size cannot exceed 101KvB (the default descendant size limit). If you're looking for review comments and github isn't loading them, I have a gist compiling some topics of discussion [here](https://gist.github.com/glozow/c3acaf161c95bba491fce31585b2aaf7) ACKs for top commit: laanwj: Code review re-ACK 13650fe2e527bf0cf5d977bf5f3f1563b853ecdc jnewbery: Code review ACK 13650fe2e527bf0cf5d977bf5f3f1563b853ecdc ariard: ACK 13650fe Tree-SHA512: 8c5cbfa91a6c714e1c8710bb281d5ff1c5af36741872a7c5df6b24874d6272b4a09f816cb8a4c7de33ef8e1c2a2c252c0df5105b7802f70bc6ff821ed7cc1a2f
2021-05-24[policy] detect unsorted packagesglozow
2021-05-24[policy] limit package sizesglozow
Maximum number of transactions allowed in a package is 25, equal to the default mempool descendant limit: if a package has more transactions than this, either it would fail default mempool descendant limit or the transactions don't all have a dependency relationship (but then they shouldn't be in a package together). Same rationale for 101KvB virtual size package limit. Note that these policies are only used in test accepts so far.
2021-05-24[validation] package validation for test acceptsglozow
Only allow test accepts for now. Use the CoinsViewTemporary to keep track of coins created by each transaction so that subsequent transactions can spend them. Uncache all coins since we only ever do test accepts (Note this is different from ATMP which doesn't uncache for valid test_accepts) to minimize impact on the coins cache. Require that the input txns have no conflicts and be ordered topologically. This commit isn't able to detect unsorted packages.
2021-05-24scripted-diff: Replace `GetDataDir()` calls with `gArgs.GetDataDirNet()` callsKiminuo
-BEGIN VERIFY SCRIPT- git ls-files -- 'src' ':(exclude)src/util/system.h' ':(exclude)src/util/system.cpp' | xargs sed -i 's/GetDataDir()/gArgs.GetDataDirNet()/g'; -END VERIFY SCRIPT-
2021-05-20[validation] explicit Success/Failure ctors for MempoolAcceptResultglozow
Makes code more clear and prevents accidentally calling the wrong ctor.
2021-05-20[refactor] add option to disable RBFglozow
This is a mere refactor for now. We will use this to disable RBFing in package validation.
2021-05-20[validation] make CheckSequenceLocks context-freeglozow
Allow CheckSequenceLocks to use heights and coins from any CoinsView and CBlockIndex provided. This means that CheckSequenceLocks() doesn't need to hold the mempool lock or cs_main. The caller is responsible for ensuring the CoinsView and CBlockIndex are consistent before passing them in. The typical usage is still to create a CCoinsViewMemPool from the mempool and grab the CBlockIndex from the chainstate tip.
2021-05-14validation: document lack of inherited signaling in RBF policyAntoine Riard
2021-05-11refactor: Avoid magic value of all-zeros in assumeutxo base_blockhashMarcoFalke
Just use std::optional
2021-05-11Fix assumeutxo crash due to invalid base_blockhashMarcoFalke
Can be reviewed with --color-moved=dimmed-zebra --color-moved-ws=ignore-all-space
2021-05-11refactor: Use type-safe assumeutxo hashMarcoFalke
This avoids accidentally mixing it up with other hashes (like block hashes).
2021-05-10Merge bitcoin/bitcoin#21836: scripted-diff: Replace three dots with ellipsis ↵W. J. van der Laan
in the UI strings d66f283ac07edce432b964f7f814631f5a5bc33b scripted-diff: Replace three dots with ellipsis in the UI strings (Hennadii Stepanov) Pull request description: This PR is split from #21463. The change was suggested on [Transifex.com](https://www.transifex.com/bitcoin/bitcoin/), and it does not touch `LogPrint` and `LogPrintf` calls. The only comment on #21463 [was](https://github.com/bitcoin/bitcoin/pull/21463/commits/9030e4b5a6de54e041c59e98d91adecbebf3611a#r597220100): > Mind that these messages also end up in the log. In principle the log is already UTF-8 (as are all strings and text in bitcoind). But, just noting, that it might make browsing the log a less pleasant experience on systems with misconfigured locale like some BSDs by default. ACKs for top commit: laanwj: ACK d66f283ac07edce432b964f7f814631f5a5bc33b Tree-SHA512: 5ab1cb3160f3f996f1ad7d7486662da3eb7f06a857f4a1874963ce10caed5b86b0ad6151b1b9ebeb2b8aa5f0c85efad3b768ea9cafe5db86f78f88912b756d1e
2021-05-05Merge bitcoin/bitcoin#21681: validation: fix ActivateSnapshot to use ↵MarcoFalke
hardcoded nChainTx 91d93aac4e3fe6fff5ef492ed152c4d8fa6f2672 validation: remove nchaintx from assumeutxo metadata (James O'Beirne) 931684b24a89aba884cb18c13fa67ccca339ee8c validation: fix ActivateSnapshot to use hardcoded nChainTx (James O'Beirne) Pull request description: This fixes an oversight from the move of nChainTx from the user-supplied snapshot metadata into the hardcoded assumeutxo chainparams. Since the nChainTx is now unused in the metadata, it should be removed in a future commit. See: https://github.com/bitcoin/bitcoin/pull/19806#discussion_r612165410 ACKs for top commit: Sjors: utACK 91d93aac4e3fe6fff5ef492ed152c4d8fa6f2672 ryanofsky: Code review ACK 91d93aac4e3fe6fff5ef492ed152c4d8fa6f2672. No change to previous commit, just new commit removing now unused utxo snapshot field and updating tests. Tree-SHA512: 445bdd738faf007451f40bbcf360dd1fb4675e17a4c96546e6818c12e33dd336dadd95cf8d4b5f8df1d6ccfbc4bf5496864bb5528e416cea894857b6b732140c
2021-05-05Merge bitcoin/bitcoin#21727: refactor: Move more stuff to blockstorageW. J. van der Laan
fa09a9eac8d8ab65ce4064c35a9f21349a644982 style: Add { } to multi-line if (MarcoFalke) fadafab83379ff10d86ada179c6f9641d19464fe move-only: Move functions to blockstorage (MarcoFalke) fa7e64d58615fffea91cd64dc4a2790221ceff0a move-only: Move constants to blockstorage (MarcoFalke) fa247a327fc7c7cea6bc8f93637b8babd3015ffa refactor: Move block storage globals to blockstorage (MarcoFalke) fa81c30c6f1adac79517c958090db174eb6aeda2 refactor: Move pruning/reindex/importing globals to blockstorage (MarcoFalke) Pull request description: See #21575 ACKs for top commit: Sjors: ACK fa09a9eac8d8ab65ce4064c35a9f21349a644982 kiminuo: ACK fa09a9e laanwj: Code review ACK fa09a9eac8d8ab65ce4064c35a9f21349a644982 promag: Code review ACK fa09a9eac8d8ab65ce4064c35a9f21349a644982. Since last review Tree-SHA512: 2eb6962ff44da6b77f3058fc02ec66ab742e25ae8dcc8ec62b062896571910d43ca7c4bb16fb3ccb5e5245195b8dec6384b6c8d442fa97ca28d93bdff347d677
2021-05-02scripted-diff: Replace three dots with ellipsis in the UI stringsHennadii Stepanov
-BEGIN VERIFY SCRIPT- sed -i -E -e 's/\.\.\."\)(\.|,|\)| )/…"\)\1/' -- $(git ls-files -- 'src' ':(exclude)src/qt/bitcoinstrings.cpp') sed -i -e 's/\.\.\.\\"/…\\"/' src/qt/sendcoinsdialog.cpp sed -i -e 's|\.\.\.</string>|…</string>|' src/qt/forms/*.ui sed -i -e 's|\.\.\.)</string>|…)</string>|' src/qt/forms/sendcoinsdialog.ui -END VERIFY SCRIPT-
2021-04-30Merge bitcoin/bitcoin#19521: Coinstats IndexW. J. van der Laan
5f96d7d22d8e05876c6fc014e70488699950fe38 rpc: gettxoutsetinfo rejects hash_serialized_2 for specific height (Fabian Jahr) 23fe50436be641d7417152adc683192649ba206a test: Add test for coinstatsindex behavior in reorgs (Fabian Jahr) 90c966b0f3cfbd6bce5883f46d8527c6853a86a2 rpc: Allow gettxoutsetinfo and getblockstats for stale blocks (Fabian Jahr) b9362392aef2689bc106c20925859ede555d082b index, rpc: Add use_index option for gettxoutsetinfo (Fabian Jahr) bb7788b121a30489bc81a1f46dde6a9b19ae4ec1 test: Test coinstatsindex robustness across restarts (Fabian Jahr) e0938c29099635150014ffc9bb0cafa8049ec55a test: Add tests for block_info in gettxoutsetinfo (Fabian Jahr) 2501576eccb08af80471c7b7b843b189ad6758c0 rpc, index: Add verbose amounts tracking to Coinstats index (Fabian Jahr) 655d929836a71af23d2035d2e2e99ad8b8c340c3 test: add coinstatsindex getindexinfo coverage, improve current tests (Jon Atack) ca01bb8d689f93e1c7669b0ba7a4994c0206dabd rpc: Add Coinstats index to getindexinfo (Fabian Jahr) 57a026c30fef3138bb8db46e6865acb9dc2674f8 test: Add unit test for Coinstats index (Fabian Jahr) 6a4c0c09ab4d073a26c3c4a02783d5dcd88f6eef test: Add functional test for Coinstats index (Fabian Jahr) 3f166ecc125fce6ccd995687fa16572090a5d099 rpc: gettxoutsetinfo can be requested for specific blockheights (Fabian Jahr) 3c914d58ff323255b32e717d0ce28209ec0abdaa index: Coinstats index can be activated with command line flag (Fabian Jahr) dd58a4de21469d6d848ae309edc47f558628221d index: Add Coinstats index (Fabian Jahr) a8a46c4b3cfda4b95c92a36f8cebd3606377e57d refactor: Simplify ApplyStats and ApplyHash (Fabian Jahr) 9c8a265fd21a87228c18a1661df99fedc1866baf refactor: Pass hash_type to CoinsStats in stats object (Fabian Jahr) 2e2648a9021dfbb6e17dfa81472f057dacbc34e0 crypto: Make MuHash Remove method efficient (Fabian Jahr) Pull request description: This is part of the coinstats index project tracked in #18000 While the review of the new UTXO set hash algorithm (MuHash) takes longer recently #19328 was merged which added the possibility to run `gettxoutsetinfo` with a specific hash type. As the first type it added `hash_type=none` which skips the hashing of the UTXO set altogether. This alone did not make `gettxoutsetinfo` much faster but it allows the use of an index for the remaining coin statistics even before a new hashing algorithm has been added. Credit to Sjors for the idea to take this intermediate step. Features summary: - Users can start their node with the option `-coinstatsindex` which syncs the index in the background - After the index is synced the user can use `gettxoutsetinfo` with `hash_type=none` or `hash_type=muhash` and will get the response instantly out of the index - The user can specify a height or block hash when calling `gettxoutsetinfo` to see coin statistics at a specific block height ACKs for top commit: Sjors: re-tACK 5f96d7d22d8e05876c6fc014e70488699950fe38 jonatack: Code review re-ACK 5f96d7d22d8e05876c6fc014e70488699950fe38 per `git range-diff 13d27b4 07201d3 5f96d7d` promag: Tested ACK 5f96d7d22d8e05876c6fc014e70488699950fe38. Light code review ACK 5f96d7d22d8e05876c6fc014e70488699950fe38. Tree-SHA512: cbca78bee8e9605c19da4fbcd184625fb280200718396c694a56c7daab6f44ad23ca9fb5456d09f245d8b8d9659fdc2b3f3ce5e953c1c6cf4003dbc74c0463c2
2021-04-27Merge bitcoin/bitcoin#21523: validation: run VerifyDB on all chainstatesMarcoFalke
844ad0eccaa1dbfefc30d19804d95d67c3d5f06d doc: IsSnapshotActive (James O'Beirne) 9b604c0207b526c008617cdca210f35db5e344db validation: prepare VerifyDB for assumeutxo (James O'Beirne) 7901647d722bcc5b0554ad73b14481cbe73608e4 refactor: rename active_chainstate in VerifyDB (James O'Beirne) Pull request description: This is part of the [assumeutxo project](https://github.com/bitcoin/bitcoin/projects/11) (parent PR: #15606) --- ~~Pretty cut and dry; parameterizes `CVerifyDB` methods so that we can run the verify procedure on multiple chainstates.~~ Two minor tweaks to ensure that `VerifyDB` can be run on multiple chainstates and a corresponding rename. ACKs for top commit: fjahr: Code review re-ACK 844ad0eccaa1dbfefc30d19804d95d67c3d5f06d MarcoFalke: review ACK 844ad0eccaa1dbfefc30d19804d95d67c3d5f06d 🐥 Tree-SHA512: 26a398cf4dabc1aa0850743921dba0452b4813848a3c777586dc981716737e98e17b8110254a5c41af95dd236e0c00dc8b4eee891d69bef825a5e1911fc499d0
2021-04-27move-only: Move functions to blockstorageMarcoFalke
2021-04-27move-only: Move constants to blockstorageMarcoFalke
2021-04-27refactor: Move block storage globals to blockstorageMarcoFalke
However, keep a declaration in validation to make it possible to move smaller chunks to blockstorage without breaking compilation. Also, expose AbortNode in the header. Can be reviewed with --color-moved=dimmed-zebra --color-moved-ws=ignore-all-space
2021-04-27refactor: Move pruning/reindex/importing globals to blockstorageMarcoFalke
Can be reviewed with --color-moved=dimmed-zebra
2021-04-27Merge bitcoin/bitcoin#21009: Remove RewindBlockIndex logicW. J. van der Laan
d831e711cab83c70bf2ded62fe33f484844e73dd [validation] RewindBlockIndex no longer needed (Dhruv Mehta) Pull request description: Closes #17862 Context from [original comment](https://github.com/bitcoin/bitcoin/issues/17862#issuecomment-744285188) (minor edits): `RewindBlockIndex()` is a mechanism to allow nodes to be upgraded after segwit activation, while still keeping their chainstate/datadir in a consistent state. It works as follows: - A pre-segwit (i.e. v0.13.0 or older) node is running. - Segwit activates. The pre-segwit node remains sync'ed to the tip, but is not enforcing the new segwit rules. - The user upgrades the node to a segwit-aware version (v0.13.1 or newer). - On startup, in `AppInitMain()`, `RewindBlockIndex()` is called. This walks the chain backwards from the tip, disconnecting and erasing blocks that from after segwit activation that weren't validated with segwit rules. - those blocks are then redownloaded (with witness data) and validated with segwit rules. This logic probably isn't required any more since: - Segwit activated at height 481824, when the block chain was 130GB and the total number of txs was 250 million. Today, we're at height 667704, the blockchain is over 315GB and the total number of txs is over 600 million. Even if 20% of that added data is witness data (a high estimate), then around 150GB of transactions would need to be rewound to get back to segwit activation height. It'd probably be faster to simply validate from genesis, especially since we won't be validating any scripts before the assumevalid block. It's also unclear whether rewinding 150GB of transactions would even work. It's certainly never been tested. - Bitcoin Core v0.13 is hardly used any more. https://luke.dashjr.org/programs/bitcoin/files/charts/software.html shows less than 50 nodes running it. The software was EOL on Aug 1st 2018. It's very unlikely that anyone is running 0.13 and will want to upgrade to 0.22. This PR introduces `NeedsRedownload()` which merely checks for insufficiently validated segwit blocks and requests that the user restarts the node with `-reindex`. Reindexing the block files upon restart will make the node rebuild chain state and block index from the `blk*.dat` files on disk. The node won't be able to index the blocks with `BLOCK_OPT_WITNESS`, so they will be missing from the chain and be re-downloaded, with witness data. Removing this code allows the following (done in follow-up #21090): - removal of tests using `segwitheight=-1` in `p2p_segwit.py`. - in turn, that allows us to drop support for `-segwitheight=-1`, which is only supported for that test. - that allows us to always set `NODE_WITNESS` in our local services. The only reason we don't do that is to support `-segwitheight=-1`. - that in turn allows us to drop all of the `GetLocalServices() & NODE_WITNESS` checks inside `net_processing.cpp`, since our local services would always include `NODE_WITNESS` ACKs for top commit: jnewbery: utACK d831e711cab83c70bf2ded62fe33f484844e73dd jamesob: ACK https://github.com/bitcoin/bitcoin/pull/21009/commits/d831e711cab83c70bf2ded62fe33f484844e73dd laanwj: Cursory code review ACK d831e711cab83c70bf2ded62fe33f484844e73dd. Agree with the direction of the change, thanks for simplifying the logic here. glozow: utACK d831e711cab83c70bf2ded62fe33f484844e73dd Tree-SHA512: 3eddf5121ccd081ad7f15a5c6478ef867083edc8ba0bf1ee759e87bc070ee3d2f0698a3feba8db8dc087987c8452887b6f72cff05b3e178f41cb10a515fb8053
2021-04-23validation: prepare VerifyDB for assumeutxoJames O'Beirne
Removes assumptions of use only on the active chainstate.
2021-04-23refactor: rename active_chainstate in VerifyDBJames O'Beirne
To prepare VerifyDB semantics for multiple chainstate use.
2021-04-21[validation] RewindBlockIndex no longer neededDhruv Mehta
Instead of rewinding blocks, we request that the user restarts with -reindex
2021-04-19refactor: Pass hash_type to CoinsStats in stats objectFabian Jahr
2021-04-18Change GetBlocksDir() to ArgsManager.GetBlocksDirPath().Kiminuo
2021-04-14validation: fix ActivateSnapshot to use hardcoded nChainTxJames O'Beirne
This fixes an oversight from the move of nChainTx from the user-supplied snapshot metadata into the hardcoded assumeutxo chainparams. Since the nChainTx is now unused in the metadata, it should be removed in a future commit.
2021-04-14Merge #21585: Fix assumeutxo crash due to truncated fileMarcoFalke
fa73ce6e653d00824eb68f772fd29b7f8fb93d84 Fix assumeutxo crash due to truncated file (MarcoFalke) Pull request description: ACKs for top commit: jamesob: ACK fa73ce6e653d00824eb68f772fd29b7f8fb93d84 ryanofsky: Code review ACK fa73ce6e653d00824eb68f772fd29b7f8fb93d84. Easy fix. It seems like this could have been caught in review, though. Tree-SHA512: 3a98687c386e3995114ddf0ad7194fadd9520989290681ef703b578e3ca21aee51eadfb83aa38a489bac13d12709ea137b9b184b08e5bfa2919cca177aab90be
2021-04-13Merge #21575: refactor: Create blockstorage modulefanquake
fadcd3f78e1dd1acd7a774f8fad68dc471ff9e1f doc: Remove irrelevant link to GitHub (MarcoFalke) fa121b628d51bb0e25eb3fbd716881fa55527dc7 blockstorage: [refactor] Use chainman reference where possible (MarcoFalke) fa0c7d9ad24d3c9515d3f9c136af4071cbd79055 move-only: Move *Disk functions to blockstorage (MarcoFalke) fa91b2b2b3447a3645e7958c7dc4e1946a69cb9c move-only: Move AbortNode to shutdown (MarcoFalke) fa413f07a14744e7d7f7746e861aabd9cf938f61 move-only: Move ThreadImport to blockstorage (MarcoFalke) faf843c07f99f91603e08ea858f972516f1d669a refactor: Move load block thread into ChainstateManager (MarcoFalke) Pull request description: This picks up the closed pull request #21030 and is the first step toward fixing #21220. The basic idea is to move all disk access into a separate module with benefits: * Breaking down the massive files init.cpp and validation.cpp into logical units * Creating a standalone-module to reduce the mental complexity * Pave the way to fix validation related circular dependencies * Pave the way to mock disk access for testing, especially where it is performance critical (like fuzzing) ACKs for top commit: promag: Code review ACK fadcd3f78e, checked (almost) moved only changes. This is a nice tidy up change and doesn't change behavior. Easily reviewed commit by commit. jamesob: ACK fadcd3f78e1dd1acd7a774f8fad68dc471ff9e1f ([`jamesob/ackr/21575.1.MarcoFalke.refactor_create_blocksto`](https://github.com/jamesob/bitcoin/tree/ackr/21575.1.MarcoFalke.refactor_create_blocksto)) ryanofsky: Code review ACK fadcd3f78e1dd1acd7a774f8fad68dc471ff9e1f. New organization makes sense, moves extraneous things outside of validation.cpp. PR is also easy to review with helpfully split up moveonly commits. Tree-SHA512: 917996592b6d8f9998289d8cb2b1b78b23d1fdb3b07216c9caec1380df33baa09dc2c1e706da669d440b497e79c9c62a01ca20dc202df5ad974a75f3ef7a143b