aboutsummaryrefslogtreecommitdiff
path: root/src/index
AgeCommit message (Collapse)Author
2022-05-03Merge bitcoin/bitcoin#24470: Disallow more unsafe string->path conversions ↵MacroFake
allowed by path append operators f64aa9c411ad78259756a28756ec1eb8069b5ab4 Disallow more unsafe string->path conversions allowed by path append operators (Ryan Ofsky) Pull request description: Add more `fs::path` `operator/` and `operator+` overloads to prevent unsafe string->path conversions on Windows that would cause strings to be decoded according to the current Windows locale & code page instead of the correct string encoding. Update application code to deal with loss of implicit string->path conversions by calling `fs::u8path` or `fs::PathFromString` explicitly, or by just changing variable types from `std::string` to `fs::path` to avoid conversions altogether, or make them happen earlier. In all cases, there's no change in behavior either (1) because strings only contained ASCII characters and would be decoded the same regardless of what encoding was used, or (2) because of the 1:1 mapping between paths and strings using the `PathToString` and `PathFromString` functions. Motivation for this PR was just that I was experimenting with #24469 and noticed that operations like `fs::path / std::string` were allowed, and I thought it would be better not to allow them. ACKs for top commit: hebasto: ACK f64aa9c411ad78259756a28756ec1eb8069b5ab4 Tree-SHA512: 944cce49ed51537ee7a35ea4ea7f5feaf0c8fff2fa67ee81ec5adebfd3dcbaf41b73eb35e49973d5f852620367f13506fd12a7a9b5ae3a7a0007414d5c9df50f
2022-05-02Make sanity check in GCSFilter constructor optionalPatrick Strateman
BlockFilterIndex will perform the cheaper check of verifying the filter hash when reading the filter from disk.
2022-04-28blockstorage, refactor: pass GetFirstStoredBlock() start_block by referenceJon Atack
instead of by pointer, so as to not accept a nullptr.
2022-04-28blockstorage, refactor: make GetFirstStoredBlock() a member of BlockManagerJon Atack
instead of a global
2022-04-25Index: Use prune locks for blockfilterindexFabian Jahr
Prior to this change blocks could be pruned up to the last block before the blockfilterindex current best block.
2022-04-25refactor: Introduce GetFirstStoredBlock helper functionFabian Jahr
2022-04-21Disallow more unsafe string->path conversions allowed by path append operatorsRyan Ofsky
Add more fs::path operator/ and operator+ overloads to prevent unsafe string->path conversions on Windows that would cause strings to be decoded according to the current Windows locale & code page instead of the correct string encoding. Update application code to deal with loss of implicit string->path conversions by calling fs::u8path or fs::PathFromString explicitly, or by just changing variable types from std::string to fs::path to avoid conversions altoghther, or make them happen earlier. In all cases, there's no change in behavior either (1) because strings only contained ASCII characters and would be decoded the same regardless of what encoding was used, or (2) because of the 1:1 mapping between paths and strings using the PathToString and PathFromString functions. Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
2022-03-09refactor: more const annotations for uses of CBlockIndex*Anthony Towns
2022-02-21index: Commit DB_MUHASH and DB_BEST_BLOCK to disk togetherMartin Zumsande
If these are written to disk at different times, unclean shutdowns can lead to index corruption.
2022-02-17index: check muhash is in sync on coinstatsindex launchFabian Jahr
2022-02-17index: remove txindex references from base indexFabian Jahr
2022-01-31index: Don't commit without valid m_best_block_indexMartin Zumsande
Also report an error when coinstatsindex init fails.
2022-01-25Require CBlockIndex::GetBlockPos() to hold mutex cs_mainJon Atack
2022-01-06Add src/node/* code to node:: namespaceRussell Yanofsky
2022-01-05Merge bitcoin/bitcoin#23411: refactor: Avoid integer overflow in ApplyStats ↵MarcoFalke
when activating snapshot fa996c58e8a31ebe610d186cef408b6dd3b385a8 refactor: Avoid integer overflow in ApplyStats when activating snapshot (MarcoFalke) fac01888d17423d6c23a9ce15d98fc88fb34e3cc Move AdditionOverflow to util, Add CheckedAdd with unit tests (MarcoFalke) fa526d8fb6ab8f2678a30d4536aa9c45218f5269 Add dev doc to CCoinsStats::m_hash_type and make it const (MarcoFalke) faff051560552d4405896e01920a18f698155a56 style: Remove unused whitespace (MarcoFalke) Pull request description: A snapshot contains the utxo set, including the out value. To activate the snapshot, the hash needs to be calculated. As a side-effect, the total amount in the snapshot is calculated (as the sum of all out values), but never used. Instead of running into an integer overflow in an unused result, don't calculate the result in the first place. Other code paths (using the active utxo set) can not run into an integer overflow, since the active utxo set is valid. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=39716 ACKs for top commit: shaavan: reACK fa996c58e8a31ebe610d186cef408b6dd3b385a8 vasild: ACK fa996c58e8a31ebe610d186cef408b6dd3b385a8 Tree-SHA512: 4f207f634841f6f634fd02ae1e5907e343fd767524fd0e8149aa99fa9a1834fe50167d14874834d45236e9c325d567925f28129bacb7d80be29cf22277a16a14
2021-12-30scripted-diff: Bump copyright headersHennadii Stepanov
-BEGIN VERIFY SCRIPT- ./contrib/devtools/copyright_header.py update ./ -END VERIFY SCRIPT- Commits of previous years: * 2020: fa0074e2d82928016a43ca408717154a1c70a4db * 2019: aaaaad6ac95b402fe18d019d67897ced6b316ee0
2021-12-17refactor: Avoid integer overflow in ApplyStats when activating snapshotMarcoFalke
2021-12-16Merge bitcoin/bitcoin#23785: refactor: Move stuff to ChainstateManagerMarcoFalke
fab6d6b2d154893ab422dda87f3535d42c3e06f4 Move pindexBestInvalid to ChainstateManager (MarcoFalke) facd2137eceacb95e1f71c87ddc704d752b37272 Move m_failed_blocks to ChainstateManager (MarcoFalke) fa47b5c100f81c65c15b5a6afaf6c91bc0861264 Move AcceptBlockHeader to ChainstateManager (MarcoFalke) fa3d62cf7b3501a056b34c5458c14d2fe6a55bd7 Move FindForkInGlobalIndex from BlockManager to CChainState (MarcoFalke) Pull request description: Move globals or members of the wrong class to the right class. ACKs for top commit: naumenkogs: ACK fab6d6b2d154893ab422dda87f3535d42c3e06f4 Sjors: ACK fab6d6b2d154893ab422dda87f3535d42c3e06f4 shaavan: ACK fab6d6b2d154893ab422dda87f3535d42c3e06f4 Tree-SHA512: 926cbdfa22838517497bacb79ed5f521f64117c2aacf96a0176f62831b4713314a32abc0213df5ee067edf63e4a4300f752a26006d36e5aab415bb91209a271f
2021-12-15Move FindForkInGlobalIndex from BlockManager to CChainStateMarcoFalke
The helper was moved in commit b026e318c39f59a06e29f1b25c7f577e01b25ccb, which also mentioned that it could be moved to CChainState. So do that, as the functionality is not block-storage related. This also allows to drop one function argument.
2021-12-15doc: add explanations for assert in index and magic numbers in testMartin Zumsande
These were suggested in review of #23365 Co-authored-by: John Newbery <john@johnnewbery.com>
2021-11-26index: Fix backwards search for bestblockMartin Zumsande
This allows filters to be reconstructed when the best known block is the Genesis block without needing to reindex. It fixes Init errors seen in #23289.
2021-10-12Fix outdated comments referring to ::ChainActive()Samuel Dobson
2021-10-01Add syscall sandboxing (seccomp-bpf)practicalswift
2021-08-20Remove txindex migration codeMarcoFalke
2021-08-20Add missing includes and forward declarations, remove unused onesMarcoFalke
2021-07-28Merge bitcoin/bitcoin#22047: index, rpc: Coinstatsindex follow-upsW. J. van der Laan
779e638ca9b2b37c247577d225b93ac762b0602f coinstats: Add comments for new coinstatsindex values (Fabian Jahr) 5b3d4e724f377834e24b1f014787cc7aa7fc30fe Index: Improve logging in coinstatsindex (Fabian Jahr) d4356d4e48f59c63894b68691cc21ed4892ee716 rpc: Block until synced if coinstatsindex is used in gettxoutsetinfo (Fabian Jahr) a5f6791139554936d13f367660283899a37ff5c7 rpc: Add missing gettxoutsetinfo help docs (Fabian Jahr) 01386bfd88019397237256cb16f91de346eb66f2 Index: Return early from failed coinstatsindex init (Fabian Jahr) 1e3842385b8c0d15086c7cd8736f8c67e6c0c285 index: Use batch writing in coinstatsindex WriteBlock (Fabian Jahr) fb65dde147f63422c4148b089c2f5be0bf5ba80f scripted-diff: Fix coinstats data member names (Fabian Jahr) 8ea8c927ac05980d6a81252e40b7444e9abb74f9 index: Avoid unnecessary type casts in coinstatsindex (Fabian Jahr) Pull request description: This is a collection of smaller follow-ups to #19521, addressing several post-merge review comments. ACKs for top commit: Sjors: re-utACK 779e638ca9b2b37c247577d225b93ac762b0602f jonatack: re-ACK 779e638ca9b2b37c247577d225b93ac762b0602f diff since last review involves doc changes only; rebased to current master and verified clean debug build/no silent conflicts, unit tests, and feature_coinstatsindex functional test laanwj: Code review ACK 779e638ca9b2b37c247577d225b93ac762b0602f Talkless: re-utACK 779e638ca9b2b37c247577d225b93ac762b0602f after cosmetic changes. Tree-SHA512: cb0d038d230c582d7fe3041c89b1e04d39971fab3739d540c609cf826754c6c513b12ded08ac92180aec7a9d7a70114ece50357bd1a902de4adaae9f30b8d699
2021-07-25Index: Improve logging in coinstatsindexFabian Jahr
More accurate logging of a warning should make clear if the recovery condition was hit while catching the results of the previous block.
2021-07-15Move pblocktree global to BlockManagerMarcoFalke
2021-06-10scripted-diff: tree-wide: Remove all review-only assertionsCarl Dong
-BEGIN VERIFY SCRIPT- find_regex='((assert|CHECK_NONFATAL)\(std::addressof|TODO: REVIEW-ONLY)' \ && git grep -l -E "$find_regex" -- . \ | xargs sed -i -E "/${find_regex}/d" -END VERIFY SCRIPT-
2021-06-03Index: Return early from failed coinstatsindex initFabian Jahr
2021-06-03index: Use batch writing in coinstatsindex WriteBlockFabian Jahr
2021-06-03scripted-diff: Fix coinstats data member namesFabian Jahr
Initially these values were 'per block' in an earlier version but were then changed to total values. The names were not updated to reflect that. -BEGIN VERIFY SCRIPT- s() { git grep -l "$1" src | xargs sed -i "s/$1/$2/g"; } s 'm_block_unspendable_amount' 'm_total_unspendable_amount' s 'm_block_prevout_spent_amount' 'm_total_prevout_spent_amount' s 'm_block_new_outputs_ex_coinbase_amount' 'm_total_new_outputs_ex_coinbase_amount' s 'm_block_coinbase_amount' 'm_total_coinbase_amount' s 'block_unspendable_amount' 'total_unspendable_amount' s 'block_prevout_spent_amount' 'total_prevout_spent_amount' s 'block_new_outputs_ex_coinbase_amount' 'total_new_outputs_ex_coinbase_amount' s 'block_coinbase_amount' 'total_coinbase_amount' s 'unspendables_genesis_block' 'total_unspendables_genesis_block' s 'unspendables_bip30' 'total_unspendables_bip30' s 'unspendables_scripts' 'total_unspendables_scripts' s 'unspendables_unclaimed_rewards' 'total_unspendables_unclaimed_rewards' s 'm_unspendables_genesis_block' 'm_total_unspendables_genesis_block' s 'm_unspendables_bip30' 'm_total_unspendables_bip30' s 'm_unspendables_scripts' 'm_total_unspendables_scripts' s 'm_unspendables_unclaimed_rewards' 'm_total_unspendables_unclaimed_rewards' -END VERIFY SCRIPT-
2021-06-01Merge bitcoin/bitcoin#21767: [Bundle 6/n] Prune g_chainman usage in ↵MarcoFalke
auxiliary modules 7a799c9c2b8652e780d1fd5e1bf7d05b026c1c1a index: refactor-only: Reuse CChain ref (Carl Dong) db33cde80fff749c6adff9e91fca5f27f4bb6278 index: Add chainstate member to BaseIndex (Carl Dong) f4a47a1febfa35ab077f2a841fe31a8cd9618250 bench: Use existing chainman in AssembleBlock (Carl Dong) 91226eb91769aad5a63bc671595e1353a2b2247a bench: Use existing NodeContext in DuplicateInputs (Carl Dong) e6b4aa6eb53dc555ecab2922af35e7a2572faf4f miner: Pass in chainman to RegenerateCommitments (Carl Dong) 9ecade14252ad1972f668d2d2e4ef44fdfcb944a rest: Add GetChainman function and use it (Carl Dong) fc1c282845f6b8436d1ea4c68eb3511034c29bea rpc/blockchain: Use existing blockman in gettxoutsetinfo (Carl Dong) Pull request description: Overall PR: #20158 (tree-wide: De-globalize ChainstateManager) The first 2 commits are fixups addressing review for the last bundle: #21391 NEW note: 1. I have opened #21766 which keeps track of potential improvements where the flaws already existed before the de-globalization work, please post on that issue about these improvements, thanks! Note to reviewers: 1. This bundle may _apparently_ introduce usage of `g_chainman` or `::Chain(state|)Active()` globals, but these are resolved later on in the overall PR. [Commits of overall PR](https://github.com/bitcoin/bitcoin/pull/20158/commits) 2. There may be seemingly obvious local references to `ChainstateManager` or other validation objects which are not being used in callers of the current function in question, this is done intentionally to **_keep each commit centered around one function/method_** to ease review and to make the overall change systematic. We don't assume anything about our callers. Rest assured that once we are considering that particular caller in later commits, we will use the obvious local references. [Commits of overall PR](https://github.com/bitcoin/bitcoin/pull/20158/commits) 3. When changing a function/method that has many callers (e.g. `LookupBlockIndex` with 55 callers), it is sometimes easier (and less error-prone) to use a scripted-diff. When doing so, there will be 3 commits in sequence so that every commit compiles like so: 1. Add `new_function`, make `old_function` a wrapper of `new_function`, divert all calls to `old_function` to `new_function` **in the local module only** 2. Scripted-diff to divert all calls to `old_function` to `new_function` **in the rest of the codebase** 3. Remove `old_function` ACKs for top commit: jarolrod: ACK 7a799c9 ariard: Code Review ACK 7a799c9 fjahr: re-ACK 7a799c9c2b8652e780d1fd5e1bf7d05b026c1c1a MarcoFalke: review ACK 7a799c9c2b8652e780d1fd5e1bf7d05b026c1c1a 🌠 ryanofsky: Code review ACK 7a799c9c2b8652e780d1fd5e1bf7d05b026c1c1a. Basically no change since last review except fixed rebase conflicts and a new comment about REST Ensure() jamesob: conditional ACK 7a799c9c2b8652e780d1fd5e1bf7d05b026c1c1a ([`jamesob/ackr/21767.1.dongcarl.bundle_6_n_prune_g_chai`](https://github.com/jamesob/bitcoin/tree/ackr/21767.1.dongcarl.bundle_6_n_prune_g_chai)) Tree-SHA512: 531c00ddcb318817457db2812d9a9d930bc664e58e6f7f1c746350732b031dd624270bfa6b9f49d8056aeb6321d973f0e38e4ff914acd6768edd8602c017d10e
2021-05-27index: refactor-only: Reuse CChain refCarl Dong
2021-05-27index: Add chainstate member to BaseIndexCarl Dong
2021-05-24index: Avoid unnecessary type casts in coinstatsindexFabian Jahr
2021-05-24scripted-diff: Replace `GetDataDir()` calls with `gArgs.GetDataDirNet()` callsKiminuo
-BEGIN VERIFY SCRIPT- git ls-files -- 'src' ':(exclude)src/util/system.h' ':(exclude)src/util/system.cpp' | xargs sed -i 's/GetDataDir()/gArgs.GetDataDirNet()/g'; -END VERIFY SCRIPT-
2021-05-12index: Avoid async shutdown on init errorMarcoFalke
2021-05-12Merge bitcoin/bitcoin#19064: refactor: Cleanup thread ctor callsMarcoFalke
792be53d3e9e366b9f6aeee7a1eeb912fa28062e refactor: Replace std::bind with lambdas (Hennadii Stepanov) a508f718f3e087c96a306399582a85df2e1d53ae refactor: Use appropriate thread constructor (Hennadii Stepanov) 30e44482152488a78f2c495798a75e6f553dc0c8 refactor: Make TraceThread a non-template free function (Hennadii Stepanov) Pull request description: This PR does not change behavior. Its goal is to improve readability and maintainability of the code. ACKs for top commit: jnewbery: utACK 792be53d3e9e366b9f6aeee7a1eeb912fa28062e jonatack: tACK 792be53d3e9e366b9f6aeee7a1eeb912fa28062e MarcoFalke: cr ACK 792be53d3e9e366b9f6aeee7a1eeb912fa28062e Tree-SHA512: a03142f04f370f6bc02bd3ddfa870819b51740fcd028772241d68c84087f95a2d78207cbd5edb3f7c636fcf2d76192d9c59873f8f0af451d3b05c0cf9cf234df
2021-05-05Merge bitcoin/bitcoin#21727: refactor: Move more stuff to blockstorageW. J. van der Laan
fa09a9eac8d8ab65ce4064c35a9f21349a644982 style: Add { } to multi-line if (MarcoFalke) fadafab83379ff10d86ada179c6f9641d19464fe move-only: Move functions to blockstorage (MarcoFalke) fa7e64d58615fffea91cd64dc4a2790221ceff0a move-only: Move constants to blockstorage (MarcoFalke) fa247a327fc7c7cea6bc8f93637b8babd3015ffa refactor: Move block storage globals to blockstorage (MarcoFalke) fa81c30c6f1adac79517c958090db174eb6aeda2 refactor: Move pruning/reindex/importing globals to blockstorage (MarcoFalke) Pull request description: See #21575 ACKs for top commit: Sjors: ACK fa09a9eac8d8ab65ce4064c35a9f21349a644982 kiminuo: ACK fa09a9e laanwj: Code review ACK fa09a9eac8d8ab65ce4064c35a9f21349a644982 promag: Code review ACK fa09a9eac8d8ab65ce4064c35a9f21349a644982. Since last review Tree-SHA512: 2eb6962ff44da6b77f3058fc02ec66ab742e25ae8dcc8ec62b062896571910d43ca7c4bb16fb3ccb5e5245195b8dec6384b6c8d442fa97ca28d93bdff347d677
2021-05-01refactor: [index] Replace deprecated char with uint8_t in serializationMarcoFalke
2021-04-29refactor: Replace std::bind with lambdasHennadii Stepanov
Lambdas are shorter and more readable. Changes are limited to std::thread ctor calls only.
2021-04-27move-only: Move functions to blockstorageMarcoFalke
2021-04-25refactor: Make TraceThread a non-template free functionHennadii Stepanov
Also it is moved into its own module.
2021-04-19rpc, index: Add verbose amounts tracking to Coinstats indexFabian Jahr
2021-04-19index: Add Coinstats indexFabian Jahr
The index holds the values previously calculated in coinstats.cpp for each block, representing the state of the UTXO set at each height.
2021-04-13Merge #21575: refactor: Create blockstorage modulefanquake
fadcd3f78e1dd1acd7a774f8fad68dc471ff9e1f doc: Remove irrelevant link to GitHub (MarcoFalke) fa121b628d51bb0e25eb3fbd716881fa55527dc7 blockstorage: [refactor] Use chainman reference where possible (MarcoFalke) fa0c7d9ad24d3c9515d3f9c136af4071cbd79055 move-only: Move *Disk functions to blockstorage (MarcoFalke) fa91b2b2b3447a3645e7958c7dc4e1946a69cb9c move-only: Move AbortNode to shutdown (MarcoFalke) fa413f07a14744e7d7f7746e861aabd9cf938f61 move-only: Move ThreadImport to blockstorage (MarcoFalke) faf843c07f99f91603e08ea858f972516f1d669a refactor: Move load block thread into ChainstateManager (MarcoFalke) Pull request description: This picks up the closed pull request #21030 and is the first step toward fixing #21220. The basic idea is to move all disk access into a separate module with benefits: * Breaking down the massive files init.cpp and validation.cpp into logical units * Creating a standalone-module to reduce the mental complexity * Pave the way to fix validation related circular dependencies * Pave the way to mock disk access for testing, especially where it is performance critical (like fuzzing) ACKs for top commit: promag: Code review ACK fadcd3f78e, checked (almost) moved only changes. This is a nice tidy up change and doesn't change behavior. Easily reviewed commit by commit. jamesob: ACK fadcd3f78e1dd1acd7a774f8fad68dc471ff9e1f ([`jamesob/ackr/21575.1.MarcoFalke.refactor_create_blocksto`](https://github.com/jamesob/bitcoin/tree/ackr/21575.1.MarcoFalke.refactor_create_blocksto)) ryanofsky: Code review ACK fadcd3f78e1dd1acd7a774f8fad68dc471ff9e1f. New organization makes sense, moves extraneous things outside of validation.cpp. PR is also easy to review with helpfully split up moveonly commits. Tree-SHA512: 917996592b6d8f9998289d8cb2b1b78b23d1fdb3b07216c9caec1380df33baa09dc2c1e706da669d440b497e79c9c62a01ca20dc202df5ad974a75f3ef7a143b
2021-04-05move-only: Move *Disk functions to blockstorageMarcoFalke
Can be reviewed with the git options --color-moved=dimmed-zebra --color-moved-ws=ignore-all-space
2021-04-05refactor: Remove negative lock annotations from globalsMarcoFalke
2021-03-11scripted-diff: remove MakeUnique<T>()fanquake
-BEGIN VERIFY SCRIPT- git rm src/util/memory.h sed -i -e 's/MakeUnique/std::make_unique/g' $(git grep -l MakeUnique src) sed -i -e '/#include <util\/memory.h>/d' $(git grep -l '#include <util/memory.h>' src) sed -i -e '/util\/memory.h \\/d' src/Makefile.am -END VERIFY SCRIPT-
generated by cgit v1.2.3 (git 2.26.2) at 2024-11-06 16:35:34 +0000