aboutsummaryrefslogtreecommitdiff
path: root/src/index
AgeCommit message (Collapse)Author
2024-09-17refactor: rename BlockKey to BlockRefSjors Provoost
2024-09-16Merge bitcoin/bitcoin#30884: streams: cache file position within AutoFileAva Chow
a240e150e837b5a95ed19765a2e8b7c5b6013f35 streams: remove AutoFile::Get() entirely (Pieter Wuille) e624a9bef16b6335fd119c10698352b59bf2930a streams: cache file position within AutoFile (Pieter Wuille) Pull request description: Fixes #30833. Instead of relying on frequent `ftell` calls (which appear to cause a significant slowdown on some systems) in XOR-enabled `AutoFile`s, cache the file position within `AutoFile` itself. ACKs for top commit: achow101: ACK a240e150e837b5a95ed19765a2e8b7c5b6013f35 davidgumberg: untested reACK https://github.com/bitcoin/bitcoin/pull/30884/commits/a240e150e837b5a95ed19765a2e8b7c5b6013f35 theStack: Code-review ACK a240e150e837b5a95ed19765a2e8b7c5b6013f35 Tree-SHA512: fd3681edc018afaf955dc7a41a0c953ca80d46c1129e3c5b306c87c95aae93b2fe7b900794eb8b6f10491f9211645e7939918a28838295e6873eb226fca7006f
2024-09-13streams: remove AutoFile::Get() entirelyPieter Wuille
Co-Authored-By: David Gumberg <davidzgumberg@gmail.com>
2024-09-13streams: cache file position within AutoFilePieter Wuille
2024-09-12util: Use compile-time check for FatalErrorfMarcoFalke
2024-07-08tidy: modernize-use-equals-defaultMarcoFalke
2024-06-13refactor: remove warnings globalsstickies-v
2024-06-13move-only: move warnings from common to nodestickies-v
Since rpc/util.cpp is in common, also move GetNodeWarnings() to node::GetWarningsForRPC()
2024-04-17index: race fix, lock cs_main while 'm_synced' is subject to changeRyan Ofsky
This ensures that the index does not miss any 'new block' signals occurring in-between reading the 'next block' and setting 'm_synced'. Because, if this were to happen, the ignored blocks would never be indexed, thus stalling the index forever.
2024-04-01Fix #29767, set m_synced = true after Commit()nanlour
2024-03-22Merge bitcoin/bitcoin#29672: validation: Make translations of fatal errors ↵Ava Chow
consistent 824f47294a309ba8e58ba8d1da0af15d8d828f43 node: Use log levels in noui_ThreadSafeMessageBox (TheCharlatan) ddc7872c08b7ddf9b1e83abdb97c21303f4a9172 node: Make translations of fatal errors consistent (TheCharlatan) Pull request description: The extra `bilingual_str` argument of the fatal error notifications and `node::AbortNode()` is often unused and when used usually contains the same string as the message argument. It also seems to be confusing, since it is not consistently used for errors requiring user action. For example some assumeutxo fatal errors require the user to do something, but are not translated. So simplify the fatal error and abort node interfaces by only passing a translated string. This slightly changes the fatal errors displayed to the user. ACKs for top commit: stickies-v: re-ACK 824f47294a309ba8e58ba8d1da0af15d8d828f43 maflcko: ACK 824f47294a309ba8e58ba8d1da0af15d8d828f43 🔎 achow101: ACK 824f47294a309ba8e58ba8d1da0af15d8d828f43 hebasto: re-ACK 824f47294a309ba8e58ba8d1da0af15d8d828f43. Tree-SHA512: 2868ee7b045fe7f3ac582ce5039141b398480b7627734976201dafaaef7544b8461635a7292fee4a7f32ff1bfc26f9bd4d0c292dca424ba42fb7fc4483d7ce8d
2024-03-21node: Make translations of fatal errors consistentTheCharlatan
The extra `bilingual_str` argument of the fatal error notifications and `node::AbortNode()` is often unused and when used usually contains the same string as the message argument. It also seems to be confusing, since it is not consistently used for errors requiring user action. For example some assumeutxo fatal errors require the user to do something, but are not translated. So simplify the fatal error and abort node interfaces by only passing a translated string. This slightly changes the fatal errors displayed to the user. Also de-duplicate the abort error log since it is repeated in noui.cpp.
2024-03-20Merge bitcoin/bitcoin#29671: index: avoid "failed to commit" errors on ↵Ava Chow
initialization f65b0f6401091e4a4ca4c9f4db1cf388f0336bad index: Move last_locator_write_time and logging to end of threadsync loop (Fabian Jahr) Pull request description: In the index sync thread, when initializing an index for the first time, stop callng BaseIndex::Commit when m_best_block_index is null, to avoid a spurious "failed to commit" error from that function. This error started happening in commit https://github.com/bitcoin/bitcoin/commit/7878f97bf15b6e7c9b47d1c0d96419b97e1bdcbd from https://github.com/bitcoin/bitcoin/pull/25494 and was reported by pstratem in https://github.com/bitcoin/bitcoin/pull/26903 with an alternate fix. ACKs for top commit: achow101: ACK f65b0f6401091e4a4ca4c9f4db1cf388f0336bad ryanofsky: Code review ACK f65b0f6401091e4a4ca4c9f4db1cf388f0336bad. Just moved log "Syncing" log line since last commit to avoid having to call now() twice. furszy: ACK f65b0f6401091e4a4ca4c9f4db1cf388f0336bad TheCharlatan: ACK f65b0f6401091e4a4ca4c9f4db1cf388f0336bad Tree-SHA512: afa8f05786318d36346d167ff53ea0b3bc8abdb0ad04465d199dc3eb91e9f837369e24fcb7e24b5757b02d698ec504e61da6ac365eaf006c874fc07a424a7e20
2024-03-19index: Move last_locator_write_time and logging to end of threadsync loopFabian Jahr
This avoids having commit print a needless error message during init. Co-authored-by: furszy <mfurszy@protonmail.com>
2024-03-12index: decrease ThreadSync cs_main contentionfurszy
Only NextSyncBlock requires cs_main lock. The other function calls like Commit or Rewind will lock or not cs_main internally when they need it. Avoiding keeping cs_main locked when Commit() or Rewind() write data to disk.
2024-03-12index: cache last block filter headerfurszy
Avoid disk read operations on every new processed block.
2024-03-12index: blockfilter, decouple header lookup into its own functionfurszy
2024-03-12index: blockfilter, decouple Write into its own functionfurszy
2024-03-12bench: basic block filter index initial syncfurszy
Introduce benchmark for the block filter index sync. And makes synchronous 'Sync()' mechanism accessible.
2024-03-11scripted-diff: Replace error() with LogError()MarcoFalke
This fixes the log output when -logsourcelocations is used. Also, instead of 'ERROR:', the log will now say '[error]', like other errors logged with LogError. -BEGIN VERIFY SCRIPT- sed -i --regexp-extended 's! error\("([^"]+)"! LogError("\1\\n"!g' $( git grep -l ' error(' ./src/ ) -END VERIFY SCRIPT-
2024-03-11scripted-diff: return error(...); ==> error(...); return false;MarcoFalke
This is needed for the next commit. -BEGIN VERIFY SCRIPT- # Separate sed invocations to replace one-line, and two-line error(...) calls sed -i --regexp-extended 's!( +)return (error\(.*\);)!\1\2\n\1return false;!g' $( git grep -l 'return error(' ) sed -i --null-data --regexp-extended 's!( +)return (error\([^\n]*\n[^\n]*\);)!\1\2\n\1return false;!g' $( git grep -l 'return error(' ) -END VERIFY SCRIPT-
2024-03-11refactor: Add missing {} around error() callsMarcoFalke
This is required for the next commit to be correct.
2024-02-15refactor: De-globalize g_signalsTheCharlatan
2023-12-04refactor: Remove calls to StartShutdown from KernelNotificationsRyan Ofsky
Use SignalInterrupt object instead. There is a slight change in behavior here because the previous StartShutdown code used to abort on failure and the new code logs errors instead.
2023-11-18blockstorage: switch from CAutoFile to AutoFileAnthony Towns
Also bump includes per suggestions from iwyu.
2023-11-16serialize: Drop useless version param from GetSerializeSize()Anthony Towns
2023-11-14Use ParamsWrapper for witness serializationAnthony Towns
2023-10-20coinstats: Fix hash_serialized2 calculationFabian Jahr
The legacy serialization was vulnerable to maleation and is fixed by adopting the same serialization procedure as was already in use for MuHash. This also includes necessary test fixes where the hash_serialized2 was hardcoded as well as correction of the regtest chainparams. Co-authored-by: Sebastian Falbesoner <sebastian.falbesoner@gmail.com>
2023-09-30validation: indexing changes for assumeutxoJames O'Beirne
When using an assumedvalid chainstate, only process validationinterface callbacks from the background chainstate within indexes. This ensures that all indexes are built in-order. Later, we can possibly designate indexes which can be built out of order and continue their operation during snapshot use. Once the background sync has completed, restart the indexes so that they continue to index the now-validated snapshot chainstate.
2023-09-30validation: pass ChainstateRole for validationinterface callsJames O'Beirne
This allows consumers to decide how to handle events from background or assumedvalid chainstates.
2023-09-15Return CAutoFile from BlockManager::Open*File()MarcoFalke
This is a refactor.
2023-09-12Remove unused GetType() from CBufferedFile and CAutoFileMarcoFalke
GetType() is only called in tests, so it is unused and can be removed.
2023-09-12Merge bitcoin/bitcoin#28427: index: coinstats reorg, fail when block cannot ↵fanquake
be reversed c0bf667912064960df194ea94150976b34f7c267 index: add [nodiscard] attribute to functions writing to the db (furszy) eef595560e9ecf3a0d1db4d8ea7ecc33a49d839f index: coinstats reorg, fail when block cannot be reversed (furszy) Pull request description: Found it while reviewing https://github.com/bitcoin/bitcoin/pull/24230#discussion_r1310863359. During a reorg, continuing execution when a block cannot be reversed leaves the coinstats index in an inconsistent state. This was surely overlooked when 'CustomRewind' was implemented. ACKs for top commit: ryanofsky: Code review ACK c0bf667912064960df194ea94150976b34f7c267. Only change since last review is new commit adding [[nodiscard]] Tree-SHA512: f4fc8522508d23e4fff09a29c935971819b1bd3b2a260e08e2e2b72f9340980d74fbec742a58fe216baf61d27de057c7c8300e8fa075f8507cd1227f128af909
2023-09-08refactor: remove clientversion include from dbwrapper.hCory Fields
2023-09-08index: add [nodiscard] attribute to functions writing to the dbfurszy
2023-09-07index: coinstats reorg, fail when block cannot be reversedfurszy
During a reorg, continuing execution when a block cannot be reversed leaves the coinstats index in an inconsistent state, which was surely overlooked when 'CustomRewind' was implemented.
2023-08-28Replace READWRITEAS macro with AsBase wrapping functionMarcoFalke
Co-authored-by: Pieter Wuille <pieter@wuille.net>
2023-08-17Remove unused includes from blockfilter.hMarcoFalke
This removes unused includes, primitives/block found manually, and the others by iwyu: blockfilter.h should remove these lines: - #include <serialize.h> // lines 16-16 - #include <undo.h> // lines 18-18
2023-08-07Merge bitcoin/bitcoin#28186: kernel: Prune leveldb headersfanquake
d8f1222ac50f089a0af29eaf8ce0555bad8366ef refactor: Correct dbwrapper key naming (TheCharlatan) be8f159ac59b9e700cbd3314ed71ebf39bd5b67a build: Remove leveldb from BITCOIN_INCLUDES (TheCharlatan) c95b37d641b1eed4a62d55ca5342a6ed8c7a1ce7 refactor: Move CDBWrapper leveldb members to their own context struct (TheCharlatan) c534a615e93452a5f509aaf5f68c600391a98d6a refactor: Split dbwrapper CDBWrapper::EstimateSize implementation (TheCharlatan) 586448888b72f7c87db4dcd30fc4e4044afae13b refactor: Move HandleError to dbwrapper implementation (TheCharlatan) dede0eef7adb7413f62f5abd68cac8e01635ba4a refactor: Split dbwrapper CDBWrapper::Exists implementation (TheCharlatan) a5c2eb57484314b04ec94523d14e0ef0c6c46d4f refactor: Fix logging.h includes (TheCharlatan) 84058e0eed9c05bc30984b39131e88ad1425628f refactor: Split dbwrapper CDBWrapper::Read implementation (TheCharlatan) e4af2408f2ac59788567b6fc8cb3a68fc43da9fe refactor: Pimpl leveldb::Iterator for CDBIterator (TheCharlatan) ef941ff1281e76308c3e746e592375bec023e9e4 refactor: Split dbwrapper CDBIterator::GetValue implementation (TheCharlatan) b7a1ab5cb4e60230f62c94efb3a10d07c9af4883 refactor: Split dbwrapper CDBIterator::GetKey implementation (TheCharlatan) d7437908cdf242626263ba9d5541addcddadc594 refactor: Split dbwrapper CDBIterator::Seek implementation (TheCharlatan) ea8135de7e617259cda3fc7b1c8e7569d454fd57 refactor: Pimpl leveldb::batch for CDBBatch (TheCharlatan) b9870c920dc475ec759eaf7339ea42aecba92138 refactor: Split dbwrapper CDBatch::Erase implementation (TheCharlatan) 532ee812a499e13b123af6b8415d8de1f3804f0f refactor: Split dbwrapper CDBBatch::Write implementation (TheCharlatan) afc534df9adbf5599b286b5dc3531a4b9ac2d056 refactor: Wrap DestroyDB in dbwrapper helper (TheCharlatan) Pull request description: Leveldb headers are currently included in the `dbwrapper.h` file and thus available to many of Bitcoin Core's source files. However, leveldb-specific functionality should be abstracted by the `dbwrapper` and does not need to be available to the rest of the code. Having leveldb included in a widely-used header such as `dbwrapper.h` bloats the entire project's header tree. The `dbwrapper` is a key component of the libbitcoinkernel library. Future users of this library would not want to contend with having the leveldb headers exposed and potentially polluting their project's namespace. For these reasons, the leveldb headers are removed from the `dbwrapper` by moving leveldb-specific code to the implementation file and creating a [pimpl](https://en.cppreference.com/w/cpp/language/pimpl) where leveldb member variables are indispensable. As a final step, the leveldb include flags are removed from the `BITCOIN_INCLUDES` and moved to places where the dbwrapper is compiled. --- This pull request is part of the [libbitcoinkernel project](https://github.com/bitcoin/bitcoin/issues/27587), and more specifically its stage 1 step 3 "Decouple most non-consensus headers from libbitcoinkernel". ACKs for top commit: stickies-v: re-ACK https://github.com/bitcoin/bitcoin/commit/d8f1222ac50f089a0af29eaf8ce0555bad8366ef MarcoFalke: ACK d8f1222ac50f089a0af29eaf8ce0555bad8366ef 🔠 Tree-SHA512: 0f58309be165af0162e648233451cd80fda88726fc10c0da7bfe4ec2ffa9afe63fbf7ffae9493698d3f39653b4ad870c372eee652ecc90ab1c29d86c387070f3
2023-08-05refactor: Fix logging.h includesTheCharlatan
These were uncovered as missing by the next commit.
2023-08-03lint: remove /* Continued */ markers from codebasefanquake
2023-07-10index: verify blocks data existence only oncefurszy
At present, during init, we traverse the chain (once per index) to confirm that all necessary blocks to sync each index up to the current tip are present. To make the process more efficient, we can fetch the oldest block from the indexers and perform the chain data existence check from that point only once. This also moves the pruning violation check to the end of the 'loadinit' thread, which is where the reindex, block loading and chain activation processes happen. Making the node's startup process faster, allowing us to remove the global g_indexes_ready_to_sync flag, and enabling the execution of the pruning violation verification even when the reindex or reindex-chainstate flags are enabled (which has being skipped so far).
2023-07-10init: don't start indexes sync thread prematurelyfurszy
By moving the 'StartIndexes()' call into the 'initload' thread, we can remove the threads active wait. Optimizing the available resources. The only difference with the current state is that now the indexes threads will only be started when they can process work and not before it.
2023-07-10refactor: simplify pruning violation checkfurszy
By generalizing 'GetFirstStoredBlock' and implementing 'CheckBlockDataAvailability' we can dedup code and avoid repeating work when multiple indexes are enabled. E.g. get the oldest block across all indexes and perform the pruning violation check from that point up to the tip only once (this feature is being introduced in a follow-up commit). This commit shouldn't change behavior in any way. Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
2023-07-10make GetFirstStoredBlock assert that 'start_block' always has datafurszy
And transfer the responsibility of verifying whether 'start_block' has data or not to the caller. This is because the 'GetFirstStoredBlock' function responsibility is to return the first block containing data. And the current implementation can return 'start_block' when it has no data!. Which is misleading at least. Edge case behavior change: Previously, if the block tip lacked data but all preceding blocks contained data, there was no prune violation. And now, such scenario will result in a prune violation.
2023-07-10refactor: index, decouple 'Init' from 'Start'furszy
So indexes can be initialized without spawning the sync thread. This makes asynchronous indexes startup possible in the following commits.
2023-06-28kernel: Add fatalError method to notificationsTheCharlatan
FatalError replaces what previously was the AbortNode function in shutdown.cpp. This commit is part of the libbitcoinkernel project and further removes the shutdown's and, more generally, the kernel library's dependency on interface_ui with a kernel notification method. By removing interface_ui from the kernel library, its dependency on boost is reduced to just boost::multi_index. At the same time it also takes a step towards de-globalising the interrupt infrastructure. Co-authored-by: Russell Yanofsky <russ@yanofsky.org> Co-authored-by: TheCharlatan <seb.kung@gmail.com>
2023-06-28scripted-diff: Rename FatalError to FatalErrorfTheCharlatan
This is done in preparation for the next commit where a new FatalError function is introduced. FatalErrorf follows common convention to append 'f' for functions accepting format arguments. -BEGIN VERIFY SCRIPT- sed -i 's/FatalError/FatalErrorf/g' $( git grep -l 'FatalError') -END VERIFY SCRIPT-
2023-06-16Remove the syscall sandboxfanquake
After initially being merged in #20487, it's no-longer clear that an internal syscall sandboxing mechanism is something that Bitcoin Core should have/maintain, especially when compared to better maintained/supported alterantives, i.e firejail. Note that given where it's used, the sandbox also gets dragged into the kernel. There is some related discussion in #24771. This should not require any sort of deprecation, as this was only ever an opt-in, experimental feature. Closes #24771.
2023-06-08refactor: index: use `AbortNode` in fatal error helperSebastian Falbesoner
Deduplicates code in the `FatalError` template function by using `AbortNode` which does the exact same thing if called without any user message (i.e. without second parameter specified). The template is still kept for ease-of-use w.r.t. not having to call `tfm::format(...)` at the call-side each time, and also to keep the diff minimal.
generated by cgit v1.2.3 (git 2.26.2) at 2024-10-17 18:29:56 +0000