aboutsummaryrefslogtreecommitdiff
path: root/src/wallet/bdb.cpp
AgeCommit message (Collapse)Author
2024-02-12wallet: bdb batch 'ErasePrefix', do not create txn internallyfurszy
Transactions are intended to be started on upper layers rather than internally by the bdb batch object. This enables us to consolidate different write operations within a procedure in the same db txn, improving consistency due to the atomic property of the transaction, as well as its performance due to the reduction of disk write operations. Important Note: This approach also ensures that the BerkeleyBatch::ErasePrefix function behaves exactly as the SQLiteBatch::ErasePrefix function, which does not create a db txn internally. Furthermore, since the `BerkeleyBatch::ErasePrefix' implementation erases records one by one (by traversing the db), this change ensures that the function is always called within an active txn context. Without this measure, there's a potential risk to consistency; certain records may be removed while others could persist due to an internal failure during the procedure.
2023-10-26refactor: Return enum in LockDirectoryMarcoFalke
This makes it easier to add more Error cases in the future. Also, add missing util namespace.
2023-07-07wallet: bdb: include bdb header from our implementation files onlyCory Fields
This way the dependency can't sneak into other files without being noticed. Forward-declare bdb classes as necessary.
2023-07-07wallet: bdb: don't use bdb define in headerCory Fields
2023-07-07wallet: bdb: move BerkeleyDatabase constructor to cpp fileCory Fields
Else some compilers/stdlibs may not be able to construct std::unique_ptr<Db> without Db defined.
2023-07-07wallet: bdb: move SafeDbt to cpp fileCory Fields
Dbt requires including bdb headers.
2023-07-07wallet: bdb: move SpanFromDbt to below SafeDbt's implementationCory Fields
No functional change, just simplifies the code move in the next commit.
2023-07-06wallet: bdb: move TxnBegin to cpp file since it uses a bdb functionCory Fields
2023-07-06wallet: bdb: drop default parameterCory Fields
2023-06-28refactor: Drop unsafe AsBytePtr functionRyan Ofsky
Replace calls to AsBytePtr with direct calls to AsBytes or reinterpret_cast. AsBytePtr is just a wrapper around reinterpret_cast. It accepts any type of pointer as an argument and uses reinterpret_cast to cast the argument to a std::byte pointer. Despite taking any type of pointer as an argument, it is not useful to call AsBytePtr on most types of pointers, because byte representations of most types will be implmentation-specific. Also, because it is named similarly to the AsBytes function, AsBytePtr looks safer than it actually is. Both AsBytes and AsBytePtr call reinterpret_cast internally and may be unsafe to use with certain types, but AsBytes at least has some type checking and can only be called on Span objects, while AsBytePtr can be called on any pointer argument. Co-authored-by: Pieter Wuille <pieter@wuille.net>
2023-06-01wallet: Add GetPrefixCursor to DatabaseBatchAndrew Chow
In order to get records beginning with a prefix, we will need a cursor specifically for that prefix. So add a GetPrefixCursor function and DatabaseCursor classes for dealing with those prefixes. Tested on each supported db engine. 1) Write two different key->value elements to db. 2) Create a new prefix cursor and walk-through every returned element, verifying that it gets parsed properly. 3) Try to move the cursor outside the filtered range: expect failure and flag complete=true. Co-Authored-By: Ryan Ofsky <ryan@ofsky.org> Co-Authored-By: furszy <matiasfurszyfer@protonmail.com>
2023-05-31walletdb: Handle when database keys are emptyRyan Ofsky
2023-05-31walletdb: Consistently clear key and value streams before writingAndrew Chow
Before writing data to the output key and value streams, make sure they are cleared.
2023-05-15wallet: bugfix, GetNewCursor() misses to provide batch ptr to BerkeleyCursorfurszy
If the batch ptr is not passed, the cursor will not use the db active txn context which could lead to a deadlock if the code tries to modify the db while it is traversing it. E.g. the 'EraseRecords()' function.
2023-05-01Merge bitcoin/bitcoin#27224: refactor: Remove CAddressBookData::destdataAndrew Chow
a5986e82dd2b8f923d60f9e31760ef62b9010105 refactor: Remove CAddressBookData::destdata (Ryan Ofsky) 5938ad0bdb013953861c7cd15a95f00998a06f44 wallet: Add DatabaseBatch::ErasePrefix method (Ryan Ofsky) Pull request description: This is cleanup that doesn't change external behavior. Benefits of the cleanup are: - Removes awkward `StringMap` intermediate representation for wallet address metadata. - Simplifies `CWallet`, deals with used address and received request serialization in `walletdb.cpp` instead of higher level wallet code - Adds test coverage and documentation This PR doesn't change externally observable behavior. Internally, the only change in behavior is that `EraseDestData` deletes rows directly from the database because they are no longer stored in memory. This is more direct and efficient because it uses a single lookup and scan instead of multiple lookups. Motivation for this cleanup is making changes like #18550, #18192, #13756 easier to reason about and less likely to result in unintended behavior and bugs --- This PR is a rebased copy of #18608. For some reason that PR is locked and couldn't be reopened or commented on. This PR is an alternative to #27215 with differences described in https://github.com/bitcoin/bitcoin/pull/27215#pullrequestreview-1329028143 ACKs for top commit: achow101: ACK a5986e82dd2b8f923d60f9e31760ef62b9010105 furszy: Code ACK a5986e82 Tree-SHA512: 6bd3e402f1f60263fbd433760bdc29d04175ddaf8307207c4a67d59f6cffa732e176ba57886e02926f7a1615dce0ed9cda36c8cbc6430aa8e5b56934c23f3fe7
2023-04-21Merge bitcoin/bitcoin#27419: move-only: Extract common/args from util/systemfanquake
be55f545d53d44fdcf2d8ae802e9eae551d120c6 move-only: Extract common/args and common/config.cpp from util/system (TheCharlatan) Pull request description: This pull request is part of the `libbitcoinkernel` project https://github.com/bitcoin/bitcoin/issues/24303 https://github.com/bitcoin/bitcoin/projects/18 and more specifically its "Step 2: Decouple most non-consensus code from libbitcoinkernel". It is part of a series of patches splitting up the `util/system` files. Its preceding pull request is https://github.com/bitcoin/bitcoin/pull/27254. The pull request contains an extraction of ArgsManager related functions from util/system into their own common/ file. The background of this commit is an ongoing effort to decouple the libbitcoinkernel library from the ArgsManager. The ArgsManager belongs into the common library, since the kernel library should not depend on it. See [doc/design/libraries.md](https://github.com/bitcoin/bitcoin/blob/master/doc/design/libraries.md) for more information on this rationale. ACKs for top commit: MarcoFalke: re-ACK be55f545d53d44fdcf2d8ae802e9eae551d120c6 🚲 ryanofsky: Code review ACK be55f545d53d44fdcf2d8ae802e9eae551d120c6. Just small cleanups since the last review. hebasto: ACK be55f545d53d44fdcf2d8ae802e9eae551d120c6, I have reviewed the code and it looks OK, I agree it can be merged. Tree-SHA512: 90eb03334af0155b823030b4f2ecf286d35058d700ee2ddbbaa445be19e31eb0fe982656f35bd14ecee3ad2c3d0db3746855cb8f3777eff7253713e42873e111
2023-04-19move-only: Extract common/args and common/config.cpp from util/systemTheCharlatan
This is an extraction of ArgsManager related functions from util/system into their own common file. Config file related functions are moved to common/config.cpp. The background of this commit is an ongoing effort to decouple the libbitcoinkernel library from the ArgsManager. The ArgsManager belongs into the common library, since the kernel library should not depend on it. See doc/design/libraries.md for more information on this rationale.
2023-04-12wallet: Add DatabaseBatch::ErasePrefix methodRyan Ofsky
This new function is not used yet this commit, but next commit adds usages and test coverage for both BDB and sqlite.
2023-04-03refactor: don't avoid sys/types.h on when building for Windowsfanquake
We've already used it unguarded in `httpserver.cpp` for years, with no build issues.
2023-04-03Merge bitcoin/bitcoin#27254: refactor: Extract util/fs from util/systemfanquake
00e9b97f37e0bdf4c647236838c10b68b7ad5be3 refactor: Move fs.* to util/fs.* (TheCharlatan) 106b46d9d25b5228ef009fbbe6f9a7ae35090d15 Add missing fs.h includes (TheCharlatan) b202b3dd6393b415fa68e18dc49c9431dc6b58b2 Add missing cstddef include in assumptions.h (TheCharlatan) 18fb36367a28819bd5ab402344802796a1248979 refactor: Extract util/fs_helpers from util/system (Ben Woosley) Pull request description: This pull request is part of the `libbitcoinkernel` project https://github.com/bitcoin/bitcoin/issues/24303 https://github.com/bitcoin/bitcoin/projects/18 and more specifically its "Step 2: Decouple most non-consensus code from libbitcoinkernel". This commit was originally authored by empact and is taken from its parent PR #25152. #### Context There is an ongoing effort to decouple the `ArgsManager` used for command line parsing user-provided arguments from the libbitcoinkernel library (https://github.com/bitcoin/bitcoin/pull/25290, https://github.com/bitcoin/bitcoin/pull/25487, https://github.com/bitcoin/bitcoin/pull/25527, https://github.com/bitcoin/bitcoin/pull/25862, https://github.com/bitcoin/bitcoin/pull/26177, and https://github.com/bitcoin/bitcoin/pull/27125). The `ArgsManager` is defined in `system.h`. A similar pull request extracting functionality from `system.h` has been merged in https://github.com/bitcoin/bitcoin/pull/27238. #### Changes Next to providing better code organization, this PR removes some reliance of the tree of libbitcoinkernel header includes on `system.h` (and thus the `ArgsManager` definition) by moving filesystem related functions out of the `system.*` files. There is already a pair of `fs.h` / `fs.cpp` in the top-level `src/` directory. They were not combined with the files introduced here, to keep the patch cleaner and more importantly because they are often included without the utility functions. The new files are therefore named `fs_helpers` and the existing `fs` files are moved into the util directory. Further commits splitting more functionality out of `system.h` are still in #25152 and will be submitted in separate PRs once this PR has been processed. ACKs for top commit: hebasto: ACK 00e9b97f37e0bdf4c647236838c10b68b7ad5be3 Tree-SHA512: 31422f148d14ba3c843b99b1550a6fd77c77f350905ca324f93d4f97b652246bc58fa9696c64d1201979cf88733e40be02d262739bb7d417cf22bf506fdb7666
2023-03-26clang-tidy: Add `performance-inefficient-vector-operation` checkHennadii Stepanov
https://clang.llvm.org/extra/clang-tidy/checks/performance/inefficient-vector-operation.html
2023-03-23refactor: Move fs.* to util/fs.*TheCharlatan
The fs.* files are already part of the libbitcoin_util library. With the introduction of the fs_helpers.* it makes sense to move fs.* into the util/ directory as well.
2023-03-23refactor: Extract util/fs_helpers from util/systemBen Woosley
This is an extraction of filesystem related functions from util/system into their own utility file. The background of this commit is an ongoing effort to decouple the libbitcoinkernel library from the ArgsManager defined in system.h. Moving these functions out of system.h allows including them from a separate source file without including the ArgsManager definitions from system.h.
2023-01-31clang-tidy: Fix `modernize-use-default-member-init` in headersHennadii Stepanov
See https://clang.llvm.org/extra/clang-tidy/checks/modernize/use-default-member-init.html
2023-01-26Merge bitcoin/bitcoin#25296: Add DataStream without ser-type and ser-version ↵fanquake
and use it where possible fa035fe2d61d0c98d1bfd0153a0c3b5eb9d40de4 Remove unused CDataStream::SetType (MarcoFalke) fa29e73cdab82f98682821322cda89b1084ba887 Use DataStream where possible (MarcoFalke) fa9becfe1cea5040e7cea36324d1b0789cbbd25d streams: Add DataStream without ser-type and ser-version (MarcoFalke) Pull request description: This was done in the context of https://github.com/bitcoin/bitcoin/pull/25284 , but I think it also makes sense standalone. The basic idea is that serialization type should not be initialized when it is not needed. Same for the serialization version. So do this here for `DataStream`. `CDataStream` remains in places where it is not yet possible. ACKs for top commit: stickies-v: re-ACK [fa035fe](https://github.com/bitcoin/bitcoin/commit/fa035fe2d61d0c98d1bfd0153a0c3b5eb9d40de4) aureleoules: diff re-ACK fa035fe2d61d0c98d1bfd0153a0c3b5eb9d40de4 https://github.com/bitcoin/bitcoin/compare/fa0e6640bac8c6426af7c5744125c85c0f74b9e5..fa035fe2d61d0c98d1bfd0153a0c3b5eb9d40de4 Tree-SHA512: cb5e53d0df7c94319ffadc6ea1d887fc38516decaf43f0673396d79cc62d450a1a61173654a91b8c2b52d2cecea53fe4a500b8f6466596f35731471163fb051c
2023-01-26Use DataStream where possibleMarcoFalke
2023-01-24refactor: Remove c_str from util/checkMarcoFalke
2023-01-23Merge bitcoin/bitcoin#26690: wallet: Refactor database cursor into its own ↵fanquake
object with proper return codes 4aebd832a405090c2608e4b60bb4f34501bcea61 db: Change DatabaseCursor::Next to return status enum (Andrew Chow) d79e8dcf2981ef1964a2fde8c472b5de1ca1c963 wallet: Have cursor users use DatabaseCursor directly (Andrew Chow) 7a198bba0a1d0a0f0fd4ca947955cb52b84bdd4b wallet: Introduce DatabaseCursor RAII class for managing cursor (Andrew Chow) 69efbc011bb74fcd8dd9ed2a8a5d31bc9e323c10 Move SafeDbt out of BerkeleyBatch (Andrew Chow) Pull request description: Instead of having database cursors be tied to a particular `DatabaseBatch` object and requiring its setup and teardown be separate functions in that batch, we can have cursors be separate RAII classes. This makes it easier to create and destroy cursors as well as having cursors that have slightly different behaviors. Additionally, since reading data from a cursor is a tri-state, this PR changes the return value of the `Next` function (formerly `ReadAtCursor`) to return an Enum rather than the current system of 2 booleans. This greatly simplifies and unifies the code that deals with cursors as now there is no confusion as to what the function returns when there are no records left to be read. Extracted from #24914 ACKs for top commit: furszy: diff ACK 4aebd83 theStack: Code-review ACK 4aebd832a405090c2608e4b60bb4f34501bcea61 Tree-SHA512: 5d0be56a18de5b08c777dd5a73ba5a6ef1e696fdb07d1dca952a88ded07887b7c5c04342f9a76feb2f6fe24a45dc31f094f1f5d9500e6bdf4a44f4edb66dcaa1
2023-01-03refactor: use braced init for integer constants instead of c style castsPasta
2022-12-24scripted-diff: Bump copyright headersHennadii Stepanov
-BEGIN VERIFY SCRIPT- ./contrib/devtools/copyright_header.py update ./ -END VERIFY SCRIPT- Commits of previous years: - 2021: f47dda2c58b5d8d623e0e7ff4e74bc352dfa83d7 - 2020: fa0074e2d82928016a43ca408717154a1c70a4db - 2019: aaaaad6ac95b402fe18d019d67897ced6b316ee0
2022-12-16db: Change DatabaseCursor::Next to return status enumAndrew Chow
Next()'s result is a tri-state - failed, more to go, complete. Replace the way that this is returned with an enum with values FAIL, MORE, and DONE rather than with two booleans.
2022-12-16wallet: Have cursor users use DatabaseCursor directlyAndrew Chow
Instead of having the DatabaseBatch manage the cursor, having the consumer handle it directly
2022-12-14wallet: Introduce DatabaseCursor RAII class for managing cursorAndrew Chow
Instead of having DatabaseBatch deal with opening and closing database cursors, have a separate RAII class that deals with those. For now, DatabaseBatch manages DatabaseCursor, but this will change later.
2022-12-12Move SafeDbt out of BerkeleyBatchAndrew Chow
2022-09-16Merge bitcoin/bitcoin#25499: Use steady clock for all millis bench loggingfanquake
fa521c960337a65d4ce12cd1ef009c652ffe57e6 Use steady clock for all millis bench logging (MacroFake) Pull request description: Currently `GetTimeMillis` is used for bench logging in milliseconds integral precision. Replace it to use a steady clock that is type-safe and steady. Microsecond or float precision can be done in a follow-up. ACKs for top commit: fanquake: ACK fa521c960337a65d4ce12cd1ef009c652ffe57e6 - started making the same change. Tree-SHA512: 86a810e496fc663f815acb8771a6c770331593715cde85370226685bc50c13e8e987e3c5efd0b4e48b36ebd2372255357b709204bac750d41e94a9f7d9897fa6
2022-07-30Use steady clock for all millis bench loggingMacroFake
2022-07-27refactor: Make const refs vars where applicableAurèle Oulès
This avoids initializing variables with the copy-constructor of a non-trivially copyable type.
2022-07-20compat: document S_I* defines when building for Windowsfanquake
2022-06-16wallet: don't read db every time that a new WalletBatch is createdfurszy
Better to perform the action only one time (during 'LoadWallet'). Where the value is being used.
2022-05-12scripted-diff: replace non-standard fixed width integer types (`u_int`...` ↵Sebastian Falbesoner
-> `uint`...) -BEGIN VERIFY SCRIPT- sed -i 's/u_int/uint/g' $(git grep -l u_int) -END VERIFY SCRIPT-
2022-05-03Merge bitcoin/bitcoin#24470: Disallow more unsafe string->path conversions ↵MacroFake
allowed by path append operators f64aa9c411ad78259756a28756ec1eb8069b5ab4 Disallow more unsafe string->path conversions allowed by path append operators (Ryan Ofsky) Pull request description: Add more `fs::path` `operator/` and `operator+` overloads to prevent unsafe string->path conversions on Windows that would cause strings to be decoded according to the current Windows locale & code page instead of the correct string encoding. Update application code to deal with loss of implicit string->path conversions by calling `fs::u8path` or `fs::PathFromString` explicitly, or by just changing variable types from `std::string` to `fs::path` to avoid conversions altogether, or make them happen earlier. In all cases, there's no change in behavior either (1) because strings only contained ASCII characters and would be decoded the same regardless of what encoding was used, or (2) because of the 1:1 mapping between paths and strings using the `PathToString` and `PathFromString` functions. Motivation for this PR was just that I was experimenting with #24469 and noticed that operations like `fs::path / std::string` were allowed, and I thought it would be better not to allow them. ACKs for top commit: hebasto: ACK f64aa9c411ad78259756a28756ec1eb8069b5ab4 Tree-SHA512: 944cce49ed51537ee7a35ea4ea7f5feaf0c8fff2fa67ee81ec5adebfd3dcbaf41b73eb35e49973d5f852620367f13506fd12a7a9b5ae3a7a0007414d5c9df50f
2022-04-26scripted-diff: rename BytePtr to AsBytePtrJoão Barbosa
Building with iPhoneOS SDK fails because it also has `BytePtr` defined in /usr/include/MacTypes.h. -BEGIN VERIFY SCRIPT- sed -i 's/BytePtr/AsBytePtr/' $(git grep -l "BytePtr" src) -END VERIFY SCRIPT-
2022-04-21Disallow more unsafe string->path conversions allowed by path append operatorsRyan Ofsky
Add more fs::path operator/ and operator+ overloads to prevent unsafe string->path conversions on Windows that would cause strings to be decoded according to the current Windows locale & code page instead of the correct string encoding. Update application code to deal with loss of implicit string->path conversions by calling fs::u8path or fs::PathFromString explicitly, or by just changing variable types from std::string to fs::path to avoid conversions altoghther, or make them happen earlier. In all cases, there's no change in behavior either (1) because strings only contained ASCII characters and would be decoded the same regardless of what encoding was used, or (2) because of the 1:1 mapping between paths and strings using the PathToString and PathFromString functions. Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
2022-03-16Replace use of `ArgsManager` with `DatabaseOptions`Kiminuo
Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
2022-02-03refactor: replace boost::filesystem with std::filesystemKiminuo
Warning: Replacing fs::system_complete calls with fs::absolute calls in this commit may cause minor changes in behaviour because fs::absolute no longer strips trailing slashes; however these changes are believed to be safe. Co-authored-by: Russell Yanofsky <russ@yanofsky.org> Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
2022-01-27Merge bitcoin/bitcoin#23438: refactor: Use spans of std::byte in serializelaanwj
fa5d2e678c809c26bd40d7e7c171529d3ffb5903 Remove unused char serialize (MarcoFalke) fa24493d6394b3a477535f480664c9596f18e3c5 Use spans of std::byte in serialize (MarcoFalke) fa65bbf217b725ada35107b4ad646d250228355c span: Add BytePtr helper (MarcoFalke) Pull request description: This changes the serialize code (`.read()` and `.write()` functions) to take a `Span` instead of a pointer and size. This is a breaking change for the serialize interface, so at no additional cost we can also switch to `std::byte` (instead of using `char`). The benefits of using `Span`: * Less verbose and less fragile code when passing an already existing `Span`(-like) object to or from serialization The benefits of using `std::byte`: * `std::byte` can't accidentally be mistaken for an integer The goal here is to only change serialize to use spans of `std::byte`. If needed, `AsBytes`, `MakeUCharSpan`, ... can be used (temporarily) to pass spans of the right type. Other changes that are included here: * [#22167](https://github.com/bitcoin/bitcoin/pull/22167) (refactor: Remove char serialize by MarcoFalke) * [#21906](https://github.com/bitcoin/bitcoin/pull/21906) (Preserve const in cast on CTransactionSignatureSerializer by promag) ACKs for top commit: laanwj: Concept and code review ACK fa5d2e678c809c26bd40d7e7c171529d3ffb5903 sipa: re-utACK fa5d2e678c809c26bd40d7e7c171529d3ffb5903 Tree-SHA512: 08ee9eced5fb777cedae593b11e33660bed9a3e1711a7451a87b835089a96c99ce0632918bb4666a4e859c4d020f88fb50f2dd734216b0c3d1a9a704967ece6f
2022-01-06Add src/wallet/* code to wallet:: namespaceRussell Yanofsky
2022-01-02Use spans of std::byte in serializeMarcoFalke
This switches .read() and .write() to take spans of bytes.
2021-12-30scripted-diff: Bump copyright headersHennadii Stepanov
-BEGIN VERIFY SCRIPT- ./contrib/devtools/copyright_header.py update ./ -END VERIFY SCRIPT- Commits of previous years: * 2020: fa0074e2d82928016a43ca408717154a1c70a4db * 2019: aaaaad6ac95b402fe18d019d67897ced6b316ee0
2021-10-20wallet: Use PACKAGE_NAME to mention our softwareHennadii Stepanov
generated by cgit v1.2.3 (git 2.26.2) at 2024-10-06 02:39:51 +0000