aboutsummaryrefslogtreecommitdiff
path: root/src/streams.h
AgeCommit message (Collapse)Author
2024-09-16streams: remove AutoFile::Get() entirelyPieter Wuille
Co-Authored-By: David Gumberg <davidzgumberg@gmail.com> Github-Pull: bitcoin/bitcoin#30884 Rebased-From: a240e150e837b5a95ed19765a2e8b7c5b6013f35
2024-09-16streams: cache file position within AutoFilePieter Wuille
Github-Pull: bitcoin/bitcoin#30884 Rebased-From: e624a9bef16b6335fd119c10698352b59bf2930a
2024-07-08tidy: modernize-use-equals-defaultMarcoFalke
2024-04-01Add AutoFile::seek and tellAva Chow
It's useful to be able to seek to a specific position in a file. Allow AutoFile to seek by using fseek. It's also useful to be able to get the current position in a file. Allow AutoFile to tell by using ftell.
2023-11-30Remove unused CDataStreamMarcoFalke
2023-11-28refactor: SpanReader without nVersionMarcoFalke
The field is unused, so remove it. This is also required for future commits.
2023-11-28Merge bitcoin/bitcoin#28892: refactor: P2P transport without serialize ↵fanquake
version and type fa79a881ce0537e1d74da296a7513730438d2a02 refactor: P2P transport without serialize version and type (MarcoFalke) fa9b5f4fe32c0cfe2e477bb11912756f84a52cfe refactor: NetMsg::Make() without nVersion (MarcoFalke) 66669da4a5ca9edf2a40d20879d9a8aaf2b9e2ee Remove unused Make() overload in netmessagemaker.h (MarcoFalke) fa0ed0794161d937d2d3385963c1aa5624b60d17 refactor: VectorWriter without nVersion (MarcoFalke) Pull request description: Now that the serialize framework ignores the serialize version and serialize type, everything related to it can be removed from the code. This is the first step, removing dead code from the P2P stack. A different pull will remove it from the wallet and other parts. ACKs for top commit: ajtowns: reACK fa79a881ce0537e1d74da296a7513730438d2a02 Tree-SHA512: 785b413580d980f51f0d4f70ea5e0a99ce14cd12cb065393de2f5254891be94a14f4266110c8b87bd2dbc37467676655bce13bdb295ab139749fcd8b61bd5110
2023-11-18streams: Drop unused CAutoFileAnthony Towns
2023-11-18streams: Remove unused CAutoFile::GetVersionAnthony Towns
2023-11-18streams: Base BufferedFile on AutoFile instead of CAutoFileAnthony Towns
2023-11-17refactor: VectorWriter without nVersionMarcoFalke
The field is unused, so remove it. This is also required for future commits.
2023-11-14Drop OverrideStreamAnthony Towns
2023-11-13Merge bitcoin/bitcoin#28207: mempool: Persist with XORAndrew Chow
fa6b053b5c964fb35935fa994cb782c0731a56f8 mempool: persist with XOR (MarcoFalke) Pull request description: Currently the `mempool.dat` file stores data received from remote peers as-is. This may be problematic when a program other than Bitcoin Core tries to interpret them by accident. For example, an anti-virus program or other program may scan the file and move it into quarantine, or delete it, or corrupt it. While the local wallet is expected to re-submit any pending transactions, unrelated transactions may be missing from the mempool after a restart. This may cause fee estimates to be off, or may cause block relay to be slower. Fix this, similar to https://github.com/bitcoin/bitcoin/pull/6650, by rolling a random XOR pattern over the dat file when writing or reading it. Obviously this can only protect against programs that accidentally and unintentionally are trying to mess with the dat file. Any program that intentionally wants to mess with the dat file can still trivially do so. ACKs for top commit: achow101: re-ACK fa6b053b5c964fb35935fa994cb782c0731a56f8 glozow: reACK fa6b053b5c964fb35935fa994cb782c0731a56f8 ismaelsadeeq: ACK fa6b053b5c964fb35935fa994cb782c0731a56f8 Tree-SHA512: ded2ce3d81bc944b828263534e3178a1e45a914fe8e024f4a14c6561a73e301820944ecc75dd704b3d4221a7a3a5c0597ccab79546250c1197609ee981fe324e
2023-11-09mempool: persist with XORMarcoFalke
2023-10-20streams: Add SpanReader ignore methodRussell Yanofsky
Needed to deserialize some types from spans like CScripts
2023-10-02Merge bitcoin/bitcoin#28508: refactor: Remove SER_GETHASH, hard-code client ↵fanquake
version in CKeyPool serialize fac29a0ab19fda457b55d7a0a37c5cd3d9680f82 Remove SER_GETHASH, hard-code client version in CKeyPool serialize (MarcoFalke) fa72f09d6ff8ee204f331a69d3f5e825223c9e11 Remove CHashWriter type (MarcoFalke) fa4a9c0f4334678fb80358ead667807bf2a0a153 Remove unused GetType() from OverrideStream, CVectorWriter, SpanReader (MarcoFalke) Pull request description: Removes a bunch of redundant, dead or duplicate code. Uses the idea from and finishes the idea https://github.com/bitcoin/bitcoin/pull/28428 by theuni ACKs for top commit: ajtowns: ACK fac29a0ab19fda457b55d7a0a37c5cd3d9680f82 kevkevinpal: added one nit but otherwise ACK [fac29a0](https://github.com/bitcoin/bitcoin/pull/28508/commits/fac29a0ab19fda457b55d7a0a37c5cd3d9680f82) Tree-SHA512: cc805e2f38e73869a6691fdb5da09fa48524506b87fc93f05d32c336ad3033425a2d7608e317decd3141fde3f084403b8de280396c0c39132336fe0f7510af9e
2023-09-19Remove unused GetType() from OverrideStream, CVectorWriter, SpanReaderMarcoFalke
GetType() is never called, so it is completely unused and can be removed.
2023-09-15Make BufferedFile to be a CAutoFile wrapperMarcoFalke
This refactor allows to forward some calls to the underlying CAutoFile, instead of re-implementing the logic in the buffered file.
2023-09-15refactor: Drop unused fclose() from BufferedFileMarcoFalke
This was only explicitly used in the tests, where it can be replaced by wrapping the original raw file pointer into a CAutoFile on creation and then calling CAutoFile::fclose(). Also, it was used in LoadExternalBlockFile(), where it can also be replaced by the (implicit call to the) CAutoFile destructor after wrapping the original raw file pointer in a CAutoFile.
2023-09-12scripted-diff: Rename CBufferedFile to BufferedFileMarcoFalke
While touching all constructors in the previous commit, the class name can be adjusted to comply with the style guide. -BEGIN VERIFY SCRIPT- sed -i 's/CBufferedFile/BufferedFile/g' $( git grep -l CBufferedFile ) -END VERIFY SCRIPT-
2023-09-12Remove unused GetType() from CBufferedFile and CAutoFileMarcoFalke
GetType() is only called in tests, so it is unused and can be removed.
2023-07-19streams: Teach AutoFile how to XORMarcoFalke
2023-07-13Add AutoFile::detail_fread member functionMarcoFalke
New code can call the method without having first to retrieve the raw FILE* pointer via Get(). Also, move implementation to the cpp file. Can be reviewed with: --color-moved=dimmed-zebra --color-moved-ws=ignore-all-space
2023-07-13refactor: Modernize AutoFileMarcoFalke
* Add m_ prefix to the std::FILE member variable * Add std namespace where possible, to avoid confusion with member functions of the same name. * Add AutoFile::feof() member function, to be used in place of std::feof(AutoFile::Get()) * Simplify fclose() in terms of release() * Fix typo in the error message in the ignore member function.
2023-07-12doc: Remove comments that just repeat what the code doesMarcoFalke
No need to artificially bloat the code and waste space.
2023-07-12refactor: Remove redundant file check from AutoFile shift operatorsMarcoFalke
The shift operators will call the write or read member function, which already does the check. Also, call sites are free to directly call ::(Un)Serialize(s, obj) to skip this check, so removing it increases consistency.
2023-07-12Extract util::Xor, Add key_offset option, Add benchMarcoFalke
2023-06-01streams: Drop confusing DataStream::Serialize method and << operatorRyan Ofsky
DataStream Serialize method has surprising behavior because it just serializes raw bytes without a length prefix. When you serialize a string or vector, a length prefix is serialized before the raw object contents so the object can be unambiguously deserialized later. But DataStreams don't support deserializing at all and just dump the raw bytes. Having this inconsistency is not necessary and could be confusing (see https://github.com/bitcoin/bitcoin/pull/27790#discussion_r1212315030) so this PR just drops the DataStream::Serialize method.
2023-05-05util: improve streams.h:FindByte() performanceLarry Ruane
Avoid use of the expensive mod operator (%) when calculating the buffer offset. No functional difference. Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
2023-02-02Merge bitcoin/bitcoin#26992: refactor: Remove unused CDataStream ↵fanquake
SerializeMany constructor fa47b28dfc2a6577519e10da68ebd8da93568434 refactor: Remove unused CDataStream SerializeMany constructor (MarcoFalke) Pull request description: Seems odd to have an unused method. Moreover, the function is fragile and dangerous, because one could have a `std::vector vec_a` and type `CDataStream{vec_a, 0, 0}.size()` and `CDataStream{0, 0, vec_a}.size()`, assuming they are the same thing, when in fact they are not. (The first takes over the memory as is, the second serializes the vector). So my suggestion would be to remove the unused method and introduce a new method when this functionality is needed. For example: `static DataStream FromMany(Args&&... args)`. ACKs for top commit: stickies-v: ACK fa47b28dfc2a6577519e10da68ebd8da93568434 Tree-SHA512: 9593a034b997e33a0794f779f76f02425b1097b218cf8cb1facb7f874fa69da328ce567a79138015baeebe004ae7d103dda4f64f83e8ad375b6dae6b66d3d950
2023-01-31clang-tidy: Fix `modernize-use-default-member-init` in headersHennadii Stepanov
See https://clang.llvm.org/extra/clang-tidy/checks/modernize/use-default-member-init.html
2023-01-30refactor: Remove unused CDataStream SerializeMany constructorMarcoFalke
2023-01-26Remove unused CDataStream::SetTypeMarcoFalke
The last use was removed in the previous commit.
2023-01-24streams: Add DataStream without ser-type and ser-versionMarcoFalke
The moved parts can be reviewed with "--color-moved=dimmed-zebra". The one-char changes can be reviewed with "--word-diff-regex=.".
2022-12-24scripted-diff: Bump copyright headersHennadii Stepanov
-BEGIN VERIFY SCRIPT- ./contrib/devtools/copyright_header.py update ./ -END VERIFY SCRIPT- Commits of previous years: - 2021: f47dda2c58b5d8d623e0e7ff4e74bc352dfa83d7 - 2020: fa0074e2d82928016a43ca408717154a1c70a4db - 2019: aaaaad6ac95b402fe18d019d67897ced6b316ee0
2022-10-24util: add CBufferedFile::SkipTo() to move ahead in the streamLarry Ruane
SkipTo() reads data from the file into the CBufferedFile object (memory), but, unlike this object's read() method, SkipTo() doesn't transfer data into a caller's memory buffer. This is useful because after skipping forward in the stream in this way, the user can, if needed, rewind the stream (SetPos()) and access the object's memory buffer including ranges that were skipped over (without needing to read from the disk file).
2022-10-13Merge bitcoin/bitcoin#25667: assumeutxo: snapshot initializationAndrew Chow
bf9597606166323158bbf631137b82d41f39334f doc: add note about snapshot chainstate init (James O'Beirne) e4d799528696c5ede38c257afaffd367917e0de8 test: add testcases for snapshot initialization (James O'Beirne) cced4e7336d93a2dc88e4a61c49941887766bd72 test: move-only-ish: factor out LoadVerifyActivateChainstate() (James O'Beirne) 51fc9241c08a00f1f407f1534853a5cddbbc0a23 test: allow on-disk coins and block tree dbs in tests (James O'Beirne) 3c361391b8f5971eb3c7b620aa7ad9b437cc515e test: add reset_chainstate parameter for snapshot unittests (James O'Beirne) 00b357c215ed900145bd770525a341ba0ed9c027 validation: add ResetChainstates() (James O'Beirne) 3a29dfbfb2c16a50d854f6f81428a68aa9180509 move-only: test: make snapshot chainstate setup reusable (James O'Beirne) 8153bd9247dad3982d54488bcdb3960470315290 blockmanager: avoid undefined behavior during FlushBlockFile (James O'Beirne) ad67ff377c2b271cb4683da2fb25fd295557f731 validation: remove snapshot datadirs upon validation failure (James O'Beirne) 34d159033106cc595cfa852695610bfe419c989c add utilities for deleting on-disk leveldb data (James O'Beirne) 252abd1e8bc5cdf4368ad55e827a873240535b28 init: add utxo snapshot detection (James O'Beirne) f9f1735f139b6a1f1c7fea50717ff90dc4ba2bce validation: rename snapshot chainstate dir (James O'Beirne) d14bebf100aaaa25c7558eeed8b5c536da99885f db: add StoragePath to CDBWrapper/CCoinsViewDB (James O'Beirne) Pull request description: This is part of the [assumeutxo project](https://github.com/bitcoin/bitcoin/projects/11) (parent PR: https://github.com/bitcoin/bitcoin/pull/15606) --- Half of the replacement for #24232. The original PR grew larger than expected throughout the review process. This change adds the ability to initialize a snapshot-based chainstate during init if one is detected on disk. This is of course unused as of now (aside from in unittests) given that we haven't yet enabled actually loading snapshots. Don't be scared! There are some big move-only commits in here. Accompanying changes include: - moving the snapshot coinsdb directory from being called `chainstate_[base blockhash]` to `chainstate_snapshot`, since we only support one snapshot in use at a time. This simplifies some logic, but it necessitates writing that base blockhash out to a file within the coinsdb dir. See [discussion here](https://github.com/bitcoin/bitcoin/pull/24232#discussion_r832762880). - adding a simple fix in `FlushBlockFile()` that avoids a crash when attemping to flush to disk before `LoadBlockIndexDB()` is called, which happens when calling `MaybeRebalanceCaches()` during multiple chainstate init. - improving the unittest to allow testing with on-disk chainstates - necessary to test a simulated restart and re-initialization. ACKs for top commit: naumenkogs: utACK bf9597606166323158bbf631137b82d41f39334f ariard: Code Review ACK bf9597606 ryanofsky: Code review ACK bf9597606166323158bbf631137b82d41f39334f. Changes since last review: rebasing, switching from CAutoFile to AutoFile, adding comments, switching from BOOST_CHECK to Assert in test util, using chainman.GetMutex() in tests, destroying one ChainstateManager before creating a new one in tests fjahr: utACK bf9597606166323158bbf631137b82d41f39334f aureleoules: ACK bf9597606166323158bbf631137b82d41f39334f Tree-SHA512: 15ae75caf19f8d12a12d2647c52897904d27b265a7af6b4ae7b858592eeadb8f9da6c2394b6baebec90adc28742c053e3eb506119577dae7c1e722ebb3b7bcc0
2022-10-05Remove unused CDataStream::rdbuf methodMacroFake
It is unused and seems unlikely to be ever used.
2022-09-21refactor: use <cstdio> over stdio.hfanquake
We currently use both. Consolidate on the former.
2022-09-13validation: rename snapshot chainstate dirJames O'Beirne
This changes the snapshot's leveldb chainstate dir name from `chainstate_[blockhash]` to `chainstate_snapshot`. This simplifies later logic that loads snapshot data, and enforces the limitation of a single snapshot at any given time. Since we still need to persis the blockhash of the base block, we write that out to a file (`chainstate_snapshot/base_blockhash`) for later use during initialization, so that we can reinitialize the snapshot chainstate. Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
2022-06-29streams: Add AutoFile without ser-type and ser-versionMacroFake
The moved parts can be reviewed with "--color-moved=dimmed-zebra". The one-char changes can be reviewed with "--word-diff-regex=.".
2022-02-09scripted-diff: Rename nReadPos to m_read_pos in streams.hMarcoFalke
-BEGIN VERIFY SCRIPT- sed -i 's/nReadPos/m_read_pos/g' ./src/streams.h -END VERIFY SCRIPT-
2022-02-09Make CDataStream work properly on 64-bit systemsMarcoFalke
2022-02-09streams: Fix read-past-the-end and integer overflowsMarcoFalke
2022-02-03Remove broken and unused CDataStream methodsMarcoFalke
2022-01-02Use spans of std::byte in serializeMarcoFalke
This switches .read() and .write() to take spans of bytes.
2021-12-30scripted-diff: Bump copyright headersHennadii Stepanov
-BEGIN VERIFY SCRIPT- ./contrib/devtools/copyright_header.py update ./ -END VERIFY SCRIPT- Commits of previous years: * 2020: fa0074e2d82928016a43ca408717154a1c70a4db * 2019: aaaaad6ac95b402fe18d019d67897ced6b316ee0
2021-12-06Remove unused (and broken) functionality in SpanReaderPieter Wuille
This removes the ability to set an offset in the SpanReader constructor, as the current code is broken. All call sites use pos=0, so it is actually unused. If future call sites need it, SpanReader{a, b, c, d} is equivalent to SpanReader{a, b, c.subspan(d)}. It also removes the ability to deserialize from SpanReader directly from the constructor. This too is unused, and can be more idiomatically simulated using (SpanReader{a, b, c} >> x >> y >> z) instead of SpanReader{a, b, c, x, y, z}.
2021-12-02Generalize/simplify VectorReader into SpanReaderPieter Wuille
2021-11-09Use value_type in CDataStream where possibleMarcoFalke
Also, simplify unit tests with the CDataStream::str method.
generated by cgit v1.2.3 (git 2.26.2) at 2025-03-15 07:00:32 +0000