aboutsummaryrefslogtreecommitdiff
path: root/src/streams.h
AgeCommit message (Collapse)Author
2023-10-02Merge bitcoin/bitcoin#28508: refactor: Remove SER_GETHASH, hard-code client ↵fanquake
version in CKeyPool serialize fac29a0ab19fda457b55d7a0a37c5cd3d9680f82 Remove SER_GETHASH, hard-code client version in CKeyPool serialize (MarcoFalke) fa72f09d6ff8ee204f331a69d3f5e825223c9e11 Remove CHashWriter type (MarcoFalke) fa4a9c0f4334678fb80358ead667807bf2a0a153 Remove unused GetType() from OverrideStream, CVectorWriter, SpanReader (MarcoFalke) Pull request description: Removes a bunch of redundant, dead or duplicate code. Uses the idea from and finishes the idea https://github.com/bitcoin/bitcoin/pull/28428 by theuni ACKs for top commit: ajtowns: ACK fac29a0ab19fda457b55d7a0a37c5cd3d9680f82 kevkevinpal: added one nit but otherwise ACK [fac29a0](https://github.com/bitcoin/bitcoin/pull/28508/commits/fac29a0ab19fda457b55d7a0a37c5cd3d9680f82) Tree-SHA512: cc805e2f38e73869a6691fdb5da09fa48524506b87fc93f05d32c336ad3033425a2d7608e317decd3141fde3f084403b8de280396c0c39132336fe0f7510af9e
2023-09-19Remove unused GetType() from OverrideStream, CVectorWriter, SpanReaderMarcoFalke
GetType() is never called, so it is completely unused and can be removed.
2023-09-15Make BufferedFile to be a CAutoFile wrapperMarcoFalke
This refactor allows to forward some calls to the underlying CAutoFile, instead of re-implementing the logic in the buffered file.
2023-09-15refactor: Drop unused fclose() from BufferedFileMarcoFalke
This was only explicitly used in the tests, where it can be replaced by wrapping the original raw file pointer into a CAutoFile on creation and then calling CAutoFile::fclose(). Also, it was used in LoadExternalBlockFile(), where it can also be replaced by the (implicit call to the) CAutoFile destructor after wrapping the original raw file pointer in a CAutoFile.
2023-09-12scripted-diff: Rename CBufferedFile to BufferedFileMarcoFalke
While touching all constructors in the previous commit, the class name can be adjusted to comply with the style guide. -BEGIN VERIFY SCRIPT- sed -i 's/CBufferedFile/BufferedFile/g' $( git grep -l CBufferedFile ) -END VERIFY SCRIPT-
2023-09-12Remove unused GetType() from CBufferedFile and CAutoFileMarcoFalke
GetType() is only called in tests, so it is unused and can be removed.
2023-07-19streams: Teach AutoFile how to XORMarcoFalke
2023-07-13Add AutoFile::detail_fread member functionMarcoFalke
New code can call the method without having first to retrieve the raw FILE* pointer via Get(). Also, move implementation to the cpp file. Can be reviewed with: --color-moved=dimmed-zebra --color-moved-ws=ignore-all-space
2023-07-13refactor: Modernize AutoFileMarcoFalke
* Add m_ prefix to the std::FILE member variable * Add std namespace where possible, to avoid confusion with member functions of the same name. * Add AutoFile::feof() member function, to be used in place of std::feof(AutoFile::Get()) * Simplify fclose() in terms of release() * Fix typo in the error message in the ignore member function.
2023-07-12doc: Remove comments that just repeat what the code doesMarcoFalke
No need to artificially bloat the code and waste space.
2023-07-12refactor: Remove redundant file check from AutoFile shift operatorsMarcoFalke
The shift operators will call the write or read member function, which already does the check. Also, call sites are free to directly call ::(Un)Serialize(s, obj) to skip this check, so removing it increases consistency.
2023-07-12Extract util::Xor, Add key_offset option, Add benchMarcoFalke
2023-06-01streams: Drop confusing DataStream::Serialize method and << operatorRyan Ofsky
DataStream Serialize method has surprising behavior because it just serializes raw bytes without a length prefix. When you serialize a string or vector, a length prefix is serialized before the raw object contents so the object can be unambiguously deserialized later. But DataStreams don't support deserializing at all and just dump the raw bytes. Having this inconsistency is not necessary and could be confusing (see https://github.com/bitcoin/bitcoin/pull/27790#discussion_r1212315030) so this PR just drops the DataStream::Serialize method.
2023-05-05util: improve streams.h:FindByte() performanceLarry Ruane
Avoid use of the expensive mod operator (%) when calculating the buffer offset. No functional difference. Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
2023-02-02Merge bitcoin/bitcoin#26992: refactor: Remove unused CDataStream ↵fanquake
SerializeMany constructor fa47b28dfc2a6577519e10da68ebd8da93568434 refactor: Remove unused CDataStream SerializeMany constructor (MarcoFalke) Pull request description: Seems odd to have an unused method. Moreover, the function is fragile and dangerous, because one could have a `std::vector vec_a` and type `CDataStream{vec_a, 0, 0}.size()` and `CDataStream{0, 0, vec_a}.size()`, assuming they are the same thing, when in fact they are not. (The first takes over the memory as is, the second serializes the vector). So my suggestion would be to remove the unused method and introduce a new method when this functionality is needed. For example: `static DataStream FromMany(Args&&... args)`. ACKs for top commit: stickies-v: ACK fa47b28dfc2a6577519e10da68ebd8da93568434 Tree-SHA512: 9593a034b997e33a0794f779f76f02425b1097b218cf8cb1facb7f874fa69da328ce567a79138015baeebe004ae7d103dda4f64f83e8ad375b6dae6b66d3d950
2023-01-31clang-tidy: Fix `modernize-use-default-member-init` in headersHennadii Stepanov
See https://clang.llvm.org/extra/clang-tidy/checks/modernize/use-default-member-init.html
2023-01-30refactor: Remove unused CDataStream SerializeMany constructorMarcoFalke
2023-01-26Remove unused CDataStream::SetTypeMarcoFalke
The last use was removed in the previous commit.
2023-01-24streams: Add DataStream without ser-type and ser-versionMarcoFalke
The moved parts can be reviewed with "--color-moved=dimmed-zebra". The one-char changes can be reviewed with "--word-diff-regex=.".
2022-12-24scripted-diff: Bump copyright headersHennadii Stepanov
-BEGIN VERIFY SCRIPT- ./contrib/devtools/copyright_header.py update ./ -END VERIFY SCRIPT- Commits of previous years: - 2021: f47dda2c58b5d8d623e0e7ff4e74bc352dfa83d7 - 2020: fa0074e2d82928016a43ca408717154a1c70a4db - 2019: aaaaad6ac95b402fe18d019d67897ced6b316ee0
2022-10-24util: add CBufferedFile::SkipTo() to move ahead in the streamLarry Ruane
SkipTo() reads data from the file into the CBufferedFile object (memory), but, unlike this object's read() method, SkipTo() doesn't transfer data into a caller's memory buffer. This is useful because after skipping forward in the stream in this way, the user can, if needed, rewind the stream (SetPos()) and access the object's memory buffer including ranges that were skipped over (without needing to read from the disk file).
2022-10-13Merge bitcoin/bitcoin#25667: assumeutxo: snapshot initializationAndrew Chow
bf9597606166323158bbf631137b82d41f39334f doc: add note about snapshot chainstate init (James O'Beirne) e4d799528696c5ede38c257afaffd367917e0de8 test: add testcases for snapshot initialization (James O'Beirne) cced4e7336d93a2dc88e4a61c49941887766bd72 test: move-only-ish: factor out LoadVerifyActivateChainstate() (James O'Beirne) 51fc9241c08a00f1f407f1534853a5cddbbc0a23 test: allow on-disk coins and block tree dbs in tests (James O'Beirne) 3c361391b8f5971eb3c7b620aa7ad9b437cc515e test: add reset_chainstate parameter for snapshot unittests (James O'Beirne) 00b357c215ed900145bd770525a341ba0ed9c027 validation: add ResetChainstates() (James O'Beirne) 3a29dfbfb2c16a50d854f6f81428a68aa9180509 move-only: test: make snapshot chainstate setup reusable (James O'Beirne) 8153bd9247dad3982d54488bcdb3960470315290 blockmanager: avoid undefined behavior during FlushBlockFile (James O'Beirne) ad67ff377c2b271cb4683da2fb25fd295557f731 validation: remove snapshot datadirs upon validation failure (James O'Beirne) 34d159033106cc595cfa852695610bfe419c989c add utilities for deleting on-disk leveldb data (James O'Beirne) 252abd1e8bc5cdf4368ad55e827a873240535b28 init: add utxo snapshot detection (James O'Beirne) f9f1735f139b6a1f1c7fea50717ff90dc4ba2bce validation: rename snapshot chainstate dir (James O'Beirne) d14bebf100aaaa25c7558eeed8b5c536da99885f db: add StoragePath to CDBWrapper/CCoinsViewDB (James O'Beirne) Pull request description: This is part of the [assumeutxo project](https://github.com/bitcoin/bitcoin/projects/11) (parent PR: https://github.com/bitcoin/bitcoin/pull/15606) --- Half of the replacement for #24232. The original PR grew larger than expected throughout the review process. This change adds the ability to initialize a snapshot-based chainstate during init if one is detected on disk. This is of course unused as of now (aside from in unittests) given that we haven't yet enabled actually loading snapshots. Don't be scared! There are some big move-only commits in here. Accompanying changes include: - moving the snapshot coinsdb directory from being called `chainstate_[base blockhash]` to `chainstate_snapshot`, since we only support one snapshot in use at a time. This simplifies some logic, but it necessitates writing that base blockhash out to a file within the coinsdb dir. See [discussion here](https://github.com/bitcoin/bitcoin/pull/24232#discussion_r832762880). - adding a simple fix in `FlushBlockFile()` that avoids a crash when attemping to flush to disk before `LoadBlockIndexDB()` is called, which happens when calling `MaybeRebalanceCaches()` during multiple chainstate init. - improving the unittest to allow testing with on-disk chainstates - necessary to test a simulated restart and re-initialization. ACKs for top commit: naumenkogs: utACK bf9597606166323158bbf631137b82d41f39334f ariard: Code Review ACK bf9597606 ryanofsky: Code review ACK bf9597606166323158bbf631137b82d41f39334f. Changes since last review: rebasing, switching from CAutoFile to AutoFile, adding comments, switching from BOOST_CHECK to Assert in test util, using chainman.GetMutex() in tests, destroying one ChainstateManager before creating a new one in tests fjahr: utACK bf9597606166323158bbf631137b82d41f39334f aureleoules: ACK bf9597606166323158bbf631137b82d41f39334f Tree-SHA512: 15ae75caf19f8d12a12d2647c52897904d27b265a7af6b4ae7b858592eeadb8f9da6c2394b6baebec90adc28742c053e3eb506119577dae7c1e722ebb3b7bcc0
2022-10-05Remove unused CDataStream::rdbuf methodMacroFake
It is unused and seems unlikely to be ever used.
2022-09-21refactor: use <cstdio> over stdio.hfanquake
We currently use both. Consolidate on the former.
2022-09-13validation: rename snapshot chainstate dirJames O'Beirne
This changes the snapshot's leveldb chainstate dir name from `chainstate_[blockhash]` to `chainstate_snapshot`. This simplifies later logic that loads snapshot data, and enforces the limitation of a single snapshot at any given time. Since we still need to persis the blockhash of the base block, we write that out to a file (`chainstate_snapshot/base_blockhash`) for later use during initialization, so that we can reinitialize the snapshot chainstate. Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
2022-06-29streams: Add AutoFile without ser-type and ser-versionMacroFake
The moved parts can be reviewed with "--color-moved=dimmed-zebra". The one-char changes can be reviewed with "--word-diff-regex=.".
2022-02-09scripted-diff: Rename nReadPos to m_read_pos in streams.hMarcoFalke
-BEGIN VERIFY SCRIPT- sed -i 's/nReadPos/m_read_pos/g' ./src/streams.h -END VERIFY SCRIPT-
2022-02-09Make CDataStream work properly on 64-bit systemsMarcoFalke
2022-02-09streams: Fix read-past-the-end and integer overflowsMarcoFalke
2022-02-03Remove broken and unused CDataStream methodsMarcoFalke
2022-01-02Use spans of std::byte in serializeMarcoFalke
This switches .read() and .write() to take spans of bytes.
2021-12-30scripted-diff: Bump copyright headersHennadii Stepanov
-BEGIN VERIFY SCRIPT- ./contrib/devtools/copyright_header.py update ./ -END VERIFY SCRIPT- Commits of previous years: * 2020: fa0074e2d82928016a43ca408717154a1c70a4db * 2019: aaaaad6ac95b402fe18d019d67897ced6b316ee0
2021-12-06Remove unused (and broken) functionality in SpanReaderPieter Wuille
This removes the ability to set an offset in the SpanReader constructor, as the current code is broken. All call sites use pos=0, so it is actually unused. If future call sites need it, SpanReader{a, b, c, d} is equivalent to SpanReader{a, b, c.subspan(d)}. It also removes the ability to deserialize from SpanReader directly from the constructor. This too is unused, and can be more idiomatically simulated using (SpanReader{a, b, c} >> x >> y >> z) instead of SpanReader{a, b, c, x, y, z}.
2021-12-02Generalize/simplify VectorReader into SpanReaderPieter Wuille
2021-11-09Use value_type in CDataStream where possibleMarcoFalke
Also, simplify unit tests with the CDataStream::str method.
2021-05-05streams: Accept URef obj for VectorReader unserializeMarcoFalke
2021-02-01Merge #20464: refactor: Treat CDataStream bytes as uint8_tWladimir J. van der Laan
fa292724598c273867bc6dbf311f1440fe2541ba Remove redundant MakeUCharSpan wrappers (MarcoFalke) faf4aa2f47c0de4f3a0c5f5fe5b3ec32f611eefd Remove CDataStream::Init in favor of C++11 member initialization (MarcoFalke) fada14b948cac147198e3b685b5dd8cb72dc2911 Treat CDataStream bytes as uint8_t (MarcoFalke) fa8bdb048e65cae2d26bea3f991717a856e2fb39 refactor: Drop CDataStream constructors in favor of one taking a Span of bytes (MarcoFalke) faa96f841fe45bc49ebb6e07ac82a129fa9c40bf Remove unused CDataStream methods (MarcoFalke) Pull request description: Using `uint8_t` for raw bytes has a style benefit: * The signedness is clear from reading the code, as it does not depend on the architecture Other clean-ups in this pull include: * Remove unused methods * Constructor is simplified with `Span` * Remove `Init()` member in favor of C++11 member initialization ACKs for top commit: laanwj: code review ACK fa292724598c273867bc6dbf311f1440fe2541ba theStack: ACK fa292724598c273867bc6dbf311f1440fe2541ba 🍾 Tree-SHA512: 931ee28bd99843d7e894b48e90e1187ffb0278677c267044b3c0c255069d9bbd9298ab2e539b1002a30b543d240450eaec718ef4ee95a7fd4be0a295e926343f
2020-12-31scripted-diff: Bump copyright headersMarcoFalke
-BEGIN VERIFY SCRIPT- ./contrib/devtools/copyright_header.py update ./ -END VERIFY SCRIPT-
2020-12-31Remove CDataStream::Init in favor of C++11 member initializationMarcoFalke
2020-11-23Treat CDataStream bytes as uint8_tMarcoFalke
Also, rename CSerializeData to SerializeData
2020-11-23refactor: Drop CDataStream constructors in favor of one taking a Span of bytesMarcoFalke
2020-11-23Remove unused CDataStream methodsMarcoFalke
2020-10-09net: CAddress & CAddrMan: (un)serialize as ADDRv2Vasil Dimov
Change the serialization of `CAddrMan` to serialize its addresses in ADDRv2/BIP155 format by default. Introduce a new `CAddrMan` format version (3). Add support for ADDRv2 format in `CAddress` (un)serialization. Co-authored-by: Carl Dong <contact@carldong.me>
2020-07-26refactor: Drop unused CBufferedFile::Seek()Hennadii Stepanov
2019-12-30scripted-diff: Bump copyright of files changed in 2019MarcoFalke
-BEGIN VERIFY SCRIPT- ./contrib/devtools/copyright_header.py update ./ -END VERIFY SCRIPT-
2019-10-15Remove unused includespracticalswift
2019-09-10util: CBufferedFile fixesLarry Ruane
2019-01-25Trivial: Doxygenize existing CBufferedFile and VectorReader commentsBen Woosley
2019-01-05Merge #14357: streams: Fix broken streams_vector_reader test. Remove unused ↵MarcoFalke
seek(size_t). 4f4993fe2a Remove UBSan suppression (practicalswift) 958e1a307e streams: Remove unused seek(size_t) (practicalswift) Pull request description: Fix broken `streams_vector_reader` test. Remove unused `seek(size_t)`. Before this change the test `streams_vector_reader` triggered an unintended unsigned integer wraparound. It tried so seek using a negative value in `reader.seek(-6)`. Changes in this PR: * Fix broken `VectorReader::seek(size_t)` test case * Remove unused `seek(size_t)` Tree-SHA512: 6c6affd680626363eef9e496748f2f86a522325abab9d6b13161f41125cdc29ceb36c2c1509c90b8ff108d606df7629e55e094cc2b6253b05a892b81ce176b71
2019-01-04Merge #14855: test: Correct ineffectual WithOrVersion from transactions_testsMarcoFalke
75778a0724 test: Correct ineffectual WithOrVersion from transactions_tests (Ben Woosley) Pull request description: `WithOrVersion` uses `|` to combine the versions, and `|` with 0 is a no-op. NicolasDorier / sipa do you recall why the version is being overridden here? Introduced in ab48c5e72156b34300db4a6521cb3c9969be3937 Last updated 81e3228fcb33e8ed32d8b9fbe917444ba080073a Tree-SHA512: 2aea925497bab2da973f17752410a6759d67181a57c3b12a685d184fbfcca2984c45b702ab0bd641d75e086696a0424f1bf77c5578ca765d6882dc03b42d5f9a
generated by cgit v1.2.3 (git 2.26.2) at 2024-09-13 16:12:36 +0000