aboutsummaryrefslogtreecommitdiff
path: root/src/serialize.h
AgeCommit message (Collapse)Author
2020-04-16scripted-diff: Bump copyright headersMarcoFalke
-BEGIN VERIFY SCRIPT- ./contrib/devtools/copyright_header.py update ./ -END VERIFY SCRIPT-
2020-03-05Merge #18112: Serialization improvements step 5 (blockencodings)Wladimir J. van der Laan
353f376277ad9b87e03c9ccbc1028c4b6d12e8ea Convert blockencodings.h to new serialization framework (Pieter Wuille) e574fff53eec4a27c83b765cb69e31d8399047ea Add CustomUintFormatter (Pieter Wuille) 10633398f2dddf929d3f535aa48d138ad5e6c50f Add DifferenceFormatter (Russell Yanofsky) 56dd9f04c701aa3ac340e95065bf83de20373c8b Make VectorFormatter support stateful formatters (Russell Yanofsky) 3ca574cef0b4423f21b2c3efd8f5c9f71d52f219 Convert CCompactSize to proper formatter (Pieter Wuille) Pull request description: This is probably the most involved change in the sequence of changes extracted from #10785. In order to implement the differential encoding of BIP152, this change changes `VectorFormatter` to permit a stateful sub-formatter, which is then used by `DifferenceFormatter`. A `CustomUintFormatter` is added as well to do the 48-bit serialization of short ids. ACKs for top commit: laanwj: ACK 353f376277ad9b87e03c9ccbc1028c4b6d12e8ea, nice change ryanofsky: Code review ACK 353f376277ad9b87e03c9ccbc1028c4b6d12e8ea. Only changes since last review are suggested assert change and MASK->MAX rename Tree-SHA512: 976618991a8be62ba0738725b7cfa166a56cde998ebf1031ba6f28557032f1577b666ac7ae25cd498c0e1e740108c3c56a342620b724df41d6cc9d8bdafac037
2020-02-25Add CustomUintFormatterPieter Wuille
2020-02-19Add static_asserts to ser_X_to_Y() methodsSamer Afach
2020-02-17Fix a violation of C++ standard rules that unions cannot be switched.Samer Afach
2020-02-15Make VectorFormatter support stateful formattersRussell Yanofsky
2020-02-14Convert CCompactSize to proper formatterPieter Wuille
2020-02-10Get rid of VARINT default argumentPieter Wuille
This removes the need for the GNU C++ extension of variadic macros.
2020-02-06Make std::vector and prevector reuse the VectorFormatter logicPieter Wuille
2020-02-06Add custom vector-element formatterPieter Wuille
This allows a very compact notation for serialization of vectors whose elements are not serialized using their default encoding.
2020-01-29Add a constant for the maximum vector allocation (5 Mbyte)Pieter Wuille
2020-01-18Add FORMATTER_METHODS, similar to SERIALIZE_METHODS, but for formattersPieter Wuille
2020-01-13Convert VARINT to the formatter/Using approachPieter Wuille
2020-01-13Add a generic approach for (de)serialization of objects using code in other ↵Pieter Wuille
classes This adds the (internal) Wrapper class, and the Using function that uses it. Given a class F that implements Ser(stream, const object&) and Unser(stream, object&) functions, this permits writing e.g. READWRITE(Using<F>(object)).
2020-01-02Introduce new serialization macros without castsPieter Wuille
This new approach uses a static method which takes the object as a argument. This has the advantage that its constness can be a template parameter, allowing a single implementation that sees the object as const for serialization and non-const for deserialization, without casts. More boilerplate is included in the new macro as well.
2019-12-30scripted-diff: Bump copyright of files changed in 2019MarcoFalke
-BEGIN VERIFY SCRIPT- ./contrib/devtools/copyright_header.py update ./ -END VERIFY SCRIPT-
2019-10-15Remove unused includespracticalswift
2019-08-26Support serialization of std::vector<bool>Pieter Wuille
2019-06-18Merge #12324: speed up Unserialize_impl for prevectorWladimir J. van der Laan
86b47fa741408b061ab0bda784b8678bfd7dfa88 speed up Unserialize_impl for prevector (Akio Nakamura) Pull request description: The unserializer for prevector uses `resize()` for reserve the area, but it's prefer to use `reserve()` because `resize()` have overhead to call its constructor many times. However, `reserve()` does not change the value of `_size` (a private member of prevector). This PR make the logic of read from stream to callback function, and prevector handles initilizing new values with that call-back and ajust the value of `_size`. The changes are as follows: 1. prevector.h Add a public member function named 'append'. This function has 2 params, number of elemenst to append and call-back function that initilizing new appended values. 2. serialize.h In the following two function: - `Unserialize_impl(Stream& is, prevector<N, T>& v, const unsigned char&)` - `Unserialize_impl(Stream& is, prevector<N, T>& v, const V&)` Make a callback function from each original logic of reading values from stream, and call prevector's `append()`. 3. test/prevector_tests.cpp Add a test for `append()`. ## A benchmark result is following: [Machine] MacBook Pro (macOS 10.13.3/i7 2.2GHz/mem 16GB/SSD) [result] DeserializeAndCheckBlockTest => 22% faster DeserializeBlockTest => 29% faster [before PR] # Benchmark, evals, iterations, total, min, max, median DeserializeAndCheckBlockTest, 60, 160, 94.4901, 0.0094644, 0.0104715, 0.0098339 DeserializeBlockTest, 60, 130, 65.0964, 0.00800362, 0.00895134, 0.00824187 [After PR] # Benchmark, evals, iterations, total, min, max, median DeserializeAndCheckBlockTest, 60, 160, 77.1597, 0.00767013, 0.00858959, 0.00805757 DeserializeBlockTest, 60, 130, 49.9443, 0.00613926, 0.00691187, 0.00635527 ACKs for top commit: laanwj: utACK 86b47fa741408b061ab0bda784b8678bfd7dfa88 Tree-SHA512: 62ea121ccd45a306fefc67485a1b03a853435af762607dae2426a87b15a3033d802c8556e1923727ddd1023a1837d0e5f6720c2c77b38196907e750e15fbb902
2019-04-06serialize: Serialization support for big-endian 32-bit ints.Jim Posen
2019-02-03speed up Unserialize_impl for prevectorAkio Nakamura
The unserializer for prevector uses resize() for reserve the area, but it's prefer to use reserve() because resize() have overhead to call its constructor many times. However, reserve() does not change the value of "_size" (a private member of prevector). This PR introduce resize_uninitialized() to prevector that similar to resize() but does not call constructor, and added elements are explicitly initialized in Unserialize_imple(). The changes are as follows: 1. prevector.h Add a public member function named 'resize_uninitialized'. This function processes like as resize() but does not call constructors. So added elemensts needs explicitly initialized after this returns. 2. serialize.h In the following two function: Unserialize_impl(Stream& is, prevector<N, T>& v, const unsigned char&) Unserialize_impl(Stream& is, prevector<N, T>& v, const V&) Calls resize_uninitialized() instead of resize() 3. test/prevector_tests.cpp Add a test for resize_uninitialized().
2018-09-11Drop minor GetSerializeSize templateBen Woosley
Now that `GetType()` is not propagated, the benefits are not worth the code.
2018-09-11Drop unused GetType() from CSizeComputerBen Woosley
2018-07-27Update copyright headers to 2018DrahtBot
2018-07-24scripted-diff: Remove trailing whitespacesJoão Barbosa
-BEGIN VERIFY SCRIPT- sed --in-place'' --regexp-extended 's/[[:space:]]+$//g' $(git grep -I --files-with-matches --extended-regexp '[[:space:]]+$' -- src test ':!*.svg' ':!src/crypto/sha256_sse4*' ':!src/leveldb' ':!src/qt/locale' ':!src/secp256k1' ':!src/univalue') -END VERIFY SCRIPT-
2018-07-19Avoid creating a temporary vector for size-prefixed elementsPieter Wuille
2018-07-01Detect if char equals int8_tChun Kuan Lee
2018-04-11Merge #12916: Introduce BigEndian wrapper and use it for netaddress portsWladimir J. van der Laan
ece88fd Introduce BigEndian wrapper and use it for netaddress ports (Pieter Wuille) Pull request description: This is another small improvement taken from #10785. Instead of manually converting from/to BE format in the `CService` serializer, provide a generic way in serialize.h to serialize BE data (only 16 bits for now). Tree-SHA512: bd67cf7eed465dad08551fb62f659e755e0691e4597a9f59d285d2b79975b50e5710d35a34a185b5ad232e1deda9a4946615f9132b1ed7d96ed8087f73ace66b
2018-04-10Merge #12731: Support serialization as another type without castingWladimir J. van der Laan
818dc74 Support serialization as another type without casting (Pieter Wuille) Pull request description: This adds a `READWRITEAS(type, obj)` macro which serializes `obj` as if it were converted to `const type&` when `const`, and to `type&` when non-`const`. No actual cast is involved, so this only works when this conversion can be done automatically. This makes it usable in serialization code that uses a single implementation for both serialization and deserializing, which doesn't know the constness of the object involved. This is a redo of #12712, using a slightly different interface. Tree-SHA512: 262f0257284ff99b5ffaec9b997c194e221522ba35c3ac8eaa9bb344449d7ea0a314de254dc77449fa7aaa600f8cd9a24da65aade8c1ec6aa80c6e9a7bba5ca7
2018-04-08Introduce BigEndian wrapper and use it for netaddress portsPieter Wuille
2018-04-05Support serializing Span<unsigned char> and use that instead of FLATDATAPieter Wuille
2018-03-21Add native support for serializing char arrays without FLATDATAPieter Wuille
Support is added to serialize arrays of type char or unsigned char directly, without any wrappers. All invocations of the FLATDATA wrappers that are obsoleted by this are removed. This includes a patch by Russell Yanofsky to make char casting type safe. The serialization of CSubNet is changed to serialize a bool directly rather than though FLATDATA. This makes the serialization independent of the size of the bool type (and will use 1 byte everywhere).
2018-03-20Support serialization as another type without castingPieter Wuille
This adds a READWRITEAS(type, obj) macro which serializes obj as if it were casted to (const type&) when const, and to (type&) when non-const. This makes it usable in serialization code that uses a single implementation for both serialization and deserializing, which doesn't know the constness of the object involved.
2018-03-15Add static_assert to prevent VARINT(<signed value>)Russell Yanofsky
Using VARINT with signed types is dangerous because negative values will appear to serialize correctly, but then deserialize as positive values mod 128. This commit changes the VARINT macro to trigger an error by default if called with an signed value, and updates broken uses of VARINT to pass a special flag that lets them keep working with no change in behavior.
2018-03-13Support deserializing into temporariesPieter Wuille
Currently, the READWRITE macro cannot be passed any non-const temporaries, as the SerReadWrite function only accepts lvalue references. Deserializing into a temporary is very common, however. See for example things like 's >> VARINT(n)'. The VARINT macro produces a temporary wrapper that holds a reference to n. Fix this by accepting non-const rvalue references instead of lvalue references. We don't propagate the rvalue-ness down, as there are no useful optimizations that only apply to temporaries. Then use this new functionality to get rid of many (but not all) uses of the 'REF' macro (which casts away constness).
2018-03-13Merge READWRITEMANY into READWRITEPieter Wuille
2018-01-03Increment MIT Licence copyright header year on files modified in 2017Akira Takizawa
2017-11-30Merge #10493: Use range-based for loops (C++11) when looping over map elementsMarcoFalke
680bc2cbb Use range-based for loops (C++11) when looping over map elements (practicalswift) Pull request description: Before this commit: ```c++ for (std::map<T1, T2>::iterator x = y.begin(); x != y.end(); ++x) { T1 z = (*x).first; … } ``` After this commit: ```c++ for (auto& x : y) { T1 z = x.first; … } ``` Tree-SHA512: 954b136b7f5e6df09f39248a6b530fd9baa9ab59d7c2c7eb369fd4afbb591b7a52c92ee25f87f1745f47b41d6828b7abfd395b43daf84a55b4e6a3d45015e3a0
2017-11-16scripted-diff: Replace #include "" with #include <> (ryanofsky)MeshCollider
-BEGIN VERIFY SCRIPT- for f in \ src/*.cpp \ src/*.h \ src/bench/*.cpp \ src/bench/*.h \ src/compat/*.cpp \ src/compat/*.h \ src/consensus/*.cpp \ src/consensus/*.h \ src/crypto/*.cpp \ src/crypto/*.h \ src/crypto/ctaes/*.h \ src/policy/*.cpp \ src/policy/*.h \ src/primitives/*.cpp \ src/primitives/*.h \ src/qt/*.cpp \ src/qt/*.h \ src/qt/test/*.cpp \ src/qt/test/*.h \ src/rpc/*.cpp \ src/rpc/*.h \ src/script/*.cpp \ src/script/*.h \ src/support/*.cpp \ src/support/*.h \ src/support/allocators/*.h \ src/test/*.cpp \ src/test/*.h \ src/wallet/*.cpp \ src/wallet/*.h \ src/wallet/test/*.cpp \ src/wallet/test/*.h \ src/zmq/*.cpp \ src/zmq/*.h do base=${f%/*}/ relbase=${base#src/} sed -i "s:#include \"\(.*\)\"\(.*\):if test -e \$base'\\1'; then echo \"#include <\"\$relbase\"\\1>\\2\"; else echo \"#include <\\1>\\2\"; fi:e" $f done -END VERIFY SCRIPT-
2017-10-09Use range-based for loops (C++11) when looping over map elementspracticalswift
Before this commit: for (std::map<T1, T2>::iterator x = y.begin(); x != y.end(); ++x) { } After this commit: for (auto& x : y) { }
2017-08-16Declare single-argument (non-converting) constructors "explicit"practicalswift
In order to avoid unintended implicit conversions.
2017-07-08Fix subscript[0] in validation.cppJeremy Rubin
2017-02-06Prevent integer overflow in ReadVarInt.Gregory Maxwell
We don't normally use ReadVarInt from untrusted inputs, but we might see this in the case of corruption. This is exposed in test_bitcoin_fuzzy.
2016-12-31Increment MIT Licence copyright header year on files modified in 2016isle2983
Edited via: $ contrib/devtools/copyright_header.py update .
2016-12-09Refactor: Removed begin/end_ptr functions.Karl-Johan Alm
2016-11-19Add serialization for unique_ptr and shared_ptrPieter Wuille
2016-11-07Add optimized CSizeComputer serializersPieter Wuille
To get the advantages of faster GetSerializeSize() implementations back that were removed in "Make GetSerializeSize a wrapper on top of CSizeComputer", reintroduce them in the few places in the form of a specialized Serialize() implementation. This actually gets us in a better state than before, as these even get used when they're invoked indirectly in the serialization of another object.
2016-11-07Make CSerAction's ForRead() constexprPieter Wuille
The CSerAction's ForRead() method does not depend on any runtime data, so guarantee that requests to it can be optimized out by making it constexpr. Suggested by Cory Fields.
2016-11-07Avoid -Wshadow errorsPieter Wuille
Suggested by Pavel Janik.
2016-11-07Get rid of nType and nVersionPieter Wuille
Remove the nType and nVersion as parameters to all serialization methods and functions. There is only one place where it's read and has an impact (in CAddress), and even there it does not impact any of the recursively invoked serializers. Instead, the few places that need nType or nVersion are changed to read it directly from the stream object, through GetType() and GetVersion() methods which are added to all stream classes.