aboutsummaryrefslogtreecommitdiff
path: root/src/test/fuzz
AgeCommit message (Collapse)Author
2023-09-12[refactor] Add CChainParams member to CConnmanTheCharlatan
This is done in preparation to the next commit, but has the nice effect of removing one further data structure relying on the global `Params()`.
2023-09-12[refactor] Add missing includes for next commitTheCharlatan
2023-09-12Add PubKeyDestination for P2PK scriptsAndrew Chow
P2PK scripts are not PKHash destinations, they should have their own type. This also results in no longer showing a p2pkh address for p2pk outputs. However for backwards compatibility, ListCoinst will still do this conversion.
2023-09-12Make WitnessUnknown members privateAndrew Chow
Make sure that nothing else can change WitnessUnknown's data members by making them private. Also change the program to use a vector rather than C-style array.
2023-09-12[fuzz] Use afl++ shared-memory fuzzingdergoegge
Using shared-memory is faster than reading from stdin, see https://github.com/AFLplusplus/AFLplusplus/blob/7d2122e0596132f9344a5d0896020ebc79cd33db/instrumentation/README.persistent_mode.md
2023-09-12scripted-diff: Rename CBufferedFile to BufferedFileMarcoFalke
While touching all constructors in the previous commit, the class name can be adjusted to comply with the style guide. -BEGIN VERIFY SCRIPT- sed -i 's/CBufferedFile/BufferedFile/g' $( git grep -l CBufferedFile ) -END VERIFY SCRIPT-
2023-09-12Remove unused GetType() from CBufferedFile and CAutoFileMarcoFalke
GetType() is only called in tests, so it is unused and can be removed.
2023-09-10net: do not use send buffer to store/cache garbagePieter Wuille
Before this commit the V2Transport::m_send_buffer is used to store the garbage: * During MAYBE_V1 state, it's there despite not being sent. * During AWAITING_KEY state, while it is being sent. * At the end of the AWAITING_KEY state it cannot be wiped as it's still needed to compute the garbage authentication packet. Change this by introducing a separate m_send_garbage field, taking over the first and last role listed above. This means the garbage is only in the send buffer when it's actually being sent, removing a few special cases related to this.
2023-09-08Merge bitcoin/bitcoin#28196: BIP324 connection supportfanquake
db9888feec48c6220a2fcf92865503bbbdab02a4 net: detect wrong-network V1 talking to V2Transport (Pieter Wuille) 91e1ef8684997fb4b3e8b64ef3935a936445066b test: add unit tests for V2Transport (Pieter Wuille) 297c8889975a18258d6cc39b1ec1e94fed6630fb net: make V2Transport preallocate receive buffer space (Pieter Wuille) 3ffa5fb49ee4a6d9502aa957093bd94058630282 net: make V2Transport send uniformly random number garbage bytes (Pieter Wuille) 0be752d9f8ca27320bc3e82498c7640fabd7e8de net: add short message encoding/decoding support to V2Transport (Pieter Wuille) 8da8642062fa2c7aa2f49995b832c3d0897e37ed net: make V2Transport auto-detect incoming V1 and fall back to it (Pieter Wuille) 13a7f01557272db652b3f333af3f06af6897253f net: add V2Transport class with subset of BIP324 functionality (Pieter Wuille) dc2d7eb810ef95b06620f334c198687579916435 crypto: Spanify EllSwiftPubKey constructor (Pieter Wuille) 5f4b2c6d79e81ee0445752ad558fcc17831f4b2f net: remove unused Transport::SetReceiveVersion (Pieter Wuille) c3fad1f29df093e8fd03d70eb43f25ee9d531bf7 net: add have_next_message argument to Transport::GetBytesToSend() (Pieter Wuille) Pull request description: This is part of #27634. This implements the BIP324 v2 transport (which implements all of what the BIP calls transport layer *and* application layer), though in a non-exposed way. It is tested through an extensive fuzz test, which verifies that v2 transports can talk to v2 transports, and v1 transports can talk to v2 transports, and a unit test that exercises a number of unusual scenarios. The transport is functionally complete, including: * Autodetection of incoming V1 connections. * Garbage, both sending and receiving. * Short message type IDs, both sending and receiving. * Ignore packets (receiving only, but tested in a unit test). * Session IDs are visible in `getpeerinfo` output (for manual comparison). Things that are not included, left for future PRs, are: * Actually using the v2 transport for connections. * Support for the `NODE_P2P_V2` service flag. * Retrying downgrade to V1 when attempted outbound V2 connections immediately fail. * P2P functional and unit tests ACKs for top commit: naumenkogs: ACK db9888feec48c6220a2fcf92865503bbbdab02a4 theStack: re-ACK db9888feec48c6220a2fcf92865503bbbdab02a4 mzumsande: Code Review ACK db9888feec48c6220a2fcf92865503bbbdab02a4 Tree-SHA512: 8906ac1e733a99e1f31c9111055611f706d80bbfc2edf6a07fa6e47b21bb65baacd1ff17993cbbf588063b2f5ad30b3af674a50c7bc8e8ebf4671483a21bbfeb
2023-09-07Merge bitcoin/bitcoin#28361: fuzz: add ConstructPubKeyBytes util functionfanquake
1580e3be83bd03985b2f288ed70de510903068d9 fuzz: add ConstructPubKeyBytes function (josibake) Pull request description: In https://github.com/bitcoin/bitcoin/pull/28246 and https://github.com/bitcoin/bitcoin/pull/28122 , we add a `PubKeyDestination` and a `V0SilentPaymentsDestination`. Both of these PRs update `fuzz/util.cpp` and need a way to create well-formed pubkeys. Currently in `fuzz/util.cpp`, we have some logic for creating pubkeys in the multisig data provider. This logic is duplicated in #28246 and duplicated again in #28122. Seems much better to have a `ConstructPubKeyBytes` function that both PRs (and any future work) can reuse. This PR introduces a function to do this and has the existing code use it. While the purpose is to introduce a utility function, the previous multisig code used `ConsumeIntegralInRange(4, 7)` which would have created some uncompressed pubkeys with the prefix 0x05, which is incorrect (see https://bitcoin.stackexchange.com/questions/57855/c-secp256k1-what-do-prefixes-0x06-and-0x07-in-an-uncompressed-public-key-signif) tldr; using `PickValueFromArray` is more correct as it limits to the set of defined prefixes for compressed and uncompressed pubkeys. ACKs for top commit: Sjors: ACK 1580e3be83bd03985b2f288ed70de510903068d9 Tree-SHA512: c87c8bcd1f6b3a97ef772be93102efb912811c59f32211cfd531a116f1da8a57c8c6ff106b34f2a2b88d8b34fb5bc30d9f9ed6d2720113ffcaaa2f8d5dc9eb27
2023-09-07net: make V2Transport send uniformly random number garbage bytesPieter Wuille
2023-09-07net: make V2Transport auto-detect incoming V1 and fall back to itPieter Wuille
2023-09-07net: add V2Transport class with subset of BIP324 functionalityPieter Wuille
This introduces a V2Transport with a basic subset of BIP324 functionality: * no ability to send garbage (but receiving is supported) * no ability to send decoy packets (but receiving them is supported) * no support for short message id encoding (neither encoding or decoding) * no waiting until 12 non-V1 bytes have been received * (and thus) no detection of V1 connections on the responder side (on the sender side, detecting V1 is not supported either, but that needs to be dealt with at a higher layer, by reconnecting)
2023-09-07net: add have_next_message argument to Transport::GetBytesToSend()Pieter Wuille
Before this commit, there are only two possibly outcomes for the "more" prediction in Transport::GetBytesToSend(): * true: the transport itself has more to send, so the answer is certainly yes. * false: the transport has nothing further to send, but if vSendMsg has more message(s) left, that still will result in more wire bytes after the next SetMessageToSend(). For the BIP324 v2 transport, there will arguably be a third state: * definitely not: the transport has nothing further to send, but even if vSendMsg has more messages left, they can't be sent (right now). This happens before the handshake is complete. To implement this, we move the entire decision logic to the Transport, by adding a boolean to GetBytesToSend(), called have_next_message, which informs the transport whether more messages are available. The return values are still true and false, but they mean "definitely yes" and "definitely no", rather than "yes" and "maybe".
2023-09-07Merge bitcoin/bitcoin#25284: net: Use serialization parameters for CAddress ↵fanquake
serialization fa626af3edbe8d98b2de91dd71729ceef90389fb Remove unused legacy CHashVerifier (MarcoFalke) fafa3fc5a62702da72991497e3270034eb9159c0 test: add tests that exercise WithParams() (MarcoFalke) fac81affb527132945773a5315bd27fec61ec52f Use serialization parameters for CAddress serialization (MarcoFalke) faec591d64e40ba7ec7656cbfdda1a05953bde13 Support for serialization parameters (MarcoFalke) fac42e9d35f6ba046999b2e3a757ab720c51b6bb Rename CSerAction* to Action* (MarcoFalke) aaaa3fa9477eef9ea72e4a501d130c57b47b470a Replace READWRITEAS macro with AsBase wrapping function (MarcoFalke) Pull request description: It seems confusing that picking a wrong value for `ADDRV2_FORMAT` could have effects on consensus. (See the docstring of `ADDRV2_FORMAT`). Fix this by implementing https://github.com/bitcoin/bitcoin/issues/19477#issuecomment-1147421608 . This may also help with libbitcoinkernel, see https://github.com/bitcoin/bitcoin/pull/28327 ACKs for top commit: TheCharlatan: ACK fa626af3edbe8d98b2de91dd71729ceef90389fb ajtowns: ACK fa626af3edbe8d98b2de91dd71729ceef90389fb Tree-SHA512: 229d379da27308890de212b1fd2b85dac13f3f768413cb56a4b0c2da708f28344d04356ffd75bfcbaa4cabf0b6cc363c4f812a8f1648cff9e436811498278318
2023-09-07Merge bitcoin/bitcoin#28419: fuzz: introduce and use `ConsumePrivateKey` helperfanquake
583af18fd1d0bda5a6a1d0403ffc498a512a546d fuzz: introduce and use `ConsumePrivateKey` helper (Sebastian Falbesoner) Pull request description: In the course of reviewing BIP324 related PRs I noticed a frequent pattern of creating private keys (`CKey` instances) with data consumed from the fuzz data provider: ``` auto key_data = provider.ConsumeBytes<unsigned char>(32); key_data.resize(32); CKey key; key.Set(key_data.begin(), key_data.end(), /*fCompressedIn=*/true); ``` This PR introduces a corresponding helper `ConsumePrivateKey` in order to deduplicate code. The compressed flag can either be set to a fixed value, or, if `std::nullopt` is passed (=default), is also consumed from the fuzz data provider via `.ConsumeBool()`. Note that this is not a pure refactor, as some of the replaced call-sites previously consumed a random length (`ConsumeRandomLengthByteVector`) instead of a fixed size of 32 bytes for key data. As far as I can see, there is not much value in using a random size, as in all those cases we can only proceed or do something useful with a valid private key, and key data with sizes other than 32 bytes always lead to invalid keys. ACKs for top commit: sipa: utACK 583af18fd1d0bda5a6a1d0403ffc498a512a546d brunoerg: crACK 583af18fd1d0bda5a6a1d0403ffc498a512a546d Tree-SHA512: 58a178432ba1eb0a2f7597b6700e96477e8b10f429ef642445a52db12e74d81aec307888315b772bfda9610f90df3e1d556cf024c2bef1d520303b12584feaaa
2023-09-06Merge bitcoin/bitcoin#26567: Wallet: estimate the size of signed inputs ↵Andrew Chow
using descriptors 10546a569c6c96a5ec1b9708abf9ff5c8644f669 wallet: accurately account for the size of the witness stack (Antoine Poinsot) 9b7ec393b82ca9d7ada77d06e0835df0386a8b85 wallet: use descriptor satisfaction size to estimate inputs size (Antoine Poinsot) 8d870a98731e8db5ecc614bb5f7c064cbf30c7f4 script/signingprovider: introduce a MultiSigningProvider (Antoine Poinsot) fa7c46b503f0b69630f55dc43021d2099e3515ba descriptor: introduce a method to get the satisfaction size (Antoine Poinsot) bdba7667d2d65f31484760a8e8420c488fc5f801 miniscript: introduce a helper to get the maximum witness size (Antoine Poinsot) 4ab382c2cdb09fb4056711b4336807845cbe1ad5 miniscript: make GetStackSize independent of P2WSH context (Antoine Poinsot) Pull request description: The wallet currently estimates the size of a signed input by doing a dry run of the signing logic. This is unnecessary since all outputs we can sign for can be represented by a descriptor, and we can derive the size of a satisfaction ("signature") directly from the descriptor itself. In addition, the current approach does not generalize well: dry runs of the signing logic are only possible for the most basic scripts. See for instance the discussion in #24149 around that. This introduces a method to get the maximum size of a satisfaction from a descriptor, and makes the wallet use that instead of the dry-run. ACKs for top commit: sipa: utACK 10546a569c6c96a5ec1b9708abf9ff5c8644f669 achow101: re-ACK 10546a569c6c96a5ec1b9708abf9ff5c8644f669 Tree-SHA512: 43ed1529fbd30af709d903c8c5063235e8c6a03b500bc8f144273d6184e23a53edf0fea9ef898ed57d8a40d73208b5d935cc73b94a24fad3ad3c63b3b2027174
2023-09-06fuzz: introduce and use `ConsumePrivateKey` helperSebastian Falbesoner
2023-09-05Use serialization parameters for CAddress serializationMarcoFalke
This also cleans up the addrman (de)serialization code paths to only allow `Disk` serialization. Some unit tests previously forced a `Network` serialization, which does not make sense, because Bitcoin Core in production will always `Disk` serialize. This cleanup idea was suggested by Pieter Wuille and implemented by Anthony Towns. Co-authored-by: Pieter Wuille <pieter@wuille.net> Co-authored-by: Anthony Towns <aj@erisian.com.au>
2023-08-30fuzz: add ConstructPubKeyBytes functionjosibake
Today, this code only has one spot where it needs well-formed pubkeys, but future PRs will want to reuse this code. Add a function which creates a well-formed byte array that can be turned into a pubkey. It is not required that the pubkey is valid, just that it can be recognized as a compressed or uncompressed pubkey. Note: while the main intent of this commit is to wrap the existing logic into a function, it also switches to `PickValueFromArray` so that we are only choosing one of 0x04, 0x06, or 0x07. The previous code, `ConsumeIntegralInRange` would have also picked 0x05, which is not definied in the context of compressed vs uncompressed keys. See https://bitcoin.stackexchange.com/questions/57855/c-secp256k1-what-do-prefixes-0x06-and-0x07-in-an-uncompressed-public-key-signif for more details.
2023-08-25net: remove Sock default constructor, it's not necessaryVasil Dimov
2023-08-25wallet: accurately account for the size of the witness stackAntoine Poinsot
When estimating the maximum size of an input, we were assuming the number of elements on the witness stack could be encode in a single byte. This is a valid approximation for all the descriptors we support (including P2WSH Miniscript ones), but may not hold anymore once we support Miniscript within Taproot descriptors (since the max standard witness stack size of 100 gets lifted). It's a low-hanging fruit to account for it correctly, so just do it now.
2023-08-25descriptor: introduce a method to get the satisfaction sizeAntoine Poinsot
In the wallet code, we are currently estimating the size of a signed input by doing a dry run of the signing logic. This is unnecessary as all outputs we are able to sign for can be represented by a descriptor, and we can derive the size of a satisfaction ("signature") from the descriptor itself directly. In addition, this approach does not scale: getting the size of a satisfaction through a dry run of the signing logic is only possible for the most basic scripts. This commit introduces the computation of the size of satisfaction per descriptor. It's a bit intricate for 2 main reasons: - We want to conserve the behaviour of the current dry-run logic used by the wallet that sometimes assumes ECDSA signatures will be low-r, sometimes not (when we don't create them). - We need to account for the witness discount. A single descriptor may sometimes benefit of it, sometimes not (for instance `pk()` if used as top-level versus if used inside `wsh()`).
2023-08-25miniscript: make GetStackSize independent of P2WSH contextAntoine Poinsot
It was taking into account the P2WSH script push in the number of stack elements.
2023-08-24Merge bitcoin/bitcoin#28287: rpc, test: add `sendmsgtopeer` rpc and a test ↵Andrew Chow
for net-level deadlock situation b3a93b409e7fb33af77bd34a269a3eae71d3ba83 test: add functional test for deadlock situation (Martin Zumsande) 3557aa4d0ab59c18807661a49070b0e5cbfecde3 test: add basic tests for sendmsgtopeer to rpc_net.py (Martin Zumsande) a9a1d69391596f57851f545fae12f8851149d2c3 rpc: add test-only sendmsgtopeer rpc (Martin Zumsande) Pull request description: This adds a `sendmsgtopeer` rpc (for testing only) that allows a node to send a message (provided in hex) to a peer. While we would usually use a `p2p` object instead of a node for this in the test framework, that isn't possible in situations where this message needs to trigger an actual interaction of multiple nodes. Use this rpc to add test coverage for the bug fixed in #27981 (that just got merged): The test lets two nodes (almost) simultaneously send a single large (4MB) p2p message to each other, which would have caused a deadlock previously (making this test fail), but succeeds now. As can be seen from the discussion in #27981, it was not easy to reproduce this bug without `sendmsgtopeer`. I would imagine that `sendmsgtopeer` could also be helpful in various other test constellations. ACKs for top commit: ajtowns: ACK b3a93b409e7fb33af77bd34a269a3eae71d3ba83 sipa: ACK b3a93b409e7fb33af77bd34a269a3eae71d3ba83 achow101: ACK b3a93b409e7fb33af77bd34a269a3eae71d3ba83 Tree-SHA512: 6e22e72402f3c4dd70cddb9e96ea988444720f7a164031df159fbdd48056c8ac77ac53def045d9208a3ca07437c7c8e34f8b4ebc7066c0a84d81cd53f2f4fa5f
2023-08-23refactor: make Transport::ReceivedBytes just return success/failPieter Wuille
2023-08-23net: move message conversion to wire bytes from PushMessage to SocketSendDataPieter Wuille
This furthers transport abstraction by removing the assumption that a message can always immediately be converted to wire bytes. This assumption does not hold for the v2 transport proposed by BIP324, as no messages can be sent before the handshake completes. This is done by only keeping (complete) CSerializedNetMsg objects in vSendMsg, rather than the resulting bytes (for header and payload) that need to be sent. In SocketSendData, these objects are handed to the transport as permitted by it, and sending out the bytes the transport tells us to send. This also removes the nSendOffset member variable in CNode, as keeping track of how much has been sent is now a responsability of the transport. This is not a pure refactor, and has the following effects even for the current v1 transport: * Checksum calculation now happens in SocketSendData rather than PushMessage. For non-optimistic-send messages, that means this computation now happens in the network thread rather than the message handler thread (generally a good thing, as the message handler thread is more of a computational bottleneck). * Checksum calculation now happens while holding the cs_vSend lock. This is technically unnecessary for the v1 transport, as messages are encoded independent from one another, but is untenable for the v2 transport anyway. * Statistics updates about per-message sent bytes now happen when those bytes are actually handed to the OS, rather than at PushMessage time.
2023-08-23fuzz: add bidirectional fragmented transport testPieter Wuille
This adds a simulation test, with two V1Transport objects, which send messages to each other, with sending and receiving fragmented into multiple pieces that may be interleaved. It primarily verifies that the sending and receiving side are compatible with each other, plus a few sanity checks.
2023-08-23net: make V1Transport implicitly use current chainparamsPieter Wuille
The rest of net.cpp already uses Params() to determine chainparams in many places (and even V1Transport itself does so in some places). Since the only chainparams dependency is through the message start characters, just store those directly in the transport.
2023-08-23net: abstract sending side of transport serialization furtherPieter Wuille
This makes the sending side of P2P transports mirror the receiver side: caller provides message (consisting of type and payload) to be sent, and then asks what bytes must be sent. Once the message has been fully sent, a new message can be provided. This removes the assumption that P2P serialization of messages follows a strict structure of header (a function of type and payload), followed by (unmodified) payload, and instead lets transports decide the structure themselves. It also removes the assumption that a message must always be sent at once, or that no bytes are even sent on the wire when there is no message. This opens the door for supporting traffic shaping mechanisms in the future.
2023-08-23refactor: rename Transport class receive functionsPieter Wuille
Now that the Transport class deals with both the sending and receiving side of things, make the receive side have function names that clearly indicate they're about receiving. * Transport::Read() -> Transport::ReceivedBytes() * Transport::Complete() -> Transport::ReceivedMessageComplete() * Transport::GetMessage() -> Transport::GetReceivedMessage() * Transport::SetVersion() -> Transport::SetReceiveVersion() Further, also update the comments on these functions to (among others) remove the "deserialization" terminology. That term is better reserved for just the serialization/deserialization between objects and bytes (see serialize.h), and not the conversion from/to wire bytes as performed by the Transport.
2023-08-23refactor: merge transport serializer and deserializer into Transport classPieter Wuille
This allows state that is shared between both directions to be encapsulated into a single object. Specifically the v2 transport protocol introduced by BIP324 has sending state (the encryption keys) that depends on received messages (the DH key exchange). Having a single object for both means it can hide logic from callers related to that key exchange and other interactions.
2023-08-22rpc: add test-only sendmsgtopeer rpcMartin Zumsande
This rpc can be used when we want a node to send a message, but cannot use a python P2P object, for example for testing of low-level net transport behavior.
2023-08-18assumeutxo cleanup: Move IsInitialBlockDownload & NotifyHeaderTip to ↵Ryan Ofsky
ChainstateManager This change makes IsInitialBlockDownload and NotifyHeaderTip functions no longer tied to individual Chainstate objects. It makes them work with the ChainstateManager object instead so code is simpler and it is no longer possible to call them incorrectly with an inactive Chainstate. This change also makes m_cached_finished_ibd caching easier to reason about, because now there is only one cached value instead of two (for background and snapshot chainstates) so the cached IBD state now no longer gets reset when a snapshot is loaded. There should be no change in behavior because these functions were always called on the active ChainState objects. These changes were discussed previously https://github.com/bitcoin/bitcoin/pull/27746#discussion_r1246868905 and https://github.com/bitcoin/bitcoin/pull/27746#discussion_r1237552792 as possible followups for that PR.
2023-08-17fuzz: support std::byte in Consume{Fixed,Variable}LengthByteVectorPieter Wuille
2023-08-17crypto: require key on ChaCha20 initializationPieter Wuille
2023-08-17crypto: refactor ChaCha20 classes to use Span<std::byte> interfacePieter Wuille
2023-08-17Merge bitcoin/bitcoin#28244: Break up script/standard.{h/cpp}fanquake
91d924ede1b421df31c895f4f43359e453a09ca5 Rename script/standard.{cpp/h} to script/solver.{cpp/h} (Andrew Chow) bacdb2e208531124e85ed2d4ea2a4b508fbb5088 Clean up script/standard.{h/cpp} includes (Andrew Chow) f3c9078b4cddec5581e52de5c216ae53984ec130 Clean up things that include script/standard.h (Andrew Chow) 8bbe257bac751859a272ddf52dc0328c1b5a1ede MOVEONLY: Move datacarrier defaults to policy.h (Andrew Chow) 7a172c76d2361fc3cdf6345590e26c79a7821672 Move CTxDestination to its own file (Andrew Chow) 145f36ec81e79d2e391847520364c2420ef0e0e8 Move Taproot{SpendData/Builder} to signingprovider.{h/cpp} (Andrew Chow) 86ea8bed5473f400f7a93fcc455393a574a2f319 Move CScriptID to script.{h/cpp} (Andrew Chow) b81ebff0d99c45c071b999796b8ae3f0f2517b22 Remove ScriptHash from CScriptID constructor (Andrew Chow) cba69dda3da0e4fa39cff5ce4dc81d1242fe651b Move MANDATORY_SCRIPT_VERIFY_FLAGS from script/standard.h to policy/policy.h (Anthony Towns) Pull request description: Some future work needs to touch things in script/standard.{h/cpp}, however it is unclear if it is safe to do so as they are included in several different places that could effect standardness and consensus. It contains a mix of policy parameters, consensus parameters, and utilities only used by the wallet. This PR breaks up the various components and renames the files to clearly separate everything. * `CTxDestination` is moved to a new file `src/addresstype.{cpp/h}` * `TaprootSpendData` and `TaprootBuilder` (and their utility functions and structs) are moved to `SigningProvider` as these are used only during signing. * `CScriptID` is moved to `script/script.h` to be next to `CScript`. * `MANDATORY_SCRIPT_VERIFY_FLAGS` is moved to `interpreter.h` * The parameters `DEFAULT_ACCEPT_DATACARRIER` and `MAX_OP_RETURN_RELAY` are moved to `policy.h` * `standard.{cpp/h}` is renamed to `solver.{cpp/h}` since that's all that's left in the file after the above moves ACKs for top commit: Sjors: ACK 91d924ede1b421df31c895f4f43359e453a09ca5 ajtowns: ACK 91d924ede1b421df31c895f4f43359e453a09ca5 MarcoFalke: ACK 91d924ede1b421df31c895f4f43359e453a09ca5 😇 murchandamus: ACK 91d924ede1b421df31c895f4f43359e453a09ca5 darosior: Code review ACK 91d924ede1b421df31c895f4f43359e453a09ca5. theStack: Code-review ACK 91d924ede1b421df31c895f4f43359e453a09ca5 Tree-SHA512: d347439890c652081f6a303d99b2bde6c371c96e7f4127c5db469764a17d39981f19884679ba883e28b733fde6142351dd8288c7bc61c379b7eefe7fa7acca1a
2023-08-17Merge bitcoin/bitcoin#27675: p2p: Drop m_recently_announced_invs bloom filterfanquake
fb02ba3c5f5bcd96b5e3622ef001b8e57ce63fc0 mempool_entry: improve struct packing (Anthony Towns) 1a118062fbc4ec8f645f4ec4298d869a869c3344 net_processing: Clean up INVENTORY_BROADCAST_MAX constants (Anthony Towns) 6fa49937e488d0924044786c76b42324b659f351 test: Check tx from disconnected block is immediately requestable (glozow) e4ffabbffacc4b890d393aafcc8286916ef887d8 net_processing: don't add txids to m_tx_inventory_known_filter (Anthony Towns) 6ec1809d33bfc42b80cb6f35625dccd56be8d507 net_processing: drop m_recently_announced_invs bloom filter (Anthony Towns) a70beafdb22564043dc24fc98133fdadbaf77d8a validation: when adding txs due to a block reorg, allow immediate relay (Anthony Towns) 1e9684f39fba909b3501e9402d5b61f4bf744ff2 mempool_entry: add mempool entry sequence number (Anthony Towns) Pull request description: This PR replaces the `m_recently_announced_invs` bloom filter with a simple sequence number tracking the mempool state when we last considered sending an INV message to a node. This saves 33kB per peer (or more if we raise the rate at which we relay transactions over the network, in which case we would need to increase the size of the bloom filter proportionally). The philosophy here (compare with #18861 and #19109) is that we consider the rate limiting on INV messages to only be about saving bandwidth and not protecting privacy, and therefore after you receive an INV message, it's immediately fair game to request any transaction that was in the mempool at the time the INV message was sent. We likewise consider the BIP 133 feefilter and BIP 37 bloom filters to be bandwidth optimisations here, and treat transactions as requestable if they would have been announced without those filters. Given that philosophy, tracking the timestamp of the last INV message and comparing that against the mempool entry time allows removal of each of `m_recently_announced_invs`, `m_last_mempool_req` and `UNCONDITIONAL_RELAY_DELAY` and associated logic. ACKs for top commit: naumenkogs: ACK fb02ba3c5f5bcd96b5e3622ef001b8e57ce63fc0 amitiuttarwar: review ACK fb02ba3c5f5 glozow: reACK fb02ba3c5f5bcd96b5e3622ef001b8e57ce63fc0 Tree-SHA512: cbba5ee04c86df26b6057f3654c00a2b45ec94d354f4f157a769cecdaa0b509edaac02b3128afba39b023e82473fc5e28c915a787f84457ffe66638c6ac9c2d4
2023-08-15Merge bitcoin/bitcoin#27460: rpc: Add importmempool RPCAndrew Chow
fa776e61cd64a5ffd9a4be589ab8efeb5421861a Add importmempool RPC (MarcoFalke) fa20d734a29ba50cd19b78cb4fe39a2d826131b7 refactor: Add and use kernel::ImportMempoolOptions (MarcoFalke) fa8866990dba7817427977bfe834efdb17114d37 doc: Clarify the getmempoolinfo.loaded RPC field documentation (MarcoFalke) 6888886cecf6665da70b3dc3772b3c12ef06ad76 Remove Chainstate::LoadMempool (MarcoFalke) Pull request description: Currently it is possible to import a mempool by placing it in the datadir and starting the node. However this has many issues: * Users aren't expected to fiddle with the datadir, possibly corrupting it * An existing mempool file in the datadir may be overwritten * The node needs to be restarted * Importing an untrusted file this way is dangerous, because it can corrupt the mempool Fix all issues by adding a new RPC. ACKs for top commit: ajtowns: utACK fa776e61cd64a5ffd9a4be589ab8efeb5421861a achow101: ACK fa776e61cd64a5ffd9a4be589ab8efeb5421861a glozow: reACK fa776e61cd64a5ffd9a4be589ab8efeb5421861a Tree-SHA512: fcb1a92d6460839283c546c47a2d930c363ac1013c4c50dc5215ddf9fe5e51921d23fe0abfae0a5a7631983cfc7e2fff3788b70f95937d0a989a203be4d67546
2023-08-15Merge bitcoin/bitcoin#28267: crypto: BIP324 ciphersuite follow-upfanquake
93cb8f03807dcd3297b7fe18b6c901a8d2a01596 refactor: add missing headers for BIP324 ciphersuite (stratospher) d22d5d925c000bf25ad2410ca66c4c21eea75004 crypto: BIP324 ciphersuite follow-up (stratospher) Pull request description: follow-up to #28008. * move `dummy_tag` variable in FSChaCha20Poly1305 crypto_tests outside of the loop to be reused every time * use easy to read `cipher.last()` in `AEADChaCha20Poly1305::Decrypt()` * comment for initiator in `BIP324Cipher::Initialize()` * systematically damage ciphertext with bit positions in bip324_tests * use 4095 max bytes for `aad` in bip324 fuzz test ACKs for top commit: fanquake: ACK 93cb8f03807dcd3297b7fe18b6c901a8d2a01596 - thanks for following up here. Tree-SHA512: 361f3e226d3168fdef69a2eebe6092cfc04ba14ce009420222e762698001eaf8be69a1138dab0be237964509c2b96a41a0b4db5c1df43ef75062f143c5aa741a
2023-08-15Merge bitcoin/bitcoin#28215: fuzz: fix a couple incorrect assertions in the ↵fanquake
`coins_view` target e417c988f61bf9d3948d5c8e169626922fe6e24c fuzz: coins_view: remove an incorrect assertion (Antoine Poinsot) c5f6b1db56f67f529377bfb61f58c0a8c17b0127 fuzz: coins_view: correct an incorrect assertion (Antoine Poinsot) Pull request description: The `coins_view` fuzz target would assert in two places that the cache is consistent with the backend. But it's never the case (that's the whole point of using a cache). The only reason this didn't result in a crash was that we would never actually hit these assertions. I ran into this while introducing a new target with an in-memory `CCoinsViewDB` as the backend view (see https://github.com/bitcoin/bitcoin/pull/28216) which made the code paths with those assertions actually reachable. ACKs for top commit: dergoegge: Code review ACK e417c988f61bf9d3948d5c8e169626922fe6e24c Tree-SHA512: 5847bb2744a2f2831dace62d32b79cc491bf54e2af4ce425411d245d566622d9aff816d9be5ec8e830d10851c13f2500bf4f0c004d88b4d7cca1d483ef8960a6
2023-08-15refactor: add missing headers for BIP324 ciphersuitestratospher
2023-08-14Rename script/standard.{cpp/h} to script/solver.{cpp/h}Andrew Chow
Since script/standard only contains things that are used by the Solver and its callers, rename the files to script/solver.
2023-08-14Clean up things that include script/standard.hAndrew Chow
Remove standard.h from files that don't use anything in it, and include it in files that do.
2023-08-14Move CTxDestination to its own fileAndrew Chow
CTxDestination is really our internal representation of an address and doesn't really have anything to do with standard script types, so move them to their own file.
2023-08-14Move CScriptID to script.{h/cpp}Andrew Chow
CScriptID should be next to CScript just as CKeyID is next to CPubKey
2023-08-14crypto: BIP324 ciphersuite follow-upstratospher
follow-up to #28008. * move `dummy_tag` variable in FSChaCha20Poly1305 crypto_tests outside of the loop to be reused every time * use easy to read `cipher.last()` in `AEADChaCha20Poly1305::Decrypt()` * comment for initiator in `BIP324Cipher::Initialize()` * systematically damage ciphertext with bit positions in bip324_tests * use 4095 max bytes for aad in bip324 fuzz test
2023-08-11fuzz: coins_view: remove an incorrect assertionAntoine Poinsot
Again, this was not hit because the default implementation of `CCoinsView` return `false` for `GetCoin`.
2023-08-10Merge bitcoin/bitcoin#28008: BIP324 ciphersuitefanquake
1c7582ead6e1119899922041c1af2b4169b0bc74 tests: add decryption test to bip324_tests (Pieter Wuille) 990f0f8da92a2d11828a7c05ca93bf0520b2a95e Add BIP324Cipher, encapsulating key agreement, derivation, and stream/AEAD ciphers (Pieter Wuille) c91cedf281e5207fb5fd2ca81feec9760f7c2ed0 crypto: support split plaintext in ChaCha20Poly1305 Encrypt/Decrypt (Pieter Wuille) af2b44c76e5de8ce880381e5535ead37ab0b3ba9 bench: add benchmark for FSChaCha20Poly1305 (Pieter Wuille) aa8cee93342ee857931afec9af3ff5dbd8ce7749 crypto: add FSChaCha20Poly1305, rekeying wrapper around ChaCha20Poly1305 (Pieter Wuille) 0fee267792eb8cbdd48ad78f1712420b5d8d905b crypto: add FSChaCha20, a rekeying wrapper around ChaCha20 (Pieter Wuille) 9ff0768bdcca06836ccc673eacfa648e801930cb crypto: add the ChaCha20Poly1305 AEAD as specified in RFC8439 (Pieter Wuille) 9fd085a1a49d317abcaf1492b71c48bf1a1b3007 crypto: remove outdated variant of ChaCha20Poly1305 AEAD (Pieter Wuille) Pull request description: Depends on #27985 and #27993, based on and partially replaces #25361, part of #27634. Draft while dependencies are not merged. This adds implementations of: * The ChaCha20Poly1305 AEAD from [RFC8439 section 2.8](https://datatracker.ietf.org/doc/html/rfc8439#section-2.8), including test vectors. * The FSChaCha20 stream cipher as specified in [BIP324](https://github.com/bitcoin/bips/blob/master/bip-0324.mediawiki#rekeying-wrappers-fschacha20poly1305-and-fschacha20), a rekeying wrapper around ChaCha20. * The FSChaCha20Poly1305 AEAD as specified in [BIP324](https://github.com/bitcoin/bips/blob/master/bip-0324.mediawiki#rekeying-wrappers-fschacha20poly1305-and-fschacha20), a rekeying wrapper around ChaCha20Poly1305. * A BIP324Cipher class that encapsulates key agreement, key derivation, and stream ciphers and AEADs for [BIP324 packet encoding](https://github.com/bitcoin/bips/blob/master/bip-0324.mediawiki#overall-packet-encryption-and-decryption-pseudocode). The ChaCha20Poly1305 and FSChaCha20Poly1305 implementations are new, taking advance of the improvements in #27993. ACKs for top commit: jamesob: reACK 1c7582e theStack: ACK 1c7582ead6e1119899922041c1af2b4169b0bc74 stratospher: tested ACK 1c7582e. Tree-SHA512: 06728b4b95b21c5b732ed08faf40e94d0583f9d86ff4db3b92dd519dcd9fbfa0f310bc66ef1e59c9e49dd844ba8c5ac06e2001762a804fb5aa97027816045a46