| Age | Commit message (Collapse) | Author |
|---|
|
This fixes an intermittent error, caused by blocks arriving
out of order due to how compact block relay may revert to headers
processing when the tip hasn't caught up, and resulting in slightly
different pruning behavior.
Making sure that all blocks from the previous tests are synced before
generating more blocks makes this impossible.
See Issue #27749 for more details.
|
|
fe329dc936d1e02da406345e4223e11d1fa6fb38 test: Add test for getblockfrompeer on pruned nodes (Fabian Jahr)
cd761e6b2cfade038b8018886c334bf25a0bcbcf rpc: Add note on guarantees to getblockfrompeer (Fabian Jahr)
Pull request description:
These are additions to `getblockfrompeer` that I already [suggested on the original PR](https://github.com/bitcoin/bitcoin/pull/20295#pullrequestreview-817157738).
The two commits do the following:
1. Add a test for `getblockfrompeer` usage on pruned nodes. This is important because many use-cases for `getblockfrompeer` are in a context of a pruned node.
2. Add some information on how long the users of pruned nodes can expect the block to be available after they have used the RPC. I think the behavior is not very intuitive for users and I would not be surprised if users expect the block to be available indefinitely.
ACKs for top commit:
Sjors:
re-utACK fe329dc936d1e02da406345e4223e11d1fa6fb38
MarcoFalke:
review ACK fe329dc936d1e02da406345e4223e11d1fa6fb38 🍉
stratospher:
ACK fe329dc.
brunoerg:
re-ACK fe329dc936d1e02da406345e4223e11d1fa6fb38
Tree-SHA512: a686bd8955d9c3baf365db384e497d6ee1aa9ce2fdb0733fe6150f7e3d94bae19d55bc1b347f1c9f619e749e18b41a52b9f8c0aa2042dd311a968a4b5d251fac
|
|
-BEGIN VERIFY SCRIPT-
./contrib/devtools/copyright_header.py update ./
-END VERIFY SCRIPT-
Commits of previous years:
- 2021: f47dda2c58b5d8d623e0e7ff4e74bc352dfa83d7
- 2020: fa0074e2d82928016a43ca408717154a1c70a4db
- 2019: aaaaad6ac95b402fe18d019d67897ced6b316ee0
|
|
|
|
by adding a sync to make sure the node has received the
header before we query it for the block
|
|
syncing past their height
5826bf546e83478947edbdf49978414f0b69eb1a test: Add test for getblockfrompeer on syncing pruned nodes (Fabian Jahr)
7fa851fba8570ef256317f7d5759aa3de9088bf1 rpc: Pruned nodes can not fetch unsynced blocks (Fabian Jahr)
Pull request description:
This PR prevents `getblockfrompeer` from getting used on blocks that the node has not synced past yet if the node is in running in prune mode.
### Problem
While a node is still catching up to the tip that it is aware of via the headers, the user can currently use to fetch blocks close to or at the tip. These blocks are stored in the block/rev file that otherwise contains blocks the node is receiving as part of the syncing process.
This creates a problem for pruned nodes: The files containing a fetched block are not pruned during syncing because they contain a block close to the tip. This means the entire file (~130MB) will not be pruned until the tip has moved on far enough from the fetched block. In extreme cases with heavy pruning (like 550) and multiple blocks being fetched this could mean that the disc usage far exceeds what the user expects, potentially running out of space.
### Approach
There would be certainly other approaches that could fix the problem while still allowing the current behavior, but all of the ideas I came up with seemed like overkill for a niche problem on a new RPC where it's still unclear how and how much it will be used.
### Testing
So far I did not see a simple enough way to test this I am still looking into it and if it's complex will potentially add it in a follow-up. What would be needed is a way to have a node fetch headers but not sync the blocks yet, that seems like a pattern that could be generally useful.
To manually reproduce the problematic behavior:
1. Start a node with current `master` with `-prune=550` and an empty/new datadir, Testnet and Mainnet should both work.
2. While the node is syncing run `getblockfrompeer` on the current tip and a few other recent blocks.
3. Go to your datadir and observe the blocks folder: There should be a few full `blk*.dat` and `rev*.dat` files that are not being pruned. When you "pinned" a few of these files the blocks folder should be significantly above the target size of 550MB.
ACKs for top commit:
Sjors:
utACK 5826bf546e83478947edbdf49978414f0b69eb1a
achow101:
ACK 5826bf546e83478947edbdf49978414f0b69eb1a
aureleoules:
tACK 5826bf546e83478947edbdf49978414f0b69eb1a
Tree-SHA512: aa3f477ec755a9df2331c047cb10b3cd08292522bf6ad7a36a7ea36d7eba4894b84de8bd23003c9baea5ac0c53b77142c3c2819ae7528cece9d10a0d06c850d8
|
|
We were throwing two different errors for the same problematic:
* "Expected type {expected], got {type}" --> RPCTypeCheckArgument()
* "JSON value of type {type} is not of expected type {expected}" --> UniValue::checkType()
|
|
invalid input test coverage
2ef5294a5bb68ceb3797d2638567a172cc21699f rpc: add RPCTypeCheck for getblockfrompeer inputs (Jon Atack)
734b9669ff7b2f5e2820993443a6f868f6b0b20a test: add getblockfrompeer coverage of invalid inputs (Jon Atack)
Pull request description:
The new getblockfrompeer RPC lacks test coverage for invalid arguments, and its error messages are not harmonized with the existing RPCs.
Fix all issues.
ACKs for top commit:
brunoerg:
ACK 2ef5294a5bb68ceb3797d2638567a172cc21699f
Tree-SHA512: 454782cf6a44fd0e05483bb152153667ef5c8021358385ddcf89724fbbbd35e187362bdff757e00c99319527bc4c0b20c7187f67241d4585d767a29787142f25
|
|
|
|
|
|
|
|
|
|
923312fbf6a89efde1739da0b7209694d4f892ba rpc: use peer_id, block_hash for FetchBlock (Sjors Provoost)
34d5399211eeb61e7e7961c301fb2ddea8aa3f6a rpc: more detailed errors for getblockfrompeer (Sjors Provoost)
60243cac7286e4c4bdda7094bef4cf6d1564b583 rpc: turn already downloaded into error in getblockfrompeer (Sjors Provoost)
809d66bb65aa78048e27c2a878d6f7becaecfe11 rpc: clarify getblockfrompeer behavior when called multiple times (Sjors Provoost)
0e3d7c5ee16d5a4c061ab9a57285bceb7899b512 refactor: drop redundant hash argument from FetchBlock (Sjors Provoost)
8d1a3e6498de6087501969a9d243b0697ca3fe97 rpc: allow empty JSON object result (Sjors Provoost)
bfbf91d0b2004dde358253ac174982f784b43b59 test: fancier Python for getblockfrompeer (Sjors Provoost)
Pull request description:
Followups from #20295.
ACKs for top commit:
jonatack:
ACK 923312fbf6a89efde1739da0b7209694d4f892ba :package:
fjahr:
tested ACK 923312fbf6a89efde1739da0b7209694d4f892ba
Tree-SHA512: da9eca76e302e249409c9d7f0d16cca668ed981e2ab6ca2d1743dad0d830b94b1bc5ffb9028a00764b863201945c273cc8f4409a4c9ca3817830007dffa2bc20
|
|
-BEGIN VERIFY SCRIPT-
./contrib/devtools/copyright_header.py update ./
-END VERIFY SCRIPT-
Commits of previous years:
* 2020: fa0074e2d82928016a43ca408717154a1c70a4db
* 2019: aaaaad6ac95b402fe18d019d67897ced6b316ee0
|
|
|
|
|
|
Co-authored-by: MarcoFalke <falke.marco@gmail.com>
|
|
Co-authored-by: John Newbery <john@johnnewbery.com>
|
