diff options
| author | Wladimir J. van der Laan <laanwj@protonmail.com> | 2019-10-08 10:41:58 +0200 |
|---|---|---|
| committer | Wladimir J. van der Laan <laanwj@protonmail.com> | 2019-10-08 10:42:04 +0200 |
| commit | df50fd194fb804edba8a513b1eb699de38ca3963 (patch) | |
| tree | 64ee0dfa24851c502a5c7c0bf41e5ab7670b7ed7 /contrib/linearize | |
| parent | 866fd2888f42f8d81e3b1ab5eb647c86102158ac (diff) | |
| parent | 3284e6c09a84e9557ec72723ad636053d3ef7122 (diff) | |
Merge #16802: scripts: In linearize, search for next position of magic bytes rather than fail
3284e6c09a84e9557ec72723ad636053d3ef7122 scripts: search for next position of magic bytes rather than fail (Tim Akinbo)
Pull request description:
When using the `linearize-data.py` contrib script to export block data, there are edge cases where the script fails with an `Invalid magic: 00000000` error. This error occurs due to the presence of padding bytes that occasionally appears between consecutive blocks in the block data file.
There's an ongoing conversation about this in #14986. sipa also admitted that it is a bug in #5028. Fortunately, this is not an issue in bitcoin core as it handles this type of situation gracefully and so no fix in bitcoin core is required.
This PR is an improvement on how the script handles these "invalid magic bytes". Rather than failing, this patch allows the script to search for the next occurrence of the magic bytes and then starts reading the block from there.
ACKs for top commit:
laanwj:
ACK 3284e6c09a84e9557ec72723ad636053d3ef7122
Tree-SHA512: 18067ae0b4b62e822dfc558a86439ad6acaf939b98479e38e8e4248536574643b26eb48e96ec7139375c88b42cbe7705a64deb13a3c239e16025a6aad3d69bfa
Diffstat (limited to 'contrib/linearize')
| -rwxr-xr-x | contrib/linearize/linearize-data.py | 7 |
1 files changed, 5 insertions, 2 deletions
diff --git a/contrib/linearize/linearize-data.py b/contrib/linearize/linearize-data.py index 95754ab937..863b22f6b1 100755 --- a/contrib/linearize/linearize-data.py +++ b/contrib/linearize/linearize-data.py @@ -213,8 +213,11 @@ class BlockDataCopier: inMagic = inhdr[:4] if (inMagic != self.settings['netmagic']): - print("Invalid magic: " + inMagic.hex()) - return + # Seek backwards 7 bytes (skipping the first byte in the previous search) + # and continue searching from the new position if the magic bytes are not + # found. + self.inF.seek(-7, os.SEEK_CUR) + continue inLenLE = inhdr[4:] su = struct.unpack("<I", inLenLE) inLen = su[0] - 80 # length without header |