diff options
51 files changed, 2389 insertions, 431 deletions
diff --git a/README.mediawiki b/README.mediawiki index 3e9215d..b745f51 100644 --- a/README.mediawiki +++ b/README.mediawiki @@ -23,7 +23,7 @@ Those proposing changes should consider that ultimately consent may rest with th | BIP Status and Comments | Luke Dashjr | Process -| Draft +| Deferred |- | [[bip-0009.mediawiki|9]] | Version bits with timeout and delay @@ -36,12 +36,12 @@ Those proposing changes should consider that ultimately consent may rest with th | Alan Reiner | Informational | Withdrawn -|- style="background-color: #ffffcf" +|- style="background-color: #cfffcf" | [[bip-0011.mediawiki|11]] | M-of-N Standard Transactions | Gavin Andresen | Standard -| Accepted +| Final |- style="background-color: #ffcfcf" | [[bip-0012.mediawiki|12]] | OP_EVAL @@ -54,12 +54,12 @@ Those proposing changes should consider that ultimately consent may rest with th | Gavin Andresen | Standard | Final -|- style="background-color: #ffffcf" +|- style="background-color: #cfffcf" | [[bip-0014.mediawiki|14]] | Protocol Version and User Agent | Amir Taaki, Patrick Strateman | Standard -| Accepted +| Final |- | [[bip-0015.mediawiki|15]] | Aliases @@ -96,72 +96,72 @@ Those proposing changes should consider that ultimately consent may rest with th | Luke Dashjr | Standard | Replaced -|- style="background-color: #ffffcf" +|- style="background-color: #cfffcf" | [[bip-0021.mediawiki|21]] | URI Scheme | Nils Schneider, Matt Corallo | Standard -| Accepted -|- style="background-color: #ffffcf" +| Final +|- style="background-color: #cfffcf" | [[bip-0022.mediawiki|22]] | getblocktemplate - Fundamentals | Luke Dashjr | Standard -| Accepted -|- style="background-color: #ffffcf" +| Final +|- style="background-color: #cfffcf" | [[bip-0023.mediawiki|23]] | getblocktemplate - Pooled Mining | Luke Dashjr | Standard -| Accepted +| Final |- style="background-color: #cfffcf" | [[bip-0030.mediawiki|30]] | Duplicate transactions | Pieter Wuille | Standard | Final -|- style="background-color: #ffffcf" +|- style="background-color: #cfffcf" | [[bip-0031.mediawiki|31]] | Pong message | Mike Hearn | Standard -| Accepted -|- style="background-color: #ffffcf" +| Final +|- style="background-color: #cfffcf" | [[bip-0032.mediawiki|32]] | Hierarchical Deterministic Wallets | Pieter Wuille | Informational -| Accepted +| Final |- | [[bip-0033.mediawiki|33]] | Stratized Nodes | Amir Taaki | Standard | Draft -|- style="background-color: #ffffcf" +|- style="background-color: #cfffcf" | [[bip-0034.mediawiki|34]] | Block v2, Height in Coinbase | Gavin Andresen | Standard -| Accepted -|- style="background-color: #ffffcf" +| Final +|- style="background-color: #cfffcf" | [[bip-0035.mediawiki|35]] | mempool message | Jeff Garzik | Standard -| Accepted +| Final |- | [[bip-0036.mediawiki|36]] | Custom Services | Stefan Thomas | Standard | Draft -|- style="background-color: #ffffcf" +|- style="background-color: #cfffcf" | [[bip-0037.mediawiki|37]] | Connection Bloom filtering | Mike Hearn, Matt Corallo | Standard -| Accepted +| Final |- | [[bip-0038.mediawiki|38]] | Passphrase-protected private key @@ -216,12 +216,12 @@ Those proposing changes should consider that ultimately consent may rest with th | Justus Ranvier | Informational | Draft -|- +|- style="background-color: #cfffcf" | [[bip-0050.mediawiki|50]] | March 2013 Chain Fork Post-Mortem | Gavin Andresen | Informational -| Draft +| Final <!-- 50 series reserved for a group of post-mortems --> |- | [[bip-0060.mediawiki|60]] @@ -253,12 +253,12 @@ Those proposing changes should consider that ultimately consent may rest with th | Mike Hearn | Standard | Draft -|- style="background-color: #ffffcf" +|- style="background-color: #cfffcf" | [[bip-0065.mediawiki|65]] | OP_CHECKLOCKTIMEVERIFY | Peter Todd | Standard -| Accepted +| Final |- style="background-color: #cfffcf" | [[bip-0066.mediawiki|66]] | Strict DER signatures @@ -301,12 +301,12 @@ Those proposing changes should consider that ultimately consent may rest with th | Gavin Andresen | Standard | Final -|- +|- style="background-color: #cfffcf" | [[bip-0073.mediawiki|73]] | Use "Accept" header for response type negotiation with Payment Request URLs | Stephen Pair | Standard -| Draft +| Final |- | [[bip-0074.mediawiki|74]] | Allow zero value OP_RETURN in Payment Protocol @@ -314,6 +314,12 @@ Those proposing changes should consider that ultimately consent may rest with th | Standard | Draft |- +| [[bip-0075.mediawiki|75]] +| Out of Band Address Exchange using Payment Protocol Encryption +| Justin Newton, Matt David, Aaron Voisine, James MacWhyte +| Standard +| Draft +|- | [[bip-0080.mediawiki|80]] | Hierarchy for Non-Colored Voting Pool Deterministic Multisig Wallets | Justus Ranvier, Jimmy Song @@ -337,12 +343,12 @@ Those proposing changes should consider that ultimately consent may rest with th | Jorge Timón | Informational | Draft -|- +|- style="background-color: #ffcfcf" | [[bip-0101.mediawiki|101]] | Increase maximum block size | Gavin Andresen | Standard -| Draft +| Withdrawn |- | [[bip-0102.mediawiki|102]] | Block size increase to 2MB @@ -374,6 +380,12 @@ Those proposing changes should consider that ultimately consent may rest with th | Standard | Draft |- +| [[bip-0109.mediawiki|109]] +| Two million byte size limit with sigop and sighash limits +| Gavin Andresen +| Standard +| Draft +|- | [[bip-0111.mediawiki|111]] | NODE_BLOOM service bit | Matt Corallo, Peter Todd @@ -392,6 +404,12 @@ Those proposing changes should consider that ultimately consent may rest with th | Standard | Draft |- +| [[bip-0114.mediawiki|114]] +| Merkelized Abstract Syntax Tree +| Johnson Lau +| Standard +| Draft +|- | [[bip-0120.mediawiki|120]] | Proof of Payment | Kalle Rosenbaum @@ -428,6 +446,12 @@ Those proposing changes should consider that ultimately consent may rest with th | Standard | Draft |- +| [[bip-0126.mediawiki|126]] +| Best Practices for Heterogeneous Input Script Transactions +| Kristov Atlas +| Informational +| Draft +|- | [[bip-0130.mediawiki|130]] | sendheaders message | Suhas Daftuar @@ -446,6 +470,12 @@ Those proposing changes should consider that ultimately consent may rest with th | Process | Draft |- +| [[bip-0133.mediawiki|133]] +| feefilter message +| Alex Morcos +| Standard +| Draft +|- | [[bip-0140.mediawiki|140]] | Normalized TXID | Christian Decker @@ -462,7 +492,7 @@ Those proposing changes should consider that ultimately consent may rest with th | Address Format for Segregated Witness | Johnson Lau | Standard -| Draft +| Deferred |- | [[bip-0143.mediawiki|143]] | Transaction Signature Verification for Version 0 Witness Program @@ -481,6 +511,18 @@ Those proposing changes should consider that ultimately consent may rest with th | Luke Dashjr | Standard | Draft +|- +| [[bip-0151.mediawiki|151]] +| Peer-to-Peer Communication Encryption +| Jonas Schnelli +| Standard +| Draft +|- +| [[bip-0152.mediawiki|152]] +| Compact Block Relay +| Matt Corallo +| Standard +| Draft |} <!-- IMPORTANT! See the instructions at the top of this page, do NOT JUST add BIPs here! --> diff --git a/bip-0001.mediawiki b/bip-0001.mediawiki index 4f91a39..44fbe8b 100644 --- a/bip-0001.mediawiki +++ b/bip-0001.mediawiki @@ -85,9 +85,9 @@ Each BIP should have the following parts: ==BIP Formats and Templates== -BIPs should be written in mediawiki or markdown format. Image files should be included in a subdirectory for that BIP. +BIPs should be written in mediawiki or markdown format. -==BIP Header Preamble== +===BIP Header Preamble=== Each BIP must begin with an RFC 822 style header preamble. The headers must appear in the following order. Headers marked with "*" are optional and are described below. All other headers are required. @@ -129,9 +129,10 @@ The Created header records the date that the BIP was assigned a number, while Po BIPs may have a Requires header, indicating the BIP numbers that this BIP depends on. BIPs may also have a Superseded-By header indicating that a BIP has been rendered obsolete by a later document; the value is the number of the BIP that replaces the current document. The newer BIP must have a Replaces header containing the number of the BIP that it rendered obsolete. -Auxiliary Files -BIPs may include auxiliary files such as diagrams. Such files must be named BIP-XXXX-Y.ext, where "XXXX" is the BIP number, "Y" is a serial number (starting at 1), and "ext" is replaced by the actual file extension (e.g. "png"). +===Auxiliary Files=== + +BIPs may include auxiliary files such as diagrams. Image files should be included in a subdirectory for that BIP. Auxiliary files must be named BIP-XXXX-Y.ext, where "XXXX" is the BIP number, "Y" is a serial number (starting at 1), and "ext" is replaced by the actual file extension (e.g. "png"). ==Transferring BIP Ownership== @@ -175,6 +176,6 @@ This document was derived heavily from Python's PEP-0001. In many places text wa ==Changelog== -10 Oct 2015 - Added clarifications about sumission process and BIP number assignment. +10 Oct 2015 - Added clarifications about submission process and BIP number assignment. 01 Jan 2016 - Clarified early stages of BIP idea championing, collecting community feedback, etc. diff --git a/bip-0002.mediawiki b/bip-0002.mediawiki index 571361c..9d59405 100644 --- a/bip-0002.mediawiki +++ b/bip-0002.mediawiki @@ -2,7 +2,7 @@ BIP: 2 Title: BIP Status and Comments Author: Luke Dashjr <luke+bip@dashjr.org> - Status: Draft + Status: Deferred Type: Process Created: 2016-02-03 </pre> @@ -35,7 +35,7 @@ A process BIP may change status from Draft to Active when it achieves rough cons See [[bip-0123.mediawiki|BIP 123]] for definitions of the various BIP layers. Activation of this BIP implies activation of BIP 123. -A soft-fork BIP strictly requires a clear miner majority expressed by blockchain voting (eg, using BIP 9). In addition, if the economy seems willing to make a "no confidence" hard-fork (such as a change in proof-of-work algorithm), the soft-fork does not become Final for as long as such a hard-fork has potentially-majority support, or at most three months. Soft-fork BIPs may also set additional requirements for their adoption. Because of the possibility of changes to miner dynamics, especially in light of delegated voting (mining pools), it is highly recommended that a supermajority vote around 95% be required by the BIP itself, unless rationale is given for a lower threshold. +A soft-fork BIP strictly requires a clear miner majority expressed by blockchain voting (eg, using BIP 9). In addition, if the economy seems willing to make a "no confidence" hard-fork (such as a change in proof-of-work algorithm), the soft-fork does not become Final for as long as such a hard-fork might have majority support, or at most three months. Soft-fork BIPs may also set additional requirements for their adoption. Because of the possibility of changes to miner dynamics, especially in light of delegated voting (mining pools), it is highly recommended that a supermajority vote around 95% be required by the BIP itself, unless rationale is given for a lower threshold. A hard-fork BIP requires adoption from the entire Bitcoin economy, particularly including those selling desirable goods and services in exchange for bitcoin payments, as well as Bitcoin holders who wish to spend or would spend their bitcoins (including selling for other currencies) differently in the event of such a hard-fork. Adoption must be expressed by de facto usage of the hard-fork in practice (ie, not merely expressing public support, although that is a good step to establish agreement before adoption of the BIP). This economic adoption cannot be established merely by a super-majority, except by literally forcing the minority to accept the hard-fork (whether this is viable or not is outside the scope of this document). @@ -68,6 +68,15 @@ But they're doing something important and have invested a lot in Bitcoin! Should * This BIP does not aim to address what "should" be the basis of decisions. Such a statement, no matter how perfect in its justification, would be futile without some way to force others to use it. The BIP process does not aim to be a kind of forceful "governance" of Bitcoin, merely to provide a collaborative repository for proposing and providing information on standards, which people may voluntarily adopt or not. It can only hope to achieve accuracy in regard to the "Status" field by striving to reflect the reality of *how things actually are*, rather than *how they should be*. +What if a single merchant wishes to block a hard-fork? + +* This BIP addresses only the progression of the BIP Status field, not the deployment of the hard-fork (or any other change) itself. +* Regardless, one shop cannot operate in a vacuum: if they are indeed alone, they will soon find themselves no longer selling in exchange for bitcoin payments, as nobody else would exist willing to use the previous blockchain to pay them. If they are no longer selling, they cease to meet the criteria herein which enables their veto. + +How about a small number of merchants (maybe only two) who sell products to each other? + +* In this scenario, it would seem the previous Bitcoin is alive any working, and that the hard-fork has failed. How to resolve such a split is outside the scope of this BIP. + How can economic agreement veto a soft-fork? * The group of miners is determined by the consensus rules for the dynamic-membership multi-party signature (for Bitcoin, the proof-of-work algorithm), which can be modified with a hard-fork. Thus, if the same conditions required to modify this group are met in opposition to a soft-fork, the miner majority supporting the soft-fork is effectively void because the economy can decide to replace them with another group of would-be miners who do not support the soft-fork. @@ -118,6 +127,8 @@ For example, the preamble to BIP 1 might be updated to include the line: Comments-URI: https://en.bitcoin.it/wiki/BIP_Comments:BIP_0001 https://some-other-wiki.org/BIP_1_Comments +These fields must follow the "Discussions-To" header defined in BIP 1 (if that header is not present, it should follow the position where it would be present; generally this is immediately above the Status header). + To avoid doubt: comments and status are unrelated metrics to judge a BIP, and neither should be directly influencing the other. ===Rationale=== @@ -133,31 +144,60 @@ Will BIP comments be censored or limited to particular participants/"experts"? ==BIP licensing== -New BIPs may be accepted with the following licenses: - ===Specification=== +New BIPs may be accepted with the following licenses. Each new BIP must identify at least one acceptable license in its preamble. The License header in the preamble must be placed after the Created header. Each license must be referenced by their respecitve abbreviation given below. + +For example, a preamble might include the following License header: + + License: BSD-2-Clause + GNU-All-Permissive + +In this case, the BIP text is fully licensed under both the OSI-approved BSD 2-clause license as well as the GNU All-Permissive License, and anyone may modify and redistribute the text provided they comply with the terms of *either* license. In other words, the license list is an "OR choice", not an "AND also" requirement. + +It is also possible to license source code differently from the BIP text. A optional License-Code header is placed after the License header. Again, each license must be referenced by their respective abbreviation given below. + +For example, a preamble specifying the optional License-Code header might look like: + + License: BSD-2-Clause + GNU-All-Permissive + License-Code: GPL-2.0+ + +In this case, the code in the BIP is not available under the BSD or All-Permissive licenses, but only under the terms of the GNU General Public License (GPL), version 2 or newer. +If the code were to be available under *only* version 2 exactly, the "+" symbol should be removed from the license abbreviation. +For a later version (eg, GPL 3.0), you would increase the version number (and retain or remove the "+" depending on intent). + + License-Code: GPL-2.0 # This refers to GPL v2.0 *only*, no later license versions are acceptable. + License-Code: GPL-2.0+ # This refers to GPL v2.0 *or later*. + License-Code: GPL-3.0 # This refers to GPL v3.0 *only*, no later license versions are acceptable. + License-Code: GPL-3.0+ # This refers to GPL v3.0 *or later*. + +In the event that the text or code is not available under common license terms, the list should instead be replaced with the single term "Complex", and full details provided in the Copyright section of the BIP. + ====Recommended licenses==== -* [https://opensource.org/licenses/BSD-2-Clause OSI-approved BSD 2-clause license] -* [https://opensource.org/licenses/BSD-3-Clause OSI-approved BSD 3-clause license] -* [https://creativecommons.org/publicdomain/zero/1.0/ Creative Commons CC0 1.0 Universal] -* [http://www.gnu.org/prep/maintain/html_node/License-Notices-for-Other-Files.html GNU All-Permissive License] +* BSD-2-Clause: [https://opensource.org/licenses/BSD-2-Clause OSI-approved BSD 2-clause license] +* BSD-3-Clause: [https://opensource.org/licenses/BSD-3-Clause OSI-approved BSD 3-clause license] +* CC0-1.0: [https://creativecommons.org/publicdomain/zero/1.0/ Creative Commons CC0 1.0 Universal] +* GNU-All-Permissive: [http://www.gnu.org/prep/maintain/html_node/License-Notices-for-Other-Files.html GNU All-Permissive License] In addition, it is recommended that literal code included in the BIP be dual-licensed under the same license terms as the project it modifies. For example, literal code intended for Bitcoin Core would ideally be dual-licensed under the MIT license terms as well as one of the above with the rest of the BIP text. ====Not recommended, but acceptable licenses==== -* [http://www.apache.org/licenses/LICENSE-2.0 Apache License, version 2.0] -* [http://www.boost.org/LICENSE_1_0.txt Boost Software License, version 1.0] -* [https://creativecommons.org/licenses/by/4.0/ Creative Commons Attribution 4.0 International] -* [https://creativecommons.org/licenses/by-sa/4.0/ Creative Commons Attribution-ShareAlike 4.0 International] -* [https://opensource.org/licenses/MIT Expat/MIT/X11 license] -* [http://www.gnu.org/licenses/agpl-3.0.en.html GNU Affero General Public License (AGPL), version 3 or newer] -* [http://www.gnu.org/licenses/fdl-1.3.en.html GNU Free Documentation License] -* [http://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html GNU General Public License (GPL), version 2 or newer] -* [http://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html GNU Lesser General Public License (LGPL), version 2.1 or newer] -* [http://opencontent.org/openpub/ Open Publication License, version 1.0] +* Apache-2.0: [http://www.apache.org/licenses/LICENSE-2.0 Apache License, version 2.0] +* BSL-1.0: [http://www.boost.org/LICENSE_1_0.txt Boost Software License, version 1.0] +* CC-BY-4.0: [https://creativecommons.org/licenses/by/4.0/ Creative Commons Attribution 4.0 International] +* CC-BY-SA-4.0: [https://creativecommons.org/licenses/by-sa/4.0/ Creative Commons Attribution-ShareAlike 4.0 International] +* MIT: [https://opensource.org/licenses/MIT Expat/MIT/X11 license] +* AGPL-3.0+: [http://www.gnu.org/licenses/agpl-3.0.en.html GNU Affero General Public License (AGPL), version 3 or newer] +* FDL-1.3: [http://www.gnu.org/licenses/fdl-1.3.en.html GNU Free Documentation License, version 1.3] +* GPL-2.0+: [http://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html GNU General Public License (GPL), version 2 or newer] +* LGPL-2.1+: [http://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html GNU Lesser General Public License (LGPL), version 2.1 or newer] +* OPL: [http://opencontent.org/openpub/ Open Publication License, version 1.0] + +Additionally, PD is used to express that the work is placed in the public domain. +This may not be used for new BIPs, and is only defined for use by BIPs predating acceptance of this BIP. ===Rationale=== @@ -172,8 +212,12 @@ Why are there software licenses included? * Some BIPs, especially consensus layer, may include literal code in the BIP itself which may not be available under the exact license terms of the BIP. * Despite this, not all software licenses would be acceptable for content included in BIPs. +Why is Public Domain no longer acceptable for new BIPs? + +* In some jurisdictions, public domain is not recognised as a legitimate legal action, leaving the BIP simply copyrighted with no redistribution or modification allowed at all. + ==See Also== * [[bip-0001.mediawiki|BIP 1: BIP Purpose and Guidelines]] * [[bip-0123.mediawiki|BIP 123: BIP Classification]] -* [https://tools.ietf.org/html/rfc7282 RBF 7282: On Consensus and Humming in the IETF] +* [https://tools.ietf.org/html/rfc7282 RFC 7282: On Consensus and Humming in the IETF] diff --git a/bip-0009.mediawiki b/bip-0009.mediawiki index 78298f0..7270abd 100644 --- a/bip-0009.mediawiki +++ b/bip-0009.mediawiki @@ -18,135 +18,191 @@ This document specifies a proposed change to the semantics of the 'version' fiel BIP 34 introduced a mechanism for doing soft-forking changes without a predefined flag timestamp (or flag block height), instead relying on measuring miner support indicated by a higher version number in block headers. As it relies on comparing version numbers as integers however, it only supports one single change being rolled out at once, requiring coordination between proposals, and does not allow for permanent rejection: as long as one soft fork is not fully rolled out, no future one can be scheduled. -In addition, BIP 34 made the integer comparison (nVersion >= 2) a consensus rule after its 95% threshold was reached, removing 2<sup>31</sup>+2 values from the set of valid version numbers (all negative numbers, as nVersion is interpreted as a signed integer, as well as 0 and 1). This indicates another downside this approach: every upgrade permanently restricts the set of allowed nVersion field values. This approach was later reused in BIP 66, which further removed nVersion = 2 as valid option. As will be shown further, this is unnecessary. +In addition, BIP 34 made the integer comparison (nVersion >= 2) a consensus rule after its 95% threshold was reached, removing 2<sup>31</sup>+2 values from the set of valid version numbers (all negative numbers, as nVersion is interpreted as a signed integer, as well as 0 and 1). This indicates another downside this approach: every upgrade permanently restricts the set of allowed nVersion field values. This approach was later reused in BIP 66 and BIP 65, which further removed nVersions 2 and 3 as valid options. As will be shown further, this is unnecessary. ==Specification== -===Mechanism=== +Each soft fork deployment is specified by the following per-chain parameters (further elaborated below): -'''Bit flags''' -We are permitting several independent soft forks to be deployed in parallel. For each, a bit B is chosen from the set {0,1,2,...,28}, which is not currently in use for any other ongoing soft fork. Miners signal intent to enforce the new rules associated with the proposed soft fork by setting bit 1<sup>B</sup> in nVersion to 1 in their blocks. +# The '''name''' specifies a very brief description of the soft fork, reasonable for use as an identifier. For deployments described in a single BIP, it is recommended to use the name "bipN" where N is the appropriate BIP number. +# The '''bit''' determines which bit in the nVersion field of the block is to be used to signal the soft fork lock-in and activation. It is chosen from the set {0,1,2,...,28}. +# The '''starttime''' specifies a minimum median time past of a block at which the bit gains its meaning. +# The '''timeout''' specifies a time at which the deployment is considered failed. If the median time past of a block >= timeout and the soft fork has not yet locked in (including this block's bit state), the deployment is considered failed on all descendants of the block. -'''High bits''' -The highest 3 bits are set to 001, so the range of actually possible nVersion values is [0x20000000...0x3FFFFFFF], inclusive. This leaves two future upgrades for different mechanisms (top bits 010 and 011), while complying to the constraints set by BIP34 and BIP66. Having more than 29 available bits for parallel soft forks does not add anything anyway, as the (nVersion >= 3) requirement already makes that impossible. +===Selection guidelines=== -'''States''' -With every softfork proposal we associate a state BState, which begins -at ''defined'', and can be ''locked-in'', ''activated'', -or ''failed''. Transitions are considered after each -retarget period. +The following guidelines are suggested for selecting these parameters for a soft fork: -'''Soft Fork Support''' -Software which supports the change should begin by setting B in all blocks -mined until it is resolved. +# '''name''' should be selected such that no two softforks, concurrent or otherwise, ever use the same name. +# '''bit''' should be selected such that no two concurrent softforks use the same bit. +# '''starttime''' should be set to some date in the future, approximately one month after a software release date including the soft fork. This allows for some release delays, while preventing triggers as a result of parties running pre-release software. +# '''timeout''' should be 1 year (31536000 seconds) after starttime. - if (BState != activated && BState != failed) { - SetBInBlock(); - } +A later deployment using the same bit is possible as long as the starttime is after the previous one's +timeout or activation, but it is discouraged until necessary, and even then recommended to have a pause in between to detect buggy software. + +===States=== + +With each block and soft fork, we associate a deployment state. The possible states are: + +# '''DEFINED''' is the first state that each soft fork starts out as. The genesis block is by definition in this state for each deployment. +# '''STARTED''' for blocks past the starttime. +# '''LOCKED_IN''' for one retarget period after the first retarget period with STARTED blocks of which at least threshold have the associated bit set in nVersion. +# '''ACTIVE''' for all blocks after the LOCKED_IN retarget period. +# '''FAILED''' for one retarget period past the timeout time, if LOCKED_IN was not reached. + +===Bit flags=== + +The nVersion block header field is to be interpreted as a 32-bit little-endian integer (as present), and bits are selected within this integer as values (1 << N) where N is the bit number. + +Blocks in the STARTED state get an nVersion whose bit position bit is set to 1. The top 3 bits of such blocks must be +001, so the range of actually possible nVersion values is [0x20000000...0x3FFFFFFF], inclusive. + +Due to the constraints set by BIP 34, BIP 66 and BIP 65, we only have 0x7FFFFFFB possible nVersion values available. +This restricts us to at most 30 independent deployments. By restricting the top 3 bits to 001 we get 29 out of those +for the purposes of this proposal, and support two future upgrades for different mechanisms (top bits 010 and 011). +When a block nVersion does not have top bits 001, it is treated as if all +bits are 0 for the purposes of deployments. + +Miners should continue setting the bit in LOCKED_IN phase so uptake is visible, though this has no effect on +consensus rules. + +===New consensus rules=== -'''Success: Lock-in Threshold''' -If bit B is set in 1916 (1512 on testnet) or -more of the 2016 blocks within a retarget period, it is considered -''locked-in''. Miners should continue setting bit B, so uptake is -visible. +The new consensus rules for each soft fork are enforced for each block that has ACTIVE state. - if (NextBlockHeight % 2016 == 0) { - if (BState == defined && Previous2016BlocksCountB() >= 1916) { - BState = locked-in; - BActiveHeight = NextBlockHeight + 2016; +===State transitions=== + +<img src="bip-0009/states.png" align="middle"></img> + +The genesis block has state DEFINED for each deployment, by definition. + + State GetStateForBlock(block) { + if (block.height == 0) { + return DEFINED; + } + +All blocks within a retarget period have the same state. This means that if +floor(block1.height / 2016) = floor(block2.height / 2016), they are guaranteed to have the same state for every +deployment. + + if ((block.height % 2016) != 0) { + return GetStateForBlock(block.parent); + } + +Otherwise, the next state depends on the previous state: + + switch (GetStateForBlock(GetAncestorAtHeight(block, block.height - 2016))) { + +We remain in the initial state until either we pass the start time or the timeout. GetMedianTimePast in the code below +refers to the median nTime of a block and its 10 predecessors. The expression GetMedianTimePast(block.parent) is +referred to as MTP in the diagram above, and is treated as a monotonic clock defined by the chain. + + case DEFINED: + if (GetMedianTimePast(block.parent) >= timeout) { + return FAILED; + } + if (GetMedianTimePast(block.parent) >= starttime) { + return STARTED; + } + return DEFINED; + +After a period in the STARTED state, if we're past the timeout, we switch to FAILED. If not, we tally the bits set, +and transition to LOCKED_IN if a sufficient number of blocks in the past period set the deployment bit in their +version numbers. The threshold is ≥1916 blocks (95% of 2016), or ≥1512 for testnet (75% of 2016). +The transition to FAILED takes precendence, as otherwise an ambiguity can arise. +There could be two non-overlapping deployments on the same bit, where the first one transitions to LOCKED_IN while the +other one simultaneously transitions to STARTED, which would mean both would demand setting the bit. + +Note that a block's state never depends on its own nVersion; only on that of its ancestors. + + case STARTED: + if (GetMedianTimePast(block.parent) >= timeout) { + return FAILED; + } + int count = 0; + walk = block; + for (i = 0; i < 2016; i++) { + walk = walk.parent; + if (walk.nVersion & 0xE0000000 == 0x20000000 && (walk.nVersion >> bit) & 1 == 1) { + count++; + } + } + if (count >= threshold) { + return LOCKED_IN; + } + return STARTED; + +After a retarget period of LOCKED_IN, we automatically transition to ACTIVE. + + case LOCKED_IN: + return ACTIVE; + +And ACTIVE and FAILED are terminal states, which a deployment stays in once they're reached. + + case ACTIVE: + return ACTIVE; + + case FAILED: + return FAILED; } } -'''Success: Activation Delay''' -The consensus rules related to ''locked-in'' soft fork will be enforced in -the second retarget period; ie. there is a one retarget period in -which the remaining 5% can upgrade. At the that activation block and -after, miners should stop setting bit B, which may be reused for a different soft fork. - - if (BState == locked-in && NextBlockHeight == BActiveHeight) { - BState = activated; - ApplyRulesForBFromNextBlock(); - /* B can be reused, immediately */ - } - -'''Failure: Timeout''' -A soft fork proposal should include a ''timeout''. This is measured -as the beginning of a calendar year as per this table (suggest -adding three to the current calendar year when drafting the soft fork proposal): - -{| -! Timeout Year -! >= Seconds -! Timeout Year -! >= Seconds -|- -|2018 -|1514764800 -|2026 -|1767225600 -|- -|2019 -|1546300800 -|2027 -|1798761600 +'''Implementation''' +It should be noted that the states are maintained along block chain +branches, but may need recomputation when a reorganization happens. + +Given that the state for a specific block/deployment combination is completely determined by its ancestry before the +current retarget period (i.e. up to and including its ancestor with height block.height - 1 - (block.height % 2016)), +it is possible to implement the mechanism above efficiently and safely by caching the resulting state of every multiple-of-2016 +block, indexed by its parent. + +===Warning mechanism=== + +To support upgrade warnings, an extra "unknown upgrade" is tracked, using the "implicit bit" mask = (block.nVersion & ~expectedVersion) != 0. Mask will be non-zero whenever an unexpected bit is set in nVersion. Whenever LOCKED_IN for the unknown upgrade is detected, the software should warn loudly about the upcoming soft fork. It should warn even more loudly after the next retarget period (when the unknown upgrade is in the ACTIVE state). + +===getblocktemplate changes=== + +The template request Object is extended to include a new item: + +{| class="wikitable" +!colspan=4| template request |- -|2020 -|1577836800 -|2028 -|1830297600 +! Key !! Required !! Type !! Description |- -|2021 -|1609459200 -|2029 -|1861920000 +| rules || No || Array of Strings || list of supported softfork deployments, by name +|} + +The template Object is also extended: + +{| class="wikitable" +!colspan=4| template |- -|2022 -|1640995200 -|2030 -|1893456000 +! Key !! Required !! Type !! Description |- -|2023 -|1672531200 -|2031 -|1924992000 +| rules || Yes || Array of Strings || list of softfork deployments, by name, that are active state |- -|2024 -|1704067200 -|2032 -|1956528000 +| vbavailable || Yes || Object || set of pending, supported softfork deployments; each uses the softfork name as the key, and the softfork bit as its value |- -|2025 -|1735689600 -|2033 -|1988150400 +| vbrequired || No || Number || bit mask of softfork deployment version bits the server requires enabled in submissions |} -If the soft fork still not ''locked-in'' and the -GetMedianTimePast() of a block following a retarget period is at or -past this timeout, miners should cease setting this bit. - - if (NextBlockHeight % 2016 == 0) { - if (BState == defined && GetMedianTimePast(nextblock) >= BFinalYear) { - BState = failed; - } - } - -After another retarget period (to allow detection of buggy miners), -the bit may be reused. +The "version" key of the template is retained, and used to indicate the server's preference of deployments. +If versionbits is being used, "version" MUST be within the versionbits range of [0x20000000...0x3FFFFFFF]. +Miners MAY clear or set bits in the block version WITHOUT any special "mutable" key, provided they are listed among the template's "vbavailable" and (when clearing is desired) NOT included as a bit in "vbrequired". -'''Warning system''' -To support upgrade warnings, an extra "unknown upgrade" is tracked, using the "implicit bit" mask = (block.nVersion & ~expectedVersion) != 0. Mask will be non-zero whenever an unexpected bit is set in nVersion. Whenever lock-in for the unknown upgrade is detected, the software should warn loudly about the upcoming soft fork. It should warn even more loudly after the next retarget period. +Softfork deployment names listed in "rules" or as keys in "vbavailable" may be prefixed by a '!' character. +Without this prefix, GBT clients may assume the rule will not impact usage of the template as-is; typical examples of this would be when previously valid transactions cease to be valid, such as BIPs 16, 65, 66, 68, 112, and 113. +If a client does not understand a rule without the prefix, it may use it unmodified for mining. +On the other hand, when this prefix is used, it indicates a more subtle change to the block structure or generation transaction; examples of this would be BIP 34 (because it modifies coinbase construction) and 141 (since it modifies the txid hashing and adds a commitment to the generation transaction). +A client that does not understand a rule prefixed by '!' must not attempt to process the template, and must not attempt to use it for mining even unmodified. -'''Forks''' -It should be noted that the states are maintained along block chain -branches, but may need recomputation when a reorganization happens. - -===Support for future changes=== +==Support for future changes== The mechanism described above is very generic, and variations are possible for future soft forks. Here are some ideas that can be taken into account. '''Modified thresholds''' -The 95% threshold (based on in BIP 34) does not have to be maintained for eternity, but changes should take the effect on the warning system into account. In particular, having a lock-in threshold that is incompatible with the one used for the warning system may have long-term effects, as the warning system cannot rely on a permanently detectable condition anymore. +The 1916 threshold (based on in BIP 34's 95%) does not have to be maintained for eternity, but changes should take the effect on the warning system into account. In particular, having a lock-in threshold that is incompatible with the one used for the warning system may have long-term effects, as the warning system cannot rely on a permanently detectable condition anymore. '''Conflicting soft forks''' At some point, two mutually exclusive soft forks may be proposed. The naive way to deal with this is to never create software that implements both, but that is making a bet that at least one side is guaranteed to lose. Better would be to encode "soft fork X cannot be locked-in" as consensus rule for the conflicting soft fork - allowing software that supports both, but can never trigger conflicting changes. @@ -166,6 +222,10 @@ The fallow period at the conclusion of a soft fork attempt allows some detection of buggy clients, and allows time for warnings and software upgrades for successful soft forks. +==Deployments== + +A living list of deployment proposals can be found [[bip-0009/assignments.mediawiki|here]]. + ==Copyright== This document is placed in the public domain. diff --git a/bip-0009/assignments.mediawiki b/bip-0009/assignments.mediawiki new file mode 100644 index 0000000..e145ceb --- /dev/null +++ b/bip-0009/assignments.mediawiki @@ -0,0 +1,37 @@ +==Deployments== + +List of proposed deployments. + +State can be defined, active, failed. Dates are in UTC. + +{| class="wikitable sortable" +! Name +! Bit +! Mainnet Start +! Mainnet Expire +! Mainnet State +! Testnet Start +! Testnet Expire +! Testnet State +! BIPs +|- +| csv +| 0 +| 2016-05-01 00:00:00 +| 2017-05-01 00:00:00 +| active since #419328 +| 2016-03-01 00:00:00 +| 2017-05-01 00:00:00 +| active since #770112 +| [[/bip-0068.mediawiki|68]], [[/bip-0112.mediawiki|112]], [[/bip-0113.mediawiki|113]] +|- +| segwit +| 1 +| TBD +| TBD +| - +| 2016-05-01 00:00:00 +| 2017-05-01 00:00:00 +| active since #834624 +| [[/bip-0141.mediawiki|141]], [[/bip-0143.mediawiki|143]] +|} diff --git a/bip-0009/states.png b/bip-0009/states.png Binary files differnew file mode 100644 index 0000000..09312a1 --- /dev/null +++ b/bip-0009/states.png diff --git a/bip-0011.mediawiki b/bip-0011.mediawiki index 2499ac0..4b12340 100644 --- a/bip-0011.mediawiki +++ b/bip-0011.mediawiki @@ -2,7 +2,7 @@ BIP: 11 Title: M-of-N Standard Transactions Author: Gavin Andresen <gavinandresen@gmail.com> - Status: Accepted + Status: Final Type: Standards Track Created: 2011-10-18 Post-History: 2011-10-02 diff --git a/bip-0014.mediawiki b/bip-0014.mediawiki index 2cfb327..f11cb63 100644 --- a/bip-0014.mediawiki +++ b/bip-0014.mediawiki @@ -3,7 +3,7 @@ Title: Protocol Version and User Agent Author: Amir Taaki <genjix@riseup.net> Patrick Strateman <bitcoin-bips@covertinferno.org> - Status: Accepted + Status: Final Type: Standards Track Created: 2011-11-10 Post-History: 2011-11-02 diff --git a/bip-0021.mediawiki b/bip-0021.mediawiki index 00d9a53..e7094e5 100644 --- a/bip-0021.mediawiki +++ b/bip-0021.mediawiki @@ -3,7 +3,7 @@ Title: URI Scheme Author: Nils Schneider <nils.schneider@gmail.com> Matt Corallo <bip21@bluematt.me> - Status: Accepted + Status: Final Type: Standards Track Created: 2012-01-29 </pre> diff --git a/bip-0022.mediawiki b/bip-0022.mediawiki index b39f957..4b33e59 100644 --- a/bip-0022.mediawiki +++ b/bip-0022.mediawiki @@ -2,7 +2,7 @@ BIP: 22 Title: getblocktemplate - Fundamentals Author: Luke Dashjr <luke+bip22@dashjr.org> - Status: Accepted + Status: Final Type: Standards Track Created: 2012-02-28 </pre> @@ -26,9 +26,9 @@ Block template creation can be influenced by various parameters: |- ! Key !! Required !! Type !! Description |- -| capabilities || {{No}} || Array of Strings || SHOULD contain a list of the following, to indicate client-side support: [[#Optional: Long Polling|"longpoll"]], "coinbasetxn", "coinbasevalue", [[bip-0023.mediawiki#Block Proposal|"proposal"]], [[bip-0023.mediawiki#Logical Services|"serverlist"]], "workid", and any of the [[bip-0023.mediawiki#Mutations|mutations]] +| capabilities || No || Array of Strings || SHOULD contain a list of the following, to indicate client-side support: [[#Optional: Long Polling|"longpoll"]], "coinbasetxn", "coinbasevalue", [[bip-0023.mediawiki#Block Proposal|"proposal"]], [[bip-0023.mediawiki#Logical Services|"serverlist"]], "workid", and any of the [[bip-0023.mediawiki#Mutations|mutations]] |- -| mode || {{No}} || String || MUST be "template" or omitted +| mode || No || String || MUST be "template" or omitted |} getblocktemplate MUST return a JSON Object containing the following keys: @@ -37,29 +37,29 @@ getblocktemplate MUST return a JSON Object containing the following keys: |- ! Key !! Required !! Type !! Description |- -| bits || {{Yes}} || String || the compressed difficulty in hexadecimal +| bits || Yes || String || the compressed difficulty in hexadecimal |- -| curtime || {{Yes}} || Number || the current time as seen by the server (recommended for block time) - note this is not necessarily the system clock, and must fall within the mintime/maxtime rules +| curtime || Yes || Number || the current time as seen by the server (recommended for block time) - note this is not necessarily the system clock, and must fall within the mintime/maxtime rules |- -| height || {{Yes}} || Number || the height of the block we are looking for +| height || Yes || Number || the height of the block we are looking for |- -| previousblockhash || {{Yes}} || String || the hash of the previous block, in big-endian hexadecimal +| previousblockhash || Yes || String || the hash of the previous block, in big-endian hexadecimal |- -| sigoplimit || {{No}} || Number || number of sigops allowed in blocks +| sigoplimit || No || Number || number of sigops allowed in blocks |- -| sizelimit || {{No}} || Number || number of bytes allowed in blocks +| sizelimit || No || Number || number of bytes allowed in blocks |- -| transactions || {{Yes|Should}} || Array of Objects || Objects containing [[#Transactions Object Format|information for Bitcoin transactions]] (excluding coinbase) +| transactions || Should || Array of Objects || Objects containing [[#Transactions Object Format|information for Bitcoin transactions]] (excluding coinbase) |- -| version || {{Yes}} || Number || always 1 or 2 (at least for bitcoin) - clients MUST understand the implications of the version they use (eg, comply with [[bip-0034.mediawiki|BIP 0034]] for version 2) +| version || Yes || Number || always 1 or 2 (at least for bitcoin) - clients MUST understand the implications of the version they use (eg, comply with [[bip-0034.mediawiki|BIP 0034]] for version 2) |- -| coinbaseaux || {{No}} || Object || data that SHOULD be included in the coinbase's scriptSig content. Only the values (hexadecimal byte-for-byte) in this Object should be included, not the keys. This does not include the block height, which is required to be included in the scriptSig by [[bip-0034.mediawiki|BIP 0034]]. It is advisable to encode values inside "PUSH" opcodes, so as to not inadvertently expend SIGOPs (which are counted toward limits, despite not being executed). +| coinbaseaux || No || Object || data that SHOULD be included in the coinbase's scriptSig content. Only the values (hexadecimal byte-for-byte) in this Object should be included, not the keys. This does not include the block height, which is required to be included in the scriptSig by [[bip-0034.mediawiki|BIP 0034]]. It is advisable to encode values inside "PUSH" opcodes, so as to not inadvertently expend SIGOPs (which are counted toward limits, despite not being executed). |- -| coinbasetxn || {{Patch|this or ↓}} || Object || [[#Transactions Object Format|information for coinbase transaction]] +| coinbasetxn || this or ↓ || Object || [[#Transactions Object Format|information for coinbase transaction]] |- -| coinbasevalue || {{Patch|this or ↑}} || Number || total funds available for the coinbase (in Satoshis) +| coinbasevalue || this or ↑ || Number || total funds available for the coinbase (in Satoshis) |- -| workid || {{No}} || String || if provided, this value must be returned with results (see [[#Block Submission|Block Submission]]) +| workid || No || String || if provided, this value must be returned with results (see [[#Block Submission|Block Submission]]) |} ==== Transactions Object Format ==== diff --git a/bip-0023.mediawiki b/bip-0023.mediawiki index a53b00b..874e60a 100644 --- a/bip-0023.mediawiki +++ b/bip-0023.mediawiki @@ -2,7 +2,7 @@ BIP: 23 Title: getblocktemplate - Pooled Mining Author: Luke Dashjr <luke+bip22@dashjr.org> - Status: Accepted + Status: Final Type: Standards Track Created: 2012-02-28 </pre> diff --git a/bip-0031.mediawiki b/bip-0031.mediawiki index 8adcd29..1bfe143 100644 --- a/bip-0031.mediawiki +++ b/bip-0031.mediawiki @@ -2,7 +2,7 @@ BIP: 31 Title: Pong message Author: Mike Hearn <hearn@google.com> - Status: Accepted + Status: Final Type: Standards Track Created: 2012-04-11 </pre> diff --git a/bip-0032.mediawiki b/bip-0032.mediawiki index ec5ddf4..8692c6d 100644 --- a/bip-0032.mediawiki +++ b/bip-0032.mediawiki @@ -8,7 +8,7 @@ RECENT CHANGES: BIP: 32 Title: Hierarchical Deterministic Wallets Author: Pieter Wuille <pieter.wuille@gmail.com> - Status: Accepted + Status: Final Type: Informational Created: 2012-02-11 </pre> diff --git a/bip-0034.mediawiki b/bip-0034.mediawiki index 3c0e770..4870fc1 100644 --- a/bip-0034.mediawiki +++ b/bip-0034.mediawiki @@ -2,7 +2,7 @@ BIP: 34 Title: Block v2, Height in Coinbase Author: Gavin Andresen <gavinandresen@gmail.com> - Status: Accepted + Status: Final Type: Standards Track Created: 2012-07-06 </pre> diff --git a/bip-0035.mediawiki b/bip-0035.mediawiki index cdadd1d..c66735c 100644 --- a/bip-0035.mediawiki +++ b/bip-0035.mediawiki @@ -2,7 +2,7 @@ BIP: 35 Title: mempool message Author: Jeff Garzik <jgarzik@exmulti.com> - Status: Accepted + Status: Final Type: Standards Track Created: 2012-08-16 </pre> diff --git a/bip-0037.mediawiki b/bip-0037.mediawiki index 224fef5..eba0628 100644 --- a/bip-0037.mediawiki +++ b/bip-0037.mediawiki @@ -2,8 +2,8 @@ BIP: 37 Title: Connection Bloom filtering Author: Mike Hearn <hearn@google.com> - Matt Corallo <bip@bluematt.me> - Status: Accepted + Matt Corallo <bip37@bluematt.me> + Status: Final Type: Standards Track Created: 2012-10-24 </pre> diff --git a/bip-0038.mediawiki b/bip-0038.mediawiki index 6107e0a..650b7d0 100644 --- a/bip-0038.mediawiki +++ b/bip-0038.mediawiki @@ -210,6 +210,9 @@ Added to alpha version of Casascius Bitcoin Address Utility for Windows availabl Click "Tools" then "PPEC Keygen" (provisional name) +==Other implementations== +* Javascript - https://github.com/bitcoinjs/bip38 + ==Test vectors== ===No compression, no EC multiply=== diff --git a/bip-0039.mediawiki b/bip-0039.mediawiki index c44ad3e..b553666 100644 --- a/bip-0039.mediawiki +++ b/bip-0039.mediawiki @@ -139,4 +139,4 @@ Haskell - https://github.com/haskoin/haskoin .NET C# (PCL) - https://github.com/NicolasDorier/NBitcoin -JavaScript - https://github.com/bitpay/bitcore-mnemonic +JavaScript - https://github.com/bitpay/bitcore-mnemonic, https://github.com/bitcoinjs/bip39 diff --git a/bip-0044.mediawiki b/bip-0044.mediawiki index 6012ff5..883677a 100644 --- a/bip-0044.mediawiki +++ b/bip-0044.mediawiki @@ -267,7 +267,7 @@ is required and a pull request to the above file should be created. * [[https://play.google.com/store/apps/details?id=com.mycelium.wallet|Mycelium Bitcoin Wallet (Android)]] ([[https://github.com/mycelium-com/wallet|source]]) * [[https://copay.io/|Copay]] ([[https://github.com/bitpay/copay|source]]) * [[https://maza.club/encompass|Encompass]] ([[https://github.com/mazaclub/encompass|source]]) - +* [[https://www.coinvault.io/|CoinVault]] ==Reference== * [[bip-0032.mediawiki|BIP32 - Hierarchical Deterministic Wallets]] diff --git a/bip-0047.mediawiki b/bip-0047.mediawiki index 1a05730..b1145b3 100644 --- a/bip-0047.mediawiki +++ b/bip-0047.mediawiki @@ -1,7 +1,7 @@ RECENT CHANGES: +* (19 Apr 2016) Define version 2 payment codes +* (17 Apr 2016) Clarify usage of outpoints in notification transactions * (18 Dec 2015) Update explanations to resolve FAQs -* (12 Oct 2015) Revise blinding method for notification transactions -* (21 Sep 2015) Correct base58check version byte <pre> BIP: 47 @@ -84,7 +84,27 @@ Hardened derivation is used at this level. Except where noted, all keys derived from a payment code use the public derivation method. -==Standard Payment Codes (v1)== +==Versions== + +Payment codes contain a version byte which identifies a specific set of behavior. + +Unless otherwise specified, payment codes of different versions are interoperable. If Alice uses a version x payment code, and Bob uses a version y payment code, they can still send and receive transactions between each other. + +Currently specified versions: + +* Version 1 +** Address type: P2PKH +** Notification type: address +* Version 2 +** Address type: P2PKH +** Notification type: bloom-multisig + +===Recommended Versions=== + +* Wallets which have bloom filtering capabilities SHOULD create version 2 payment codes instead of version 1 payment codes. +* Version 1 payment codes are only recommended for wallets which lack access to bloom filtering capability. + +==Version 1== ===Representation=== @@ -119,20 +139,25 @@ It is assumed that Alice can easily obtain Bob's payment code via a suitable met * Payment code: an extended public key and associated metadata which is associated with a particular identity/account * Notification address: the P2PKH address associated with the 0<sup>th</sup> public key derived from a payment code * Notification transaction: a transaction which sends an output to a notification address which includes an embedded payment code +* Designated input: the first input in the notification transaction which exposes an secp256k1 pubkey in either its signature script, or in the redeem script or pubkey script of the output being spent +* Designated pubkey: the first secp256k1 pubkey pushed to the stack during script execution for the designated input +* Outpoint: the specific output of a previous transaction which is being spent. See the Reference section for the binary serialization ====Notification Transaction==== Prior to the first time Alice initiates a transaction to Bob, Alice MUST inform Bob of her payment code via the following procedure: +Note: this procedure is used if Bob uses a version 1 payment code (regardless of the the version of Alice's payment code). If Bob's payment code is not version 1, see the appropriate section in this specification. + # Alice constructs a transaction which sends a small quantity of bitcoins to Bob's notification address (notification transaction) ## The inputs selected for this transaction MUST NOT be easily associated with Alice's notification address # Alice derives a unique shared secret using ECDH: -## Alice selects the private key corresponding to the first exposed public key, of the first pubkey-exposing input, of the transaction: <pre>a</pre> +## Alice selects the private key corresponding to the designated pubkey: <pre>a</pre> ## Alice selects the public key associated with Bob's notification address: <pre>B, where B = bG</pre> ## Alice calculates a secret point: <pre>S = aB</pre> ## Alice calculates a 64 byte blinding factor: <pre>s = HMAC-SHA512(x, o)</pre> ### "x" is the x value of the secret point -### "o" is the outpoint being spent by the first pubkey-exposing input of the transaction. +### "o" is the outpoint being spent by the designated input # Alice serializes her payment code in binary form. # Alice renders her payment code (P) unreadable to anyone except Bob: ## Replace the x value with x': <pre>x' = x XOR (first 32 bytes of s)</pre> @@ -143,12 +168,12 @@ Prior to the first time Alice initiates a transaction to Bob, Alice MUST inform # Bob watches for any transactions which create an output at his notification address. # When a transaction is received, the client examines it to determine if it contains a standard OP_RETURN output with an 80 byte payload (notification transactions). # If the first byte of the payload in a notification transaction is 0x01: -## Bob selects the first exposed public key, of the first pubkey-exposing input, of the transaction: <pre>A, where A = aG</pre> +## Bob selects the designated pubkey: <pre>A, where A = aG</pre> ## Bob selects the private key associated with his notification address: <pre>b</pre> ## Bob calculates a secret point: <pre>S = bA</pre> ## Bob calculates the binding factor: <pre>s = HMAC-SHA512(x, o)</pre> ### "x" is the x value of the secret point -### "o" is the outpoint being spent by the first pubkey-exposing input of the transaction. +### "o" is the outpoint being spent by the designated input. ## Bob interprets the 80 byte payload as a payment code, except: ### Replace the x value with x': <pre>x' = x XOR (first 32 bytes of s)</pre> ### Replace the chain code with c': <pre>c' = c XOR (last 32 bytes of s)</pre> @@ -177,6 +202,17 @@ Alice SHOULD use an input script in one of the following standard forms to expos Compatible wallets MAY provide a method for a user to manually specify the public key associated with a notification transaction in order to recover payment codes sent via non-standard notification transactions. +=====Post-Notification Privacy Considerations===== + +Incautious handling of change outputs from notification transactions may cause unintended loss of privacy. + +The recipient of a transaction which spends a change output from a prior notification transaction will learn about the potential connection between the sender and the recipient of the notification transaction. + +The following actions are recommended to reduce this risk: + +* Wallets which support mixing SHOULD mix change outputs from notification transactions prior to spending them +* Wallets which do not support mixing MAY simulate mixing by creating a transaction which spends the change output to the next external BIP44 address + ====Sending==== # Each time Alice wants to initiate a transaction to Bob, Alice derives a unique P2PKH address for the transaction using ECDH follows: @@ -291,12 +327,51 @@ The sender transmits their payment code in base58 form to the calculated Bitmess In order to use Bitmessage notification, the recipient must have a Bitmessage client which listens at the address which the senders will derive and is capable of relaying received payment codes to the Bitcoin wallet. +==Version 2== + +Version 2 payment codes behave identifically to version 1 payment codes, except as modified below. + +===Representation=== + +====Binary Serialization==== + +* Byte 0: version. required value: 0x02 + +===Protocol=== + +====Definitions==== + +* Notification change output: the change output from a notification transaction which which resides in the sender's wallet, but can be automatically located by the intended recipient +* Payment code identifier: a 33 byte representation of a payment code constructed by prepending 0x02 to the SHA256 hash of the binary serialization of the payment code + +====Notification Transaction==== + +Note: this procedure is used if Bob uses a version 2 payment code (regardless of the the version of Alice's payment code). If Bob's payment code is not version 2, see the appropriate section in this specification. + +# Construct a notification transaction as per the version 1 instructions, except do not create the output to Bob's notification address +# Create a notification change address as follows: +## Obtain the pubkey corresponding to the next change address +## Construct a multisig output in the form: +<pre>OP_1 <Bob's payment code identifier> <change address pubkey> OP_2 OP_CHECKMULTISIG</pre> + +The relative ordering of the payment code identifier and change address pubkey in the above script MAY be randomized + +Bob detects notification transactions by adding his payment code identifier to his bloom filter. + +# When the filter returns a notification transaction, the sender's payment code is unblinded using the same procedure as for version 1 notification transactions. + +Alice's wallet should spend the notification change output at the next appropriate opportunity. + +==Test Vectors== + +* [[https://gist.github.com/SamouraiDev/6aad669604c5930864bd|BIP47 Reusable Payment Codes Test Vectors]] + ==Reference== * [[bip-0032.mediawiki|BIP32 - Hierarchical Deterministic Wallets]] * [[bip-0043.mediawiki|BIP43 - Purpose Field for Deterministic Wallets]] * [[bip-0044.mediawiki|BIP44 - Multi-Account Hierarchy for Deterministic Wallets]] -* [[https://bitcoin.org/en/glossary/outpoint|Outpoint]] +* [[https://bitcoin.org/en/developer-reference#outpoint|Outpoint]] * [[https://github.com/petertodd/dust-b-gone|dust-b-gone]] * [[https://en.bitcoin.it/wiki/Base58Check_encoding|Base58Check encoding]] * [[https://bitmessage.org/bitmessage.pdf|Bitmessage]] diff --git a/bip-0050.mediawiki b/bip-0050.mediawiki index dbec7aa..fbc1c0f 100644 --- a/bip-0050.mediawiki +++ b/bip-0050.mediawiki @@ -2,7 +2,7 @@ BIP: 50 Title: March 2013 Chain Fork Post-Mortem Author: Gavin Andresen <gavinandresen@gmail.com> - Status: Draft + Status: Final Type: Informational Created: 2013-03-20 </pre> @@ -30,7 +30,7 @@ With the insufficiently high BDB lock configuration, it implicitly had become a Because max-sized blocks had been successfully processed on the testnet, it did not occur to anyone that there could be blocks that were smaller but require more locks than were available. Prior to 0.7 unmodified mining nodes self-imposed a maximum block size of 500,000 bytes, which further prevented this case from being triggered. 0.7 made the target size configurable and miners had been encouraged to increase this target in the week prior to the incident. -Bitcoin Core 0.8 did not use Berkeley DB. It switched to LevelDB instead, which did not implement the same locking limits as BDB. Therefore it was able to process the forking block successfully. +Bitcoin 0.8 did not use Berkeley DB. It switched to LevelDB instead, which did not implement the same locking limits as BDB. Therefore it was able to process the forking block successfully. Note that BDB locks are also required during processing of re-organizations. Versions prior to 0.8 may be unable to process some valid re-orgs. diff --git a/bip-0065.mediawiki b/bip-0065.mediawiki index df995de..99298bf 100644 --- a/bip-0065.mediawiki +++ b/bip-0065.mediawiki @@ -2,7 +2,7 @@ BIP: 65 Title: OP_CHECKLOCKTIMEVERIFY Author: Peter Todd <pete@petertodd.org> - Status: Accepted + Status: Final Type: Standards Track Created: 2014-10-01 </pre> diff --git a/bip-0068.mediawiki b/bip-0068.mediawiki index b7bb0db..0303924 100644 --- a/bip-0068.mediawiki +++ b/bip-0068.mediawiki @@ -43,12 +43,14 @@ This specification only interprets 16 bits of the sequence number as relative lo For time based relative lock-time, 512 second granularity was chosen because bitcoin blocks are generated every 600 seconds. So when using block-based or time-based, the same amount of time can be encoded with the available number of bits. Converting from a sequence number to seconds is performed by multiplying by 512 = 2^9, or equivalently shifting up by 9 bits. When the relative lock-time is time-based, it is interpreted as a minimum block-time constraint over the input's age. A relative time-based lock-time of zero indicates an input which can be included in any block. More generally, a relative time-based lock-time n can be included into any block produced 512 * n seconds after the mining date of the output it is spending, or any block thereafter. -The mining date of the output is equals to the median-time-past of the previous block which mined it. +The mining date of the output is equal to the median-time-past of the previous block which mined it. The block produced time is equal to the median-time-past of its previous block. When the relative lock-time is block-based, it is interpreted as a minimum block-height constraint over the input's age. A relative block-based lock-time of zero indicates an input which can be included in any block. More generally, a relative block lock-time n can be included n blocks after the mining date of the output it is spending, or any block thereafter. +The new rules are not applied to the nSequence field of the input of the coinbase transaction. + ==Implementation== A reference implementation is provided by the following pull request @@ -64,21 +66,21 @@ enum { /* Setting nSequence to this value for every input in a transaction * disables nLockTime. */ static const uint32_t SEQUENCE_FINAL = 0xffffffff; - + +/* Below flags apply in the context of BIP 68*/ /* If this flag set, CTxIn::nSequence is NOT interpreted as a - * relative lock-time. Setting the most significant bit of a - * sequence number disabled relative lock-time. */ + * relative lock-time. */ static const uint32_t SEQUENCE_LOCKTIME_DISABLE_FLAG = (1 << 31); - + /* If CTxIn::nSequence encodes a relative lock-time and this flag * is set, the relative lock-time has units of 512 seconds, * otherwise it specifies blocks with a granularity of 1. */ static const uint32_t SEQUENCE_LOCKTIME_TYPE_FLAG = (1 << 22); - + /* If CTxIn::nSequence encodes a relative lock-time, this mask is * applied to extract that lock-time from the sequence field. */ static const uint32_t SEQUENCE_LOCKTIME_MASK = 0x0000ffff; - + /* In order to use the same number of bits to encode roughly the * same wall-clock duration, and because blocks are naturally * limited to occur every 600s on average, the minimum granularity @@ -87,7 +89,7 @@ static const uint32_t SEQUENCE_LOCKTIME_MASK = 0x0000ffff; * multiplying by 512 = 2^9, or equivalently shifting up by * 9 bits. */ static const int SEQUENCE_LOCKTIME_GRANULARITY = 9; - + /** * Calculates the block height and previous block's median time past at * which the transaction will be considered final in the context of BIP 68. @@ -174,16 +176,17 @@ bool SequenceLocks(const CTransaction &tx, int flags, std::vector<int>* prevHeig bool CheckSequenceLocks(const CTransaction &tx, int flags) { AssertLockHeld(cs_main); + AssertLockHeld(mempool.cs); CBlockIndex* tip = chainActive.Tip(); CBlockIndex index; index.pprev = tip; // CheckSequenceLocks() uses chainActive.Height()+1 to evaluate // height based locks because when SequenceLocks() is called within - // CBlock::AcceptBlock(), the height of the block *being* - // evaluated is what is used. Thus if we want to know if a - // transaction can be part of the *next* block, we need to call - // SequenceLocks() with one more than chainActive.Height(). + // ConnectBlock(), the height of the block *being* + // evaluated is what is used. + // Thus if we want to know if a transaction can be part of the + // *next* block, we need to use one more than chainActive.Height() index.nHeight = tip->nHeight + 1; // pcoinsTip contains the UTXO set for chainActive.Tip() @@ -217,9 +220,13 @@ This BIP was edited by BtcDrak, Nicolas Dorier and kinoshitajona. ==Deployment== -This BIP is to be deployed by either version-bits BIP9 or by isSuperMajority(). Exact details TDB. +This BIP is to be deployed by "versionbits" BIP9 using bit 0. -BIP68 MUST be deployed together with BIP113. It is also recommended to deploy BIP112 at the same time. +For Bitcoin '''mainnet''', the BIP9 '''starttime''' will be midnight 1st May 2016 UTC (Epoch timestamp 1462060800) and BIP9 '''timeout''' will be midnight 1st May 2017 UTC (Epoch timestamp 1493596800). + +For Bitcoin '''testnet''', the BIP9 '''starttime''' will be midnight 1st March 2016 UTC (Epoch timestamp 1456790400) and BIP9 '''timeout''' will be midnight 1st May 2017 UTC (Epoch timestamp 1493596800). + +This BIP must be deployed simultaneously with BIP112 and BIP113 using the same deployment mechanism. ==Compatibility== @@ -245,6 +252,8 @@ The most efficient way to calculate sequence number from relative lock-time is w Bitcoin mailing list discussion: https://www.mail-archive.com/bitcoin-development@lists.sourceforge.net/msg07864.html +BIP9: https://github.com/bitcoin/bips/blob/master/bip-0009.mediawiki + BIP112: https://github.com/bitcoin/bips/blob/master/bip-0112.mediawiki BIP113: https://github.com/bitcoin/bips/blob/master/bip-0113.mediawiki diff --git a/bip-0069.mediawiki b/bip-0069.mediawiki index 0eca369..4094126 100644 --- a/bip-0069.mediawiki +++ b/bip-0069.mediawiki @@ -162,6 +162,8 @@ Outputs: * [[https://github.com/bitcoinjs/bip69/blob/master/index.js|BitcoinJS]] * [[https://github.com/bitcoinjs/bip69/blob/master/test/fixtures.json|BitcoinJS Test Fixtures]] * [[https://www.npmjs.com/package/bip69|NodeJS]] +* [[https://github.com/blockchain/My-Wallet-V3/blob/v3.8.0/src/transaction.js#L120-L142|Blockchain.info public beta]] +* [[https://github.com/btcsuite/btcutil/blob/master/txsort/txsort.go|Btcsuite]] ==Acknowledgements== diff --git a/bip-0073.mediawiki b/bip-0073.mediawiki index a020fb5..41c89a3 100644 --- a/bip-0073.mediawiki +++ b/bip-0073.mediawiki @@ -2,7 +2,7 @@ BIP: 73 Title: Use "Accept" header for response type negotiation with Payment Request URLs Author: Stephen Pair <stephen@bitpay.com> - Status: Draft + Status: Final Type: Standards Track Created: 2013-08-27 </pre> diff --git a/bip-0075.mediawiki b/bip-0075.mediawiki new file mode 100644 index 0000000..9ce90f6 --- /dev/null +++ b/bip-0075.mediawiki @@ -0,0 +1,383 @@ +<pre> + BIP: 75 + Title: Out of Band Address Exchange using Payment Protocol Encryption + Author: Justin Newton <justin@netki.com> + Matt David <mgd@mgddev.com> + Aaron Voisine <voisine@gmail.com> + James MacWhyte <macwhyte@gmail.com> + Status: Draft + Type: Standards Track + Created: 2015-11-20 +</pre> + +==Abstract== + +This BIP is an extension to BIP 70 that provides two enhancements to the existing Payment Protocol. + +# It allows the requester (Sender) of a PaymentRequest to voluntarily sign the original request and provide a certificate to allow the payee to know the identity of who they are transacting with. + +# It encrypts the PaymentRequest that is returned, before handing it off to the SSL/TLS layer to prevent man in the middle viewing of the Payment Request details. + +The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and +"OPTIONAL" in this document are to be interpreted as described in RFC 2119. + +==Definitions== +{| class="wikitable" +| Sender || Entity wishing to transfer value that they control +|- +| Receiver || Entity receiving a value transfer +|} + +==Motivation== + +The motivation for defining this extension to the [[bip-0070.mediawiki|BIP70]] Payment Protocol is to allow 2 parties to exchange payment information in a permissioned and encrypted way such that wallet address communication can become a more automated process. Additionally, this extension allows for the requester of a PaymentRequest to supply a certificate and signature in order to facilitate identification for address release. This also allows for automated creation of off blockchain transaction logs that are human readable, containing who you transacted with, in addition to the information that it contains today. + +The motivation for this extension to [[bip-0070.mediawiki|BIP70]] is threefold: + +# Ensure that the payment details can only be seen by the participants in the transaction, and not by any third party. + +# Enhance the Payment Protocol to allow for store and forward servers in order to allow, for example, mobile wallets to sign and serve Payment Requests. + +# Allow a sender of funds the option of sharing their identity with the receiver. This information could then be used to: + +#* Make Bitcoin logs more human readable +#* Give the user the ability to decide who to release payment details to +#* Allow an entity such as a political campaign to ensure donors match regulatory and legal requirements +#* Allow for an open standards based way for regulated financial entities to meet regulatory requirements +#* Automate the active exchange of payment addresses, so static addresses and BIP32 X-Pubs can be avoided to maintain privacy and convenience + +In short we wanted to make Bitcoin more human, while at the same time improving transaction privacy. + +==Example Use Cases== + +1. Address Book + +A Bitcoin wallet developer would like to offer the ability to store an "address book" of payees, so users could send multiple payments to known entities without having to request an address every time. Static addresses compromise privacy, and address reuse is considered a security risk. BIP32 X-Pubs allow the generation of unique addresses, but watching an X-Pub chain for each person you wish to receive funds from is too resource-intensive for mobile applications, and there is always a risk of unknowingly sending funds to an X-Pub address after the owner has lost access to the corresponding private key. + +With this BIP, Bitcoin wallets could maintain an "address book" that only needs to store each payee's public key. Adding an entry to one's address book could be done by using a Wallet Name, scanning a QR code, sending a URI through a text message or e-mail, or searching a public repository. When the user wishes to make a payment, their wallet would do all the work in the background to communicate with the payee's wallet to receive a unique payment address. If the payee's wallet has been lost, replaced, or destroyed, no communication will be possible, and the sending of funds to a "dead" address is prevented. + +2. Individual Permissioned Address Release + +A Bitcoin wallet developer would like to allow users to view a potential sending party's identifying information before deciding whether or not to share payment information with them. Currently, [[bip-0070.mediawiki|BIP70]] specifies that the Merchant Server respond to a "pay now" style request with a PaymentRequest, releasing address and X.509 certificate identity information of the potential receiving party. + +With this BIP, Bitcoin wallets could prompt a wallet user to release payment information while displaying identity information about the potential sending party via an included certificate. This gives the receiving party more control over who receives their payment and identity information, and could be helpful for businesses that need to follow KYC policies or wallets that want to focus on privacy. + +3. Using Store & Forward Servers + +A Bitcoin wallet developer would like to use a public Store & Forward service for an asynchronous address exchange. This is a common case for mobile and offline wallets. + +With this BIP, returned payment information is encrypted with an ECDH-computed shared key before sending to a Store & Forward service. In this case, a successful attack against a Store & Forward service would not be able to read or modify wallet address or payment information, only delete encrypted messages. + +==New Messages== +Updated [/bip-0075/paymentrequest.proto paymentrequest.proto] contains the existing PaymentRequest Protocol Buffer messages as well as the messages newly defined in this BIP. + +'''NOTE''': Public keys from both parties must be known to each other in order to facilitate encrypted communication. Although including both public keys in every message may get redundant, it provides the most flexibility as each message is completely self-contained. + +===InvoiceRequest=== +The '''InvoiceRequest''' message allows a Sender to send information to the Receiver such that the Receiver can create and return a PaymentRequest. + +<pre> +message InvoiceRequest { + required bytes sender_public_key = 1; + optional uint64 amount = 2 [default = 0]; + optional string pki_type = 3 [default = "none"]; + optional bytes pki_data = 4; + optional string memo = 5; + optional string notification_url = 6; + optional bytes signature = 7; +} +</pre> + +{| class="wikitable" +! Field Name !! Description +|- +| sender_public_key || Sender's EC public key +|- +| amount || amount is integer-number-of-satoshis (default: 0) +|- +| pki_type || none / x509+sha256 (default: "none") +|- +| pki_data || Depends on pki_type +|- +| memo || Human-readable description of invoice request for the receiver +|- +| notification_url || Secure (usually TLS-protected HTTP) location where an [[#EncryptedProtocolMessage|EncryptedProtocolMessage]] SHOULD be sent when ready +|- +| signature || PKI-dependent signature +|} + +===ProtocolMessageType Enum=== +This enum is used in the newly defined [[#ProtocolMessage|ProtocolMessage]] and [[#EncryptedProtocolMessage|EncryptedProtocolMessage]] messages to define the serialized message type. The '''ProtocolMessageType''' enum is defined in an extensible way to allow for new message type additions to the Payment Protocol. +<pre> +enum ProtocolMessageType { + INVOICE_REQUEST = 0; + PAYMENT_REQUEST = 1; + PAYMENT = 2; + PAYMENT_ACK = 3; +} +</pre> + +===ProtocolMessage=== +The '''ProtocolMessage''' message is an encapsulating wrapper for any Payment Protocol message. It allows two-way, non-encrypted communication of Payment Protocol messages. The message also includes a status code and a status message that is used for error communication such that the protocol does not rely on transport-layer error handling. +<pre> +message ProtocolMessage { + required ProtocolMessageType message_type = 1; + required bytes serialized_message = 2; + optional uint64 status_code = 3; + optional string status_message = 4; + optional bytes identifier = 5; +} +</pre> + +{| class="wikitable" +! Field Name !! Description +|- +|message_type || Message Type of serialized_message +|- +|serialized_message || Serialized Payment Protocol Message +|- +|status_code || Payment Protocol Status Code +|- +|status_message || Human-readable Payment Protocol status message +|- +|identifier || Unique key to identify this entire exchange on the server. SHA256 of initial serialized InvoiceRequest SHOULD be used by default +|} + +===EncryptedProtocolMessage=== +The '''EncryptedProtocolMessage''' message is an encapsualting wrapper for any Payment Protocol message. It allows two-way, authenticated and encrypted communication of Payment Protocol messages in order to keep their contents secret. The message also includes a status code and status message that is used for error communication such that the protocol does not rely on transport-layer error handling. +<pre> +message EncryptedProtocolMessage { + required ProtocolMessageType message_type = 1; + required bytes encrypted_message = 2; + required bytes receiver_public_key = 3; + required bytes sender_public_key = 4; + required uint64 nonce = 5; + optional bytes signature = 6; + optional bytes identifier = 7; + optional uint64 status_code = 8; + optional string status_message = 9; +} +</pre> +{| class="wikitable" +! Field Name !! Description +|- +| message_type || Message Type of Decrypted encrypted_message +|- +| encrypted_message || AES-256-GCM Encrypted (as defined in BIP75) Payment Protocol Message +|- +| receiver_public_key || Receiver's DER-encoded EC Public Key +|- +| sender_public_key || Sender's DER-encoded EC Public Key +|- +| nonce || Microseconds since epoch +|- +| signature || DER-encoded Signature over the full EncryptedProtocolMessage with EC Key Belonging to Sender / Receiver, respectively +|- +| identifier || Unique key to identify this entire exchange on the server. SHA256 of initial serialized InvoiceRequest SHOULD be used by default +|- +| status_code || Payment Protocol Status Code +|- +| status_message || Human-readable Payment Protocol status message +|} + +==Payment Protocol Process with InvoiceRequests== +The full process overview for using '''InvoiceRequests''' in the Payment Protocol is defined below. +<br/><br/> +All Payment Protocol messages MUST be encapsulated in either a [[#ProtocolMessage|ProtocolMessage]] or [[#EncryptedProcotolMessage|EncryptedProtocolMessage]]. Once the process begins using [[#EncryptedProtocolMessage|EncryptedProtocolMessage]] messages, all subsequent communications MUST use [[#EncryptedProtocolMessage|EncryptedProtocolMessages]]. +<br/><br/> +All Payment Protocol messages SHOULD be communicated using [[#EncryptedProtocolMessage|EncryptedProtocolMessage]] encapsulating messages with the exception that an [[#InvoiceRequest|InvoiceRequest]] MAY be communicated using the [[#ProtocolMessage|ProtocolMessage]] if the receiver's public key is unknown. +<br/><br/> + +The process of creating encrypted Payment Protocol messages is enumerated in [[#Sending_Encrypted_Payment_Protocol_Messages_using_EncryptedProtocolMessages|Sending Encrypted Payment Protocol Messages using EncryptedProtocolMessages]], and the process of decrypting encrypted messages can be found under [[#Validating_and_Decrypting_Payment_Protocol_Messages_using_EncryptedProtocolMessages|Validating and Decrypting Payment Protocol Messages using EncryptedProtocolMessages]]. + +A standard exchange from start to finish would look like the following: + +# Sender creates InvoiceRequest +# Sender encapsulates InvoiceRequest in (Encrypted)ProtocolMessage +# Sender sends (Encrypted)ProtocolMessage to Receiver +# Receiver retrieves InvoiceRequest in (Encrypted)ProtocolMessage from Sender +# Receiver creates PaymentRequest +# Receiver encapsulates PaymentRequest in EncryptedProtocolMessage +# Receiver transmits EncryptedProtocolMessage to Sender +# Sender validates PaymentRequest retrieved from the EncryptedProtocolMessage +# The PaymentRequest is processed according to [[bip-0070.mediawiki|BIP70]], including optional Payment and PaymentACK messages encapsulated in EncryptedProtocolMessage messages. + +'''NOTE:''' See [[#Initial_Public_Key_Retrieval_for_InvoiceRequest_Encryption|Initial Public Key Retrieval for InvoiceRequest Encryption]] for possible options to retrieve Receiver's public key. + +<img src="bip-0075/encrypted-invoice-request-process.png" alt="Flow diagram of Encrypted InvoiceRequest"> + +==Message Interaction Details== + +===HTTP Content Types for New Message Types=== +When communicated via '''HTTP''', the listed messages MUST be transmitted via TLS-protected HTTP using the appropriate Content-Type header as defined here per message: +<br/> +{| class="wikitable" +! Message Type !! Content Type +|- +| ProtocolMessage || application/bitcoin-paymentprotocol-message +|- +| EncryptedProtocolMessage || application/bitcoin-encrypted-paymentprotocol-message +|} + +===Payment Protocol Status Communication=== + +In the case of an error that causes the Payment Protocol process to be stopped or requires that message be retried, a [[#ProtocolMessage|ProtocolMessage]] or [[#EncryptedProtocolMessage|EncryptedProtocolMessage]] MUST be returned by the party generating the error status_code. The content of the message MUST contain the same '''serialized_message''' or '''encrypted_message''' and identifier (if present) and MUST have the status_code set appropriately. +<br/><br/> +The status_message value SHOULD be set with a human readable explanation of the status code. For example, if in an [[#EncryptedProtocolMessage|EncryptedProtocolMessage]], the AES-256-GCM decryption fails to authenticate, an Authentication Failed (102) '''status_code''' MUST be returned to prevent oracle attacks. + +====Payment Protocol Status Codes==== +{| class="wikitable" +! Status Code !! Description +|- +| 1 || OK +|- +| 100 || General / Unknown Error +|- +| 102 || Authentication Failed +|- +| 102 || Encrypted Message Required +|- +| 200 || Amount Too High +|- +| 201 || Amount Too Low +|- +| 202 || Amount Invalid +|- +| 203 || Payment Does Not Meet PaymentRequest Requirements +|- +| 300 || Certificate Required +|- +| 301 || Certificate Expired +|- +| 302 || Certificate Invalid for Transaction +|- +| 303 || Certificate Revoked +|- +| 304 || Certificate Not Well Rooted +|- +|} + +===Transport Layer Communication Errors=== + +Communication errors MUST be communicated to the party that initiated the communication via the communication layer's existing error messaging faciltiies. In the case of TLS-protected HTTP, this SHOULD be done through standard HTTP Status Code messaging ([https://tools.ietf.org/html/rfc7231 RFC 7231 Section 6]). + +==Extended Payment Protocol Process Details== +This BIP extends the Payment Protocol as defined in [[bip-0070.mediawiki|BIP70]]. + +For the following we assume the Sender already knows the Receiver's public key, and the exchange is being facilitated by a Store & Forward server which requires valid signatures for authentication. + +'''nonce''' MUST be set to a non-repeating number '''and''' MUST be chosen by the encryptor. The current epoch time in microseconds SHOULD be used, unless the creating device doesn't have access to a RTC (in the case of a smart card, for example). The service receiving the message containing the '''nonce''' MAY use whatever method to make sure that the '''nonce''' is never repeated. + +===InvoiceRequest Message Creation=== +* Create an [[#InvoiceRequest|InvoiceRequest]] message +* '''sender_public_key''' MUST be set to the public key of an EC keypair +* '''amount''' is optional. If the amount is not specified by the [[#InvoiceRequest|InvoiceRequest]], the Receiver MAY specify the amount in the returned PaymentRequest. If an amount is specified by the [[#InvoiceRequest|InvoiceRequest]] and a PaymentRequest cannot be generated for that amount, the [[#InvoiceRequest|InvoiceRequest]] SHOULD return the same [[#InvoiceRequest|InvoiceRequest]] in a [[#ProtocolMessage|ProtocolMessage]] or [[#EncryptedProtocolMessage|EncryptedProtocolMessage]] with the status_code and status_message fields set appropriately. +* '''memo''' is optional. This MAY be set to a human readable description of the InvoiceRequest +* Set '''notification_url''' to URL that the Receiver will submit completed PaymentRequest (encapsulated in an [[#EncryptedProtocolMessage|EncryptedProtocolMessage]]) to +* If NOT including certificate, set '''pki_type''' to "none" +* If including certificate: +** Set '''pki_type''' to "x509+sha256" +** Set '''pki_data''' as it would be set in BIP-0070 ([https://github.com/bitcoin/bips/blob/master/bip-0070.mediawiki#Certificates Certificates]) +** Sign [[#InvoiceRequest|InvoiceRequest]] with signature = "" using the X509 Certificate's private key +** Set '''signature''' value to the computed signature + +===InvoiceRequest Validation=== +* Validate '''sender_public_key''' is a valid EC public key +* Validate '''notification_url''', if set, contains characters deemed valid for a URL (avoiding XSS related characters, etc). +* If '''pki_type''' is None, [[#InvoiceRequest|InvoiceRequest]] is VALID +* If '''pki_type''' is x509+sha256 and '''signature''' is valid for the serialized [[#InvoiceRequest|InvoiceRequest]] where signature is set to "", [[#InvoiceRequest|InvoiceRequest]] is VALID + +===Sending Encrypted Payment Protocol Messages using EncryptedProtocolMessages=== +* Encrypt the serialized Payment Protocol message using AES-256-CBC setup as described in [[#ECDH_Point_Generation_and_AES256_GCM_Mode_Setup|ECDH Point Generation and AES-256 (GCM Mode) Setup]] +* Create [[#EncryptedProtocolMessage|EncryptedProtocolMessage]] message +* Set '''encrypted_message''' to be the encrypted value of the Payment Protocol message +* '''sender_public_key''' MUST be set to the public key of the Sender's EC keypair +* '''receiver_public_key''' MUST be set to the public key of the Receiver's EC keypair +* '''nonce''' MUST be set to the nonce used in the AES-256-CBC encryption operation +* Set '''identifier''' to the identifier value received in the originating InvoiceRequest's ProtocolMessage or EncryptedProtocolMessage wrapper message +* Set '''signature''' to "" +* Sign the serialized [[#EncryptedProtocolMessage|EncryptedProtocolMessage]] message with the communicating party's EC public key +* Set '''signature''' to the result of the signature operation above + +'''SIGNATURE NOTE:''' [[#EncryptedProtocolMessage|EncryptedProtocolMessage]] messages are signed with the public keys of the party transmitting the message. This allows a Store & Forward server or other transmission system to prevent spam or other abuses. For those who are privacy conscious and don't want the server to track the interactions between two public keys, the Sender can generate a new public key for each interaction to keep their identity anonymous. + +===Validating and Decrypting Payment Protocol Messages using EncryptedProtocolMessages=== +* The '''nonce''' MUST not be repeated. The service receiving the [[#EncryptedProtocolMessage|EncryptedProtocolMessage]] MAY use whatever method to make sure that the nonce is never repeated. +* Decrypt the serialized Payment Protocol message using AES-256-GCM setup as described in [[#ECDH_Point_Generation_and_AES256_GCM_Mode_Setup|ECDH Point Generation and AES-256 (GCM Mode) Setup]] +* Deserialize the serialized Payment Protocol message + +===ECDH Point Generation and AES-256 (GCM Mode) Setup=== +'''NOTE''': AES-256-GCM is used because it provides authenticated encryption facilities, thus negating the need for a separate message hash for authentication. + +* Generate the '''secret point''' using [https://en.wikipedia.org/wiki/Elliptic_curve_Diffie–Hellman ECDH] using the local entity's private key and the remote entity's public key as inputs +* Initialize [http://csrc.nist.gov/publications/nistpubs/800-90A/SP800-90A.pdf HMAC_DRBG] +** Use '''SHA512(secret point's X value in Big-Endian bytes)''' for Entropy +** Use the given message's '''nonce''' field for Nonce + +* Initialize AES-256 in GCM Mode +** Initialize HMAC_DRBG with Security Strength of 256 bits +** Use HMAC_DRBG.GENERATE(32) as the Encryption Key (256 bits) +** Use HMAC_DRBG.GENERATE(12) as the Initialization Vector (IV) (96 bits) + +====AES-256 GCM Authentication Tag Use==== +The 16 byte authentication tag resulting from the AES-GCM encrypt operation MUST be prefixed to the returned ciphertext. The decrypt operation will use the first 16 bytes of the ciphertext as the GCM authentication tag and the remainder of the ciphertext as the ciphertext in the decrypt operation. + +====AES-256 GCM Additional Authenticated Data==== +When either '''status_code''' OR '''status_message''' are present, the AES-256 GCM authenticated data used in both the encrypt and decrypt operations MUST be: STRING(status_code) || status_message. Otherwise, there is no additional authenticated data. This provides that, while not encrypted, the status_code and status_message are authenticated. + +===Initial Public Key Retrieval for InvoiceRequest Encryption=== +Initial public key retrieval for [[#InvoiceRequest|InvoiceRequest]] encryption via [[#EncryptedProtocolMessage|EncryptedProtocolMessage]] encapsulation can be done in a number of ways including, but not limited to, the following: +# Wallet Name public key asset type resolution - DNSSEC-validated name resolution returns Base64 encoded DER-formatted EC public key via TXT Record [https://www.ietf.org/rfc/rfc5480.txt RFC 5480] +# Key Server lookup - Key Server lookup (similar to PGP's pgp.mit.edu) based on key server identifier (i.e., e-mail address) returns Base64 encoded DER-formatted EC public key [https://www.ietf.org/rfc/rfc5480.txt RFC 5480] +# QR Code - Use of QR-code to encode DER-formatted EC public key [https://www.ietf.org/rfc/rfc5480.txt RFC 5480] +# Address Service Public Key Exposure + +==Payment / PaymentACK Messages with a HTTP Store & Forward Server== +If a Store & Forward server wishes to protect themselves from spam or abuse, they MAY enact whatever rules they deem fit, such as the following: + +* Once an InvoiceRequest or PaymentRequest is received, all subsequent messages using the same identifier must use the same Sender and Receiver public keys. +* For each unique identifier, only one message each of type InvoiceRequest, PaymentRequest, and PaymentACK may be submitted. Payment messages may be submitted/overwritten multiple times. All messages submitted after a PaymentACK is received will be rejected. +* Specific messages are only saved until they have been verifiably received by the intended recipient or a certain amount of time has passed, whichever comes first. + +<br/><br/> +Clients SHOULD keep in mind Receivers can broadcast a transaction without returning an ACK. If a Payment message needs to be updated, it SHOULD include at least one input referenced in the original transaction to prevent the Receiver from broadcasting both transactions and getting paid twice. + +==Public Key & Signature Encoding== +* All EC public keys ('''sender_public_key''', '''receiver_public_key''') included in any message defined in this BIP MUST be DER [ITU.X690.1994] encoded. +* All ECC signatures included in any message defined in this BIP MUST use the SHA-256 hashing algorithm and MUST be DER [ITU.X690.1994] encoded. + +==Implementation== +A reference implementation for a Store & Forward server supporting this proposal can be found here: + +[https://github.com/netkicorp/addressimo Addressimo] + +A reference client implementation can be found in the InvoiceRequest functional testing for Addressimo here: + +[https://github.com/netkicorp/addressimo/blob/master/functest/functest_bip75.py BIP75 Client Reference Implementation] + +==BIP70 Extension== +The following flowchart is borrowed from [[bip-0070.mediawiki|BIP70]] and expanded upon in order to visually describe how this BIP is an extension to [[bip-0070.mediawiki|BIP70]]. + +<img src="bip-0075/bip70-extension.png" alt="Flowchart explaining how this BIP extends BIP 70"> + +==Mobile to Mobile Examples== + +===Full Payment Protocol=== +The following diagram shows a sample flow in which one mobile client is sending value to a second mobile client with the use of an InvoiceRequest, a Store & Forward server, PaymentRequest, Payment and PaymentACK. In this case, the PaymentRequest, Payment and PaymentACK messages are encrypted using [[#EncryptedProtocolMessage|EncryptedProtocolMessage]] '''and''' the Receiver submits the transaction to the Bitcoin network. + +<img src="bip-0075/mobile-sf-ir-with-payment.png" alt="Payment Required flow diagram"> + +===Encrypting Initial InvoiceRequest via EncryptedProtocolMessage=== +The following diagram shows a sample flow in which one mobile client is sending value to a second mobile client using an [[#EncryptedProtocolMessage|EncryptedProtocolMessage]] to transmit the InvoiceRequest using encryption, Store & Forward server, and PaymentRequest. In this case, all Payment Protocol messages are encrypting using [[#EncryptedProtocolMessage|EncryptedProtocolMessage]] '''and''' the Sender submits the transaction to the Bitcoin network. + +<img src="bip-0075/mobile-sf-encrypted-ir-without-payment.png" alt="Encrypted InvoiceRequest without payment"> + +==References== + +* [[bip-0070.mediawiki|BIP70 - Payment Protocol]] +* [https://en.wikipedia.org/wiki/Elliptic_curve_Diffie–Hellman ECDH] +* [http://csrc.nist.gov/publications/nistpubs/800-90A/SP800-90A.pdf HMAC_DRBG] +* [http://csrc.nist.gov/publications/nistpubs/800-38D/SP-800-38D.pdf NIST Special Publication 800-38D - Recommendation for Block Cipher Modes of Operation: Galois/Counter Mode (GCM) and GMAC] +* [https://tools.ietf.org/html/rfc6979 RFC6979] +* [https://en.bitcoin.it/wiki/Address_reuse Address Reuse] +* [http://csrc.nist.gov/publications/fips/fips180-4/fips-180-4.pdf FIPS 180-4 (Secure Hash Standard)] diff --git a/bip-0075/bip70-extension.png b/bip-0075/bip70-extension.png Binary files differnew file mode 100755 index 0000000..e02db32 --- /dev/null +++ b/bip-0075/bip70-extension.png diff --git a/bip-0075/encrypted-invoice-request-process.png b/bip-0075/encrypted-invoice-request-process.png Binary files differnew file mode 100644 index 0000000..918cf70 --- /dev/null +++ b/bip-0075/encrypted-invoice-request-process.png diff --git a/bip-0075/invoice-request-process.png b/bip-0075/invoice-request-process.png Binary files differnew file mode 100644 index 0000000..47a4fac --- /dev/null +++ b/bip-0075/invoice-request-process.png diff --git a/bip-0075/mobile-sf-encrypted-ir-without-payment.png b/bip-0075/mobile-sf-encrypted-ir-without-payment.png Binary files differnew file mode 100755 index 0000000..fb0b5d1 --- /dev/null +++ b/bip-0075/mobile-sf-encrypted-ir-without-payment.png diff --git a/bip-0075/mobile-sf-ir-with-payment.png b/bip-0075/mobile-sf-ir-with-payment.png Binary files differnew file mode 100755 index 0000000..729fa79 --- /dev/null +++ b/bip-0075/mobile-sf-ir-with-payment.png diff --git a/bip-0075/mobile-sf-ir-without-payment.png b/bip-0075/mobile-sf-ir-without-payment.png Binary files differnew file mode 100755 index 0000000..19cf842 --- /dev/null +++ b/bip-0075/mobile-sf-ir-without-payment.png diff --git a/bip-0075/paymentrequest.proto b/bip-0075/paymentrequest.proto new file mode 100644 index 0000000..3c1ef40 --- /dev/null +++ b/bip-0075/paymentrequest.proto @@ -0,0 +1,84 @@ +// +// Simple Bitcoin Payment Protocol messages +// +// Use fields 1000+ for extensions; +// to avoid conflicts, register extensions via pull-req at +// https://github.com/bitcoin/bips/bip-0070/extensions.mediawiki +// + +package payments; +option java_package = "org.bitcoin.protocols.payments"; +option java_outer_classname = "Protos"; + +// Generalized form of "send payment to this/these bitcoin addresses" +message Output { + optional uint64 amount = 1 [default = 0]; // amount is integer-number-of-satoshis + required bytes script = 2; // usually one of the standard Script forms +} +message PaymentDetails { + optional string network = 1 [default = "main"]; // "main" or "test" + repeated Output outputs = 2; // Where payment should be sent + required uint64 time = 3; // Timestamp; when payment request created + optional uint64 expires = 4; // Timestamp; when this request should be considered invalid + optional string memo = 5; // Human-readable description of request for the customer + optional string payment_url = 6; // URL to send Payment and get PaymentACK + optional bytes merchant_data = 7; // Arbitrary data to include in the Payment message +} +message PaymentRequest { + optional uint32 payment_details_version = 1 [default = 1]; + optional string pki_type = 2 [default = "none"]; // none / x509+sha256 / x509+sha1 + optional bytes pki_data = 3; // depends on pki_type + required bytes serialized_payment_details = 4; // PaymentDetails + optional bytes signature = 5; // pki-dependent signature +} +message X509Certificates { + repeated bytes certificate = 1; // DER-encoded X.509 certificate chain +} +message Payment { + optional bytes merchant_data = 1; // From PaymentDetails.merchant_data + repeated bytes transactions = 2; // Signed transactions that satisfy PaymentDetails.outputs + repeated Output refund_to = 3; // Where to send refunds, if a refund is necessary + optional string memo = 4; // Human-readable message for the merchant +} +message PaymentACK { + required Payment payment = 1; // Payment message that triggered this ACK + optional string memo = 2; // Human-readable message for customer +} + +// BIP-IR Extensions +message InvoiceRequest { + required bytes sender_public_key = 1; // Sender's DER-Encoded EC Public Key + optional uint64 amount = 3 [default = 0]; // amount is integer-number-of-satoshis + optional string pki_type = 4 [default = "none"]; // none / x509+sha256 + optional bytes pki_data = 5; // Depends on pki_type + optional string memo = 6; // Human-readable description of invoice request for the receiver + optional string notification_url = 7; // URL to notify on EncryptedPaymentRequest ready + optional bytes signature = 8; // PKI-dependent signature +} + +enum ProtocolMessageType { + INVOICE_REQUEST = 0; + PAYMENT_REQUEST = 1; + PAYMENT = 2; + PAYMENT_ACK = 3; +} + +message ProtocolMessage { + required ProtocolMessageType message_type = 1; // Message Type of serialized_message + required bytes serialized_message = 2; // Serialized Payment Protocol Message + optional uint64 status_code = 3; // Payment Protocol Status Code + optional string status_message = 4; // Human-readable Payment Protocol status message + optional bytes identifier = 5; // Unique key to identify this entire exchange on the server. SHA256 of initial serialized InvoiceRequest SHOULD be used by default +} + +message EncryptedProtocolMessage { + required ProtocolMessageType message_type = 1; // Message Type of Decrypted encrypted_message + required bytes encrypted_message = 2; // AES-256-GCM Encrypted (as defined in BIP75) Payment Protocol Message + required bytes receiver_public_key = 3; // Receiver's DER-encoded EC Public Key + required bytes sender_public_key = 4; // Sender's DER-encoded EC Public Key + required uint64 nonce = 5; // Microseconds since epoch + optional bytes signature = 6; // Signature over the full EncryptedProtocolMessage with EC Key Belonging to Sender / Receiver, respectively + optional bytes identifier = 7; // Unique key to identify this entire exchange on the server. SHA256 of initial serialized InvoiceRequest SHOULD be used by default + optional uint64 status_code = 8; // Payment Protocol Status Code + optional string status_message = 9; // Human-readable Payment Protocol status message +}
\ No newline at end of file diff --git a/bip-0101.mediawiki b/bip-0101.mediawiki index a76db0f..cc8cfd5 100644 --- a/bip-0101.mediawiki +++ b/bip-0101.mediawiki @@ -2,7 +2,7 @@ BIP: 101 Title: Increase maximum block size Author: Gavin Andresen <gavinandresen@gmail.com> - Status: Draft + Status: Withdrawn Type: Standards Track Created: 2015-06-22 </pre> diff --git a/bip-0109.mediawiki b/bip-0109.mediawiki new file mode 100644 index 0000000..667ef5f --- /dev/null +++ b/bip-0109.mediawiki @@ -0,0 +1,82 @@ +<pre> + BIP: 109 + Title: Two million byte size limit with sigop and sighash limits + Author: Gavin Andresen <gavinandresen@gmail.com> + Status: Draft + Type: Standards Track + Created: 2016-01-28 +</pre> + +==Abstract== + +One-time increase in total amount of transaction data permitted in a block from 1MB to 2MB, with limits on signature operations and hashing. + +==Motivation== + +# Continue current economic policy. +# Exercise hard fork network upgrade. +# Mitigate potential CPU exhaustion attacks + +==Specification== + +=== MAX_BLOCK_SIZE increased to 2,000,000 bytes === + +The maximum number of bytes in a canonically serialized block shall be increased from +1,000,000 bytes to 2,000,000 bytes. + +=== Switch to accurately-counted sigop limit of 20,000 per block === + +The existing MAX_SIGOPS limit of 20,000 signature operations per block shall be retained, +but only ECDSA verifications actually performed to validate the block shall be counted. + +In particular: + +* The coinbase scriptSig is not counted +* Signature operations in un-executed branches of a Script are not counted +* OP_CHECKMULTISIG evaluations are counted accurately; if the signature for a 1-of-20 OP_CHECKMULTISIG is satisified by the public key nearest the top of the execution stack, it is counted as one signature operation. If it is satisfied by the public key nearest the bottom of the execution stack, it is counted as twenty signature operations. +* Signature operations involving invalidly encoded signatures or public keys are not counted towards the limit + +=== Add a new limit of 1,300,000,000 bytes hashed to compute transaction signatures per block === + +The amount of data hashed to compute signature hashes is limited to 1,300,000,000 bytes per block. The same rules for counting are used as for counting signature operations. + +=== Activation: 75% hashpower support trigger, followed by 28-day 'grace period' === + +Solo miners or mining pool operators express their support for this BIP by setting the fourth-highest-bit in the block's 32-bit version number (0x10000000 in hex). The first block with that bit set, a timestamp less than or equal to the expiration time, and with at least 750 out of 1000 blocks preceding it (with heights H-1000..H-1) with that bit set, shall define the beginning of a grace period. Blocks with timestamps greater than or equal to the triggering block's timestamp plus 28 days (60*60*24*28 seconds) shall be subject to the new limits. + +As always, miners are expected to use their best judgement for what is best for the entire Bitcoin ecosystem when making decisions about what consensus-level changes to support. + +=== Expiration: 1-Jan-2018 === + +If this BIP is not triggered before 1-Jan-2018 00:00:00 GMT it should be considered withdrawn. + +Miners that support this BIP should set bit 0x10000000 in the block version until 1-Jan-2018. After that date, that bit can be safely re-used for future consensus rule upgrades. + +==Backward compatibility== + +Fully validating older clients are not compatible with this change. +The first block exceeding the old limits on block size or inaccurately counted signature operations will partition older clients off the new network. + +SPV (simple payment validation) wallets are compatible with this change. + +==Rationale== + +In the short term, an increase is needed to handle increasing transaction volume. + +The limits on signature operations and amount of signature hashing done prevent possible CPU exhaustion attacks by "rogue miners" producing very expensive-to-validate two megabyte blocks. The signature hashing limit is chosen to be impossible to reach with any non-attack transaction or block, to minimize the impact on existing mining or wallet software. + +The choices of constants for the deployment scheme were motivated by prior experience with upgrades to the Bitcoin consensus rules: + +* 0x10000000 was chosen to be compatible with the BIP 9 proposal for parallel deployment of soft forks +* 75% was chosen instead of 95% to minimize the opportunity for a single large mining pool or miner to be able to veto an increase, either because of ideological opposition or threat of violence or extortion. +* A four-week grace period after the voting period was chosen as a balance between giving people sufficient time to upgrade and keeping people's attention on the urgent need to upgrade. + +==Implementation== + +https://github.com/gavinandresen/bitcoin-git/tree/two_mb_bump + +See also http://gavinandresen.ninja/a-guided-tour-of-the-2mb-fork + +==Copyright== + +This work is placed in the public domain. diff --git a/bip-0111.mediawiki b/bip-0111.mediawiki index e0ae9e8..f759f5c 100644 --- a/bip-0111.mediawiki +++ b/bip-0111.mediawiki @@ -1,7 +1,7 @@ <pre> BIP: 111 Title: NODE_BLOOM service bit - Author: Matt Corallo <bip@bluematt.me> + Author: Matt Corallo <bip111@bluematt.me> Peter Todd <pete@petertodd.org> Status: Draft Type: Standards Track diff --git a/bip-0112.mediawiki b/bip-0112.mediawiki index 9c2d199..e19e0e9 100644 --- a/bip-0112.mediawiki +++ b/bip-0112.mediawiki @@ -138,10 +138,10 @@ A simple output, paying to Alice might then look like: HASH160 <revokehash> EQUAL IF - <Bob key hash> + <Bob's pubkey> ELSE "24h" CHECKSEQUENCEVERIFY DROP - <Alice key hash> + <Alice's pubkey> ENDIF CHECKSIG @@ -153,10 +153,10 @@ With CHECKLOCKTIMEVERIFY, this would look like: HASH160 <revokehash> EQUAL IF - <Bob key hash> + <Bob's pubkey> ELSE "2015/12/15" CHECKLOCKTIMEVERIFY DROP - <Alice key hash> + <Alice's pubkey> ENDIF CHECKSIG @@ -181,13 +181,13 @@ Alice might look like the following in Alice's commitment transaction: IF "24h" CHECKSEQUENCEVERIFY 2DROP - <Alice key hash> + <Alice's pubkey> ELSE <Commit-Revocation-Hash> EQUAL NOTIF - "24h" CHECKLOCKTIMEVERIFY DROP + "2015/10/20 10:33" CHECKLOCKTIMEVERIFY DROP ENDIF - <Bob key hash> + <Bob's pubkey> ENDIF CHECKSIG @@ -196,12 +196,12 @@ and correspondingly in Bob's commitment transaction: HASH160 DUP <R-HASH> EQUAL SWAP <Commit-Revocation-Hash> EQUAL ADD IF - <Alice key hash> + <Alice's pubkey> ELSE "2015/10/20 10:33" CHECKLOCKTIMEVERIFY "24h" CHECKSEQUENCEVERIFY 2DROP - <Bob key hash> + <Bob's pubkey> ENDIF CHECKSIG @@ -229,131 +229,126 @@ The 2-way pegged sidechain requires a new REORGPROOFVERIFY opcode, the semantics Refer to the reference implementation, reproduced below, for the precise semantics and detailed rationale for those semantics. - - /* If this flag set, CTxIn::nSequence is NOT interpreted as a - * relative lock-time. */ - static const uint32_t SEQUENCE_LOCKTIME_DISABLE_FLAG = (1 << 31); - - /* If CTxIn::nSequence encodes a relative lock-time and this flag - * is set, the relative lock-time has units of 512 seconds, - * otherwise it specifies blocks with a granularity of 1. */ - static const uint32_t SEQUENCE_LOCKTIME_TYPE_FLAG = (1 << 22); - - /* If CTxIn::nSequence encodes a relative lock-time, this mask is - * applied to extract that lock-time from the sequence field. */ - static const uint32_t SEQUENCE_LOCKTIME_MASK = 0x0000ffff; +<pre> +/* Below flags apply in the context of BIP 68 */ +/* If this flag set, CTxIn::nSequence is NOT interpreted as a + * relative lock-time. */ +static const uint32_t SEQUENCE_LOCKTIME_DISABLE_FLAG = (1 << 31); + +/* If CTxIn::nSequence encodes a relative lock-time and this flag + * is set, the relative lock-time has units of 512 seconds, + * otherwise it specifies blocks with a granularity of 1. */ +static const uint32_t SEQUENCE_LOCKTIME_TYPE_FLAG = (1 << 22); + +/* If CTxIn::nSequence encodes a relative lock-time, this mask is + * applied to extract that lock-time from the sequence field. */ +static const uint32_t SEQUENCE_LOCKTIME_MASK = 0x0000ffff; - case OP_NOP3: - { - if (!(flags & SCRIPT_VERIFY_CHECKSEQUENCEVERIFY)) { - // not enabled; treat as a NOP3 - if (flags & SCRIPT_VERIFY_DISCOURAGE_UPGRADABLE_NOPS) { - return set_error(serror, SCRIPT_ERR_DISCOURAGE_UPGRADABLE_NOPS); - } - break; +case OP_NOP3: +{ + if (!(flags & SCRIPT_VERIFY_CHECKSEQUENCEVERIFY)) { + // not enabled; treat as a NOP3 + if (flags & SCRIPT_VERIFY_DISCOURAGE_UPGRADABLE_NOPS) { + return set_error(serror, SCRIPT_ERR_DISCOURAGE_UPGRADABLE_NOPS); } - - if (stack.size() < 1) - return set_error(serror, SCRIPT_ERR_INVALID_STACK_OPERATION); - - // Note that elsewhere numeric opcodes are limited to - // operands in the range -2**31+1 to 2**31-1, however it is - // legal for opcodes to produce results exceeding that - // range. This limitation is implemented by CScriptNum's - // default 4-byte limit. - // - // Thus as a special case we tell CScriptNum to accept up - // to 5-byte bignums, which are good until 2**39-1, well - // beyond the 2**32-1 limit of the nSequence field itself. - const CScriptNum nSequence(stacktop(-1), fRequireMinimal, 5); - - // In the rare event that the argument may be < 0 due to - // some arithmetic being done first, you can always use - // 0 MAX CHECKSEQUENCEVERIFY. - if (nSequence < 0) - return set_error(serror, SCRIPT_ERR_NEGATIVE_LOCKTIME); - - // To provide for future soft-fork extensibility, if the - // operand has the disabled lock-time flag set, - // CHECKSEQUENCEVERIFY behaves as a NOP. - if ((nSequence & CTxIn::SEQUENCE_LOCKTIME_DISABLE_FLAG) != 0) - break; - - // Compare the specified sequence number with the input. - if (!checker.CheckSequence(nSequence)) - return set_error(serror, SCRIPT_ERR_UNSATISFIED_LOCKTIME); - break; } - - bool TransactionSignatureChecker::CheckSequence(const CScriptNum& nSequence) const - { - // Relative lock times are supported by comparing the passed - // in operand to the sequence number of the input. - const int64_t txToSequence = (int64_t)txTo->vin[nIn].nSequence; - - // Fail if the transaction's version number is not set high - // enough to trigger BIP 68 rules. - if (static_cast<uint32_t>(txTo->nVersion) < 2) - return false; - - // Sequence numbers with their most significant bit set are not - // defined by BIP68. Testing that the transaction's sequence - // number do not have this bit set prevents using this property - // to get around a CHECKSEQUENCEVERIFY check. - if (txToSequence & CTxIn::SEQUENCE_LOCKTIME_DISABLE_FLAG) - return false; - - // Mask off any bits that do not have BIP68 consensus-enforced meaning - // before doing the integer comparisons of ::VerifySequence. - const uint32_t nLockTimeMask = CTxIn::SEQUENCE_LOCKTIME_TYPE_FLAG - | CTxIn::SEQUENCE_LOCKTIME_MASK; - - if (!::VerifySequence(txToSequence & nLockTimeMask, - CTxIn::SEQUENCE_LOCKTIME_TYPE_FLAG, - nSequence & nLockTimeMask)) - return false; - - return true; - } - - static bool VerifySequence(int64_t txToSequence, int64_t nThreshold, const CScriptNum& nSequence) - { - // There are two kinds of nLockTime: lock-by-blockheight - // and lock-by-blocktime, distinguished by whether - // nSequence < nThreshold (CTxIn::SEQUENCE_LOCKTIME_TYPE_FLAG). - // - // We want to compare apples to apples, so fail the script - // unless the type of nSequence being tested is the same as - // the nSequence in the transaction. - if (!( - (txToSequence < nThreshold && nSequence < nThreshold) || - (txToSequence >= nThreshold && nSequence >= nThreshold) - )) - return false; - - // Now that we know we're comparing apples-to-apples, the - // comparison is a simple numeric one. - if (nSequence > txToSequence) - return false; - - return true; - } + if (stack.size() < 1) + return set_error(serror, SCRIPT_ERR_INVALID_STACK_OPERATION); + + // Note that elsewhere numeric opcodes are limited to + // operands in the range -2**31+1 to 2**31-1, however it is + // legal for opcodes to produce results exceeding that + // range. This limitation is implemented by CScriptNum's + // default 4-byte limit. + // + // Thus as a special case we tell CScriptNum to accept up + // to 5-byte bignums, which are good until 2**39-1, well + // beyond the 2**32-1 limit of the nSequence field itself. + const CScriptNum nSequence(stacktop(-1), fRequireMinimal, 5); + + // In the rare event that the argument may be < 0 due to + // some arithmetic being done first, you can always use + // 0 MAX CHECKSEQUENCEVERIFY. + if (nSequence < 0) + return set_error(serror, SCRIPT_ERR_NEGATIVE_LOCKTIME); + + // To provide for future soft-fork extensibility, if the + // operand has the disabled lock-time flag set, + // CHECKSEQUENCEVERIFY behaves as a NOP. + if ((nSequence & CTxIn::SEQUENCE_LOCKTIME_DISABLE_FLAG) != 0) + break; + + // Compare the specified sequence number with the input. + if (!checker.CheckSequence(nSequence)) + return set_error(serror, SCRIPT_ERR_UNSATISFIED_LOCKTIME); + + break; +} + +bool TransactionSignatureChecker::CheckSequence(const CScriptNum& nSequence) const +{ + // Relative lock times are supported by comparing the passed + // in operand to the sequence number of the input. + const int64_t txToSequence = (int64_t)txTo->vin[nIn].nSequence; + + // Fail if the transaction's version number is not set high + // enough to trigger BIP 68 rules. + if (static_cast<uint32_t>(txTo->nVersion) < 2) + return false; + + // Sequence numbers with their most significant bit set are not + // consensus constrained. Testing that the transaction's sequence + // number do not have this bit set prevents using this property + // to get around a CHECKSEQUENCEVERIFY check. + if (txToSequence & CTxIn::SEQUENCE_LOCKTIME_DISABLE_FLAG) + return false; + + // Mask off any bits that do not have consensus-enforced meaning + // before doing the integer comparisons + const uint32_t nLockTimeMask = CTxIn::SEQUENCE_LOCKTIME_TYPE_FLAG | CTxIn::SEQUENCE_LOCKTIME_MASK; + const int64_t txToSequenceMasked = txToSequence & nLockTimeMask; + const CScriptNum nSequenceMasked = nSequence & nLockTimeMask; + + // There are two kinds of nSequence: lock-by-blockheight + // and lock-by-blocktime, distinguished by whether + // nSequenceMasked < CTxIn::SEQUENCE_LOCKTIME_TYPE_FLAG. + // + // We want to compare apples to apples, so fail the script + // unless the type of nSequenceMasked being tested is the same as + // the nSequenceMasked in the transaction. + if (!( + (txToSequenceMasked < CTxIn::SEQUENCE_LOCKTIME_TYPE_FLAG && nSequenceMasked < CTxIn::SEQUENCE_LOCKTIME_TYPE_FLAG) || + (txToSequenceMasked >= CTxIn::SEQUENCE_LOCKTIME_TYPE_FLAG && nSequenceMasked >= CTxIn::SEQUENCE_LOCKTIME_TYPE_FLAG) + )) + return false; + + // Now that we know we're comparing apples-to-apples, the + // comparison is a simple numeric one. + if (nSequenceMasked > txToSequenceMasked) + return false; + + return true; +} +</pre> ==Reference Implementation== A reference implementation is provided by the following pull request: -https://github.com/bitcoin/bitcoin/pull/6564 +https://github.com/bitcoin/bitcoin/pull/7524 ==Deployment== -This BIP is to be deployed by either version-bits BIP9 or by isSuperMajority(). Exact details TDB. +This BIP is to be deployed by "versionbits" BIP9 using bit 0. + +For Bitcoin '''mainnet''', the BIP9 '''starttime''' will be midnight 1st May 2016 UTC (Epoch timestamp 1462060800) and BIP9 '''timeout''' will be midnight 1st May 2017 UTC (Epoch timestamp 1493596800). -It is recommended to deploy BIP68 and BIP113 at the same time as this BIP. +For Bitcoin '''testnet''', the BIP9 '''starttime''' will be midnight 1st March 2016 UTC (Epoch timestamp 1456790400) and BIP9 '''timeout''' will be midnight 1st May 2017 UTC (Epoch timestamp 1493596800). +This BIP must be deployed simultaneously with BIP68 and BIP113 using the same deployment mechanism. ==Credits== @@ -371,6 +366,8 @@ Thanks to Eric Lombrozo and Anthony Towns for contributing example use cases. ==References== +[https://github.com/bitcoin/bips/blob/master/bip-0009.mediawiki BIP 9] Versionbits + [https://github.com/bitcoin/bips/blob/master/bip-0068.mediawiki BIP 68] Relative lock-time through consensus-enforced sequence numbers [https://github.com/bitcoin/bips/blob/master/bip-0065.mediawiki BIP 65] OP_CHECKLOCKTIMEVERIFY diff --git a/bip-0113.mediawiki b/bip-0113.mediawiki index 15fa4f3..8290c13 100644 --- a/bip-0113.mediawiki +++ b/bip-0113.mediawiki @@ -34,8 +34,9 @@ value to monotonically advance, thereby removing the capability for miners to claim more transaction fees by lying about the timestamps of their block. -This proposal seeks to ensure reliable behaviour in locktime calculations as -required by BIP65 (CHECKLOCKTIMEVERIFY), BIP68 (sequence numbers), and BIP112 (CHECKSEQUENCEVERIFY). +This proposal seeks to ensure reliable behaviour in locktime calculations +as required by BIP65 (CHECKLOCKTIMEVERIFY) and matching the behavior of +BIP68 (sequence numbers) and BIP112 (CHECKSEQUENCEVERIFY). ==Specification== @@ -58,11 +59,12 @@ block time for the purpose of checking lock-time constraints: return pbegin[(pend - pbegin)/2]; } -Lock-time constraints are checked by the consensus method IsFinalTx(), -or LockTime() under BIP68. These methods take the block time as one -parameter. This BIP proposes that after activation calls to -IsFinalTx() or LockTime() within consensus code use the return value -of `GetMedianTimePast(pindexPrev)` instead. +Lock-time constraints are checked by the consensus method IsFinalTx(). +This method takes the block time as one parameter. This BIP proposes +that after activation calls to IsFinalTx() within consensus code use +the return value of `GetMedianTimePast(pindexPrev)` instead. + +The new rule applies to all transactions, including the coinbase transaction. A reference implementation of this proposal is provided by the following pull request: @@ -72,33 +74,13 @@ https://github.com/bitcoin/bitcoin/pull/6566 ==Deployment== -We reuse the double-threshold switchover mechanism from BIPs 34 and -66, with the same thresholds, but for nVersion = 8. The new rules are -in effect for every block (at height H) with nVersion = 8 and at least -750 out of 1000 blocks preceding it (with heights H-1000..H-1) also -have nVersion = 8. Furthermore, when 950 out of the 1000 blocks -preceding a block do have nVersion = 8, nVersion = 3 blocks become -invalid, and all further blocks enforce the new rules. - -When assessing the block version as mask of ~0x20000007 must be applied -to work around the complications caused by -[http://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-August/010396.html BIP101's premature use] -of the [https://gist.github.com/sipa/bf69659f43e763540550 undecided version bits proposal]. - -By applying ~0x20000007 with nVersion = 8, the thresholds should be tested -comparing block nVersion >= 4 as this will save a bit for future use. +This BIP is to be deployed by "versionbits" BIP9 using bit 0. -It is recommended that this soft-fork deployment trigger include other related -proposals for improving Bitcoin's lock-time capabilities, including: +For Bitcoin '''mainnet''', the BIP9 '''starttime''' will be midnight 1st May 2016 UTC (Epoch timestamp 1462060800) and BIP9 '''timeout''' will be midnight 1st May 2017 UTC (Epoch timestamp 1493596800). -[https://github.com/bitcoin/bips/blob/master/bip-0065.mediawiki BIP 65]: -CHECKLOCKTIMEVERIFY, +For Bitcoin '''testnet''', the BIP9 '''starttime''' will be midnight 1st March 2016 UTC (Epoch timestamp 1456790400) and BIP9 '''timeout''' will be midnight 1st May 2017 UTC (Epoch timestamp 1493596800). -[https://github.com/bitcoin/bips/blob/master/bip-0068.mediawiki BIP 68]: -Consensus-enforced transaction replacement signalled via sequence numbers, - -and [https://github.com/bitcoin/bips/blob/master/bip-0112.mediawiki BIP 112]: -CHECKSEQUENCEVERIFY. +This BIP must be deployed simultaneously with BIP68 and BIP112 using the same deployment mechanism. ==Acknowledgements== @@ -121,6 +103,9 @@ concerns with existing protocols. ==References== + +[https://github.com/bitcoin/bips/blob/master/bip-0009.mediawiki BIP9: Versionbits] + [https://github.com/bitcoin/bips/blob/master/bip-0065.mediawiki BIP65: OP_CHECKLOCKTIMEVERIFY] [https://github.com/bitcoin/bips/blob/master/bip-0068.mediawiki BIP68: Consensus-enforced transaction replacement signaled via sequence numbers] diff --git a/bip-0114.mediawiki b/bip-0114.mediawiki new file mode 100644 index 0000000..aee8646 --- /dev/null +++ b/bip-0114.mediawiki @@ -0,0 +1,230 @@ +<pre> + BIP: 114 + Title: Merkelized Abstract Syntax Tree + Author: Johnson Lau <jl2012@xbt.hk> + Status: Draft + Type: Standards Track + Created: 2016-04-02 +</pre> + +==Abstract== +This BIP defines a new witness program type that uses a Merkle tree to encode mutually exclusive branches in a script. This enables complicated redemption conditions that are currently not possible, improves privacy by hiding unexecuted scripts, and allows inclusion of non-consensus enforced data with very low or no additional cost. + +==Motivation== +===Evolution of Bitcoin script system=== +Bitcoin uses a script system to specify the conditions for redemption of transaction outputs. In its original design, the conditions for redemption are directly recorded in the scriptPubKey by the sender of the funds. This model has several drawbacks, particularly for complicated scripts: +# It could be difficult for the receiver to specify the conditions; +# Large scripts take up more UTXO space; +# The sender will pay for the additional block space; +# To prevent DoS attack, scripts are limited to 10,000 bytes and 201 op codes; +# Any unexecuted branches and non-consensus enforced data in the script are visible to the public, consuming block space while damaging privacy. + +The [[bip-0016.mediawiki|BIP16]] (Pay-to-script-hash, "P2SH") fixes the first 3 problems by using a fixed-length 20-byte script hash in the scriptPubKey, and moving the responsibility for supplying the script to the redeemer. However, due to the data push size limit in script, a P2SH script may not be bigger than 520 bytes. Also, P2SH still requires the redeemer to publish all unexecuted branches of the script. + +The [[bip-0141.mediawiki|BIP141]] defines 2 new types of scripts that support segregated witness. The pay-to-witness-script-hash (P2WSH) is similar to P2SH is many ways. By supplying the script in witness, P2WSH restores the original 10,000 byte script limit. However, it still requires publishing of unexecuted branches. + +===Merkelized Abstract Syntax Tree=== +The idea of Merkelized Abstract Syntax Tree (MAST) is to use a Merkle tree to encode mutually exclusive branches in a script. When spending, the redeemer may provide only the branch they are executing, and hashes that connect the branch to the fixed size Merkel root. This reduces the size of redemption stack from O(n) to O(log n) (n as the number of mutually exclusive branches). This enables complicated redemption conditions that is currently not possible due to the script size and op code limit, improves privacy by hiding unexecuted branches, and allows inclusion of non-consensus enforced data with very low or no additional cost. + +==Specification== +In [[bip-0141.mediawiki|BIP141]], witness programs with a version byte of 1 or larger are considered to be anyone-can-spend scripts. The following new validation rules are applied if the witness program version byte is 1 and the program size is 32 bytes. The witness program is the <code>MAST Root</code>. + +To redeem an output of this kind, the witness must consist of an input stack to feed to the script, followed by a <code>Postion</code> value, a serialized Merkle path (<code>Path</code>), and a serialized script (<code>MAST Script</code>). + +The <code>Position</code>, <code>Path</code>, and <code>MAST Script</code> are popped off the initial witness stack. + +The double-SHA256 of the MAST Script (≤ TBD bytes) must be correctly connected to the <code>MAST Root</code> with the <code>ComputeMerkleRootFromBranch</code> function, with the specified <code>Path</code> and <code>Position</code>. + +<code>Path</code> is the serialized Merkle path for the <code>MAST Script</code>. Size of <code>Path</code> must be a multiple of 32 bytes, and not more than 1024 bytes (which allows 32 levels). Each 32 byte word is a double-SHA256 merkle node in the merkle branch connecting to the <code> MAST Root</code>. If the size of <code>Path</code> is zero, the double-SHA256 of the <code>MAST Script</code> must match the <code>MAST Root</code>. + +<code>Position</code> indicates the location of the <code>MAST Script</code> in the Merkle tree, with zero indicating the leftmost position. It is an unsigned little-endian integer with not more than 4 bytes. It must be encoded in the most parsimonious way possible, with no leading zero and not larger than the maximum number of items allowed by the depth of the tree (as implied by the size of <code>Path</code>). + +The <code>MAST Script</code> is then deserialized, and executed after normal script evaluation with the remaining witness stack (≤ TBD bytes for each stack item). The script must not fail, and result in exactly a single TRUE on the stack. + +Sigops in MAST program are counted to the block sigop limit in the same way as the version 0 witness program (see BIP141). + +If the version byte is 1, but the witness program is not 32 bytes, the script must fail. + +== Examples == +=== Calculation of MAST Root === +To calculate the MAST Root for 4 branches, the double-SHA256 of each branch is first calculated: + <1> EQUAL (0x5187), HASH256=3b647cb856a965fa6feffb4621eb2a4f4c2453693b2f64e021383ccbd80a1abb + <2> EQUAL (0x5287), HASH256=37772654bdce9b3d59e1169ea16ddbaa8a2ae8ee265db64863d0b76f02c882fa + <3> EQUAL (0x5387), HASH256=2e972642436151cd96e4b0868077b6362ffb5eb30b420a6f1c5e1c6fff02bc33 + <4> EQUAL (0x5487), HASH256=4c954fc1e635ce8417341465f85b59d700806f6e57bb96b2a25bec5ca3f9f154 + +Serialize the hashes of the first pair and calculate the hash. Same for the other pair: + HASH256(HASH256(5187)|HASH256(5287)) = 64fbdfc0a82ecc3b33434bfea63440e9a5275fa5e533200d2eaf18281e8b28b6 + HASH256(HASH256(5387)|HASH256(5487)) = aeadea837d5e640a1444208f7aca3be63bc8ab3c6b28a19878a00cc9c631ac31 + +Serialize the 2 hashes from the previous step and calculate the hash, which is the <code>MAST Root</code>: + MAST Root = 6746003b5c9d342b2c210d406802c351e7eb5943412dcfc4718be625a8a59c0e + +The scriptPubKey with native witness program is: + <1> <0x6746003b5c9d342b2c210d406802c351e7eb5943412dcfc4718be625a8a59c0e> + (0x51206746003b5c9d342b2c210d406802c351e7eb5943412dcfc4718be625a8a59c0e) + +To redeem with the <code><4> EQUAL</code> branch, the witness is + 04 (Stack for evaluation) + 03 (Position) + 2e972642436151cd96e4b0868077b6362ffb5eb30b420a6f1c5e1c6fff02bc3364fbdfc0a82ecc3b33434bfea63440e9a5275fa5e533200d2eaf18281e8b28b6 (Path) + 5487 (MAST Script) + +=== Imbalance MAST === +When constructing a MAST, if the user believes that some of the branches are more likely to be executed, they may put them closer to the <code>MAST Root</code>. It will save some witness space when the preferred branches are actually executed. + +=== Escrow with Timeout === +The following is the "Escrow with Timeout" example in [[bip-0112.mediawiki|BIP112]]: + IF + 2 <Alice's pubkey> <Bob's pubkey> <Escrow's pubkey> 3 CHECKMULTISIGVERIFY + ELSE + "30d" CHECKSEQUENCEVERIFY DROP + <Alice's pubkey> CHECKSIGVERIFY + ENDIF + +Using compressed public key, the size of this script is 150 bytes. + +With MAST, this script could be broken down into 2 mutually exclusive branches: + 2 <Alice's pubkey> <Bob's pubkey> <Escrow's pubkey> 3 CHECKMULTISIGVERIFY (105 bytes) + "30d" CHECKSEQUENCEVERIFY DROP <Alice's pubkey> CHECKSIGVERIFY (42 bytes) + +With 2 branches, the <code>Path</code> will be 32 bytes (as the hash of the unexecuted branch), and the <code>Position</code> will be 1 byte as either 0 or 1. Since only one branch will be published, it is more difficult for a blockchain analyst to determine the details of the escrow. + +=== Hashed Time-Lock Contract === +The following is the "Hashed TIme-Lock Contract" example in [[bip-0112.mediawiki|BIP112]]: + HASH160 DUP <R-HASH> EQUAL + IF + "24h" CHECKSEQUENCEVERIFY + 2DROP + <Alice's pubkey> + ELSE + <Commit-Revocation-Hash> EQUAL + NOTIF + "Timestamp" CHECKLOCKTIMEVERIFY DROP + ENDIF + <Bob's pubkey> + ENDIF + CHECKSIG + +To create a MAST Root, it is flattened to 3 mutually exclusive branches: + HASH160 <R-HASH> EQUALVERIFY "24h" CHECKSEQUENCEVERIFY DROP <Alice's pubkey> CHECKSIG + HASH160 <Commit-Revocation-Hash> EQUALVERIFY <Bob's pubkey> CHECKSIG + "Timestamp" CHECKLOCKTIMEVERIFY DROP <Bob's pubkey> CHECKSIG + +which significantly improves readability and reduces the witness size when it is redeemed. + +=== Large multi-signature constructs === +The current CHECKMULTISIG supports up to 20 public keys. Although it is possible to extend it beyond 20 keys by using multiple CHECKSIGs and IF/ELSE conditions, the construction could be very complicated and soon use up the 10,000 bytes and 201 op codes limit. + +With MAST, large and complex multi-signature constructs could be flattened to many simple CHECKMULTISIG conditions. For example, a 3-of-2000 multi-signature scheme could be expressed as 1,331,334,000 3-of-3 CHECKMULTISIGs, which forms a 31-level MAST. The scriptPubKey still maintains a fixed size of 34 bytes, and the redemption witness will be very compact, with less than 1,500 bytes. + +=== Commitment of non-consensus enforced data === +Currently, committing non-consensus enforced data in the scriptPubKey requires the use of OP_RETURN which occupies additional block space. With MAST, users may commit such data as a branch. Depends on the number of executable branches, inclusion of such a commitment may incur no extra witness space, or 32 bytes at most. + +An useful case would be specifying "message-signing keys", which are not valid for spending, but allow users to sign any message without touching the cold storage "funding key". + +== Backward compatibility == +As a soft fork, older software will continue to operate without modification. Non-upgraded nodes, however, will consider MAST programs as anyone-can-spend scripts. Wallets should always be wary of anyone-can-spend scripts and treat them with suspicion. + +== Deployment == +This BIP depends on [[bip-0141.mediawiki|BIP141]] and will be deployed by version-bits [[bip-0009.mediawiki|BIP9]] after BIP141 is enforced. Exact details TBD. + +== Credits == +The idea of MAST originates from Russell O’Connor, Pieter Wuille, and [https://bitcointalk.org/index.php?topic=255145.msg2757327#msg2757327 Peter Todd]. + +== Reference Implementation == +https://github.com/jl2012/bitcoin/tree/segwit_mast + +<source lang="cpp"> +//New rules apply if version byte is 1 and witness program size is 32 bytes +if (witversion == 1) { + if (program.size() == 32) { + + //Witness stack must have at least 3 items + if (witness.stack.size() < 3) + return set_error(serror, SCRIPT_ERR_WITNESS_PROGRAM_MISMATCH); + + //Script is the last witness stack item + scriptPubKey = CScript(witness.stack.back().begin(), witness.stack.back().end()); + uint256 hashScriptPubKey; + CHash256().Write(&scriptPubKey[0], scriptPubKey.size()).Finalize(hashScriptPubKey.begin()); + + //Path is the second last witness stack item + std::vector<unsigned char> pathdata = witness.stack.at(witness.stack.size() - 2); + + // Size of Path must be a multiple of 32 bytes (0 byte is allowed) + if (pathdata.size() & 0x1F) + return set_error(serror, SCRIPT_ERR_WITNESS_PROGRAM_MISMATCH); + + // Depth of the tree is size of Path divided by 32 + unsigned int depth = pathdata.size() >> 5; + + // Maximum allowed depth is 32 + if (depth > 32) + return set_error(serror, SCRIPT_ERR_WITNESS_PROGRAM_MISMATCH); + std::vector<uint256> path; + path.resize(depth); + for (unsigned int i = 0; i < depth; i++) + memcpy(path[i].begin(), &pathdata[32 * i], 32); + + //Position is the third last witness stack item + std::vector<unsigned char> positiondata = witness.stack.at(witness.stack.size() - 3); + + //Position may have 4 bytes at most + if (positiondata.size() > 4) + return set_error(serror, SCRIPT_ERR_WITNESS_PROGRAM_MISMATCH); + + uint32_t position = 0; + + //Position is an unsigned little-endian integer with no leading zero byte + if (positiondata.size() > 0) { + if (positiondata.back() == 0x00) + return set_error(serror, SCRIPT_ERR_WITNESS_PROGRAM_MISMATCH); + for (size_t i = 0; i != positiondata.size(); ++i) + position |= static_cast<uint32_t>(positiondata[i]) << 8 * i; + } + + //Position must not be larger than the maximum number of items allowed by the depth of tree + if (depth < 32) { + if (position >= (1U << depth)) + return set_error(serror, SCRIPT_ERR_WITNESS_PROGRAM_MISMATCH); + } + + //Calculate the Merkle Root and compare with the witness program + uint256 root = ComputeMerkleRootFromBranch(hashScriptPubKey, path, position); + if (memcmp(root.begin(), &program[0], 32)) + return set_error(serror, SCRIPT_ERR_WITNESS_PROGRAM_MISMATCH); + + //Remaining stack items used for evaluation + stack = std::vector<std::vector<unsigned char> >(witness.stack.begin(), witness.stack.end() - 3); + } + + else { + //Invalid if version byte is 1 but witness program size is not 32 bytes + return set_error(serror, SCRIPT_ERR_WITNESS_PROGRAM_WRONG_LENGTH); + } +} +</source> + +Copying from <code>src/consensus/merkle.cpp</code>: +<source lang="cpp"> +uint256 ComputeMerkleRootFromBranch(const uint256& leaf, const std::vector<uint256>& vMerkleBranch, uint32_t nIndex) { + uint256 hash = leaf; + for (std::vector<uint256>::const_iterator it = vMerkleBranch.begin(); it != vMerkleBranch.end(); ++it) { + if (nIndex & 1) { + hash = Hash(BEGIN(*it), END(*it), BEGIN(hash), END(hash)); + } else { + hash = Hash(BEGIN(hash), END(hash), BEGIN(*it), END(*it)); + } + nIndex >>= 1; + } + return hash; +} +</source> + + +== References == +*[[bip-0141.mediawiki|BIP141 Segregated Witness (Consensus layer)]] + +== Copyright == +This document is placed in the public domain. diff --git a/bip-0126.mediawiki b/bip-0126.mediawiki new file mode 100644 index 0000000..eed0c3e --- /dev/null +++ b/bip-0126.mediawiki @@ -0,0 +1,98 @@ +<pre> + BIP: 126 + Title: Best Practices for Heterogeneous Input Script Transactions + Author: Kristov Atlas <kristov@openbitcoinprivacyproject.org> + Status: Draft + Type: Informational + Created: 2016-02-10 +</pre> + +==Abstract== + +When a Bitcoin transaction contains inputs that reference previous transaction outputs sent to different Bitcoin addresses, personally identifiable information of the user will leak into the blockchain in an uncontrolled manner. While undesirable, these transactions are frequently unavoidable due to the natural fragmentation of wallet balances over time. + +This document proposes a set of best practice guidelines which minimize the uncontrolled disclosure of personally identifiable information by defining standard forms for transactions containing heterogenous input scripts. + +==Copyright== + +This BIP is in the public domain. + +==Definitions== + +* '''Heterogenous input script transaction (HIT)''': A transaction containing multiple inputs where the scripts of the previous transaction outputs being consumed are not identical (e.g. a transaction spending outputs which were sent to more than one Bitcoin address) +* '''Unavoidable heterogenous input script transaction''': A HIT created as a result of a user’s desire to create a new output with a value larger than the value of his wallet's largest existing unspent output +* '''Intentional heterogenous input script transaction''': A HIT created as part of a user protection protocol for reducing uncontrolled disclosure of personally-identifying information (PII) + +Throughout this procedure, when input scripts are evaluated for uniqueness, "input script" should be interpreted to mean, "the script of the previous output referenced by an input to a transaction". + +==Motivations== + +The recommendations in this document are designed to accomplish three goals: + +# Maximise the effectiveness of user-protecting protocols: Users may find that protection protocols are counterproductive if such transactions have a distinctive fingerprint which renders them ineffective. +# Minimise the adverse consequences of unavoidable heterogenous input transactions: If unavoidable HITs are indistinguishable from intentional HITs, a user creating an unavoidable HIT benefits from ambiguity with respect to graph analysis. +# Limiting the effect on UTXO set growth: To date, non-standardized intentional HITs tend to increase the network's UTXO set with each transaction; this standard attempts to minimize this effect by standardizing unavoidable and intentional HITs to limit UTXO set growth. + +In order to achieve these goals, this specification proposes a set of best practices for heterogenous input script transaction creation. These practices accommodate all applicable requirements of both intentional and unavoidable HITs while maximising the effectiveness of both in terms of preventing uncontrolled disclosure of PII. + +In order to achieve this, two forms of HIT are proposed: Standard form and alternate form. + +==Standard form heterogenous input script transaction== + +===Rules=== + +A HIT is Standard form if it adheres to all of the following rules: + +# The number of unique output scripts must be equal to the number of unique inputs scripts (irrespective of the number of inputs and outputs). +# All output scripts must be unique. +# At least one pair of outputs must be of equal value. +# The largest output in the transaction is a member of a set containing at least two identically-sized outputs. + +===Rationale=== + +The requirement for equal numbers of unique input/output scripts instead of equal number of inputs/outputs accommodates user-protecting UTXO selection behavior. Wallets may contain spendable outputs with identical scripts due to intentional or accidental address reuse, or due to dusting attacks. In order to minimise the adverse consequences of address reuse, any time a UTXO is included in a transaction as an input, all UTXOs with the same spending script should also be included in the transaction. + +The requirement that all output scripts are unique prevents address reuse. Restricting the number of outputs to the number of unique input scripts prevents this policy from growing the network’s UTXO set. A standard form HIT transaction will always have a number of inputs greater than or equal to the number of outputs. + +The requirement for at least one pair of outputs in an intentional HIT to be of equal value results in optimal behavior, and causes intentional HITs to resemble unavoidable HITs. + +==Alternate form heterogenous input script transactions== + +The formation of a standard form HIT is not possible in the following cases: + +# The HIT is unavoidable, and the user’s wallet contains an insufficient number or size of UTXOs to create a standard form HIT. +# The user wishes to reduce the number of utxos in their wallet, and does not have any sets of utxos with identical scripts. + +When one of the following cases exist, a compliant implementation may create an alternate form HIT by constructing a transaction as follows: + +===Procedure=== + +# Find the smallest combination of inputs whose value is at least the value of the desired spend. +## Add these inputs to the transaction. +## Add a spend output to the transaction. +## Add a change output to the transaction containing the difference between the current set of inputs and the desired spend. +# Repeat step 1 to create a second pair of outputs, where one output has the same value as the spend output of the previous step. +# (optional) Repeat step 2 until the desired number of inputs have been consumed and/or the desired number outputs have been created. +# Adjust the change outputs as necessary to pay the desired transaction fee. + +Clients which create intentional HITs must have the capability to form alternate form HITs, and must do so for a non-zero fraction of the transactions they create. + +===Rules=== + +An HIT formed via the preceding procedure will adhere to the following conditions: + +# The number of unique inputs scripts must exceed the number of output scripts. +# All output scripts must be unique. +# At least one pair of outputs must be of equal value. +## "Standard outputs" refers to the set of outputs with equal value +## "Standard value" refers to the value of the standard outputs +## "Change outputs" refers to all outputs which are not standard outputs +# For a HIT containing n standard outputs, there must exist at least one possible way to organize the inputs and outputs into n sets, where all sets satisfy the following: +## The set contains one or more inputs, exactly one standard output, and exactly one change output +## An input or output that appears in one set must not appear in any other set +## The sum of the inputs in the set minus the value of the change output is equal to the standard value with a tolerance equal to the transaction fee. +## Change outputs with a value of zero (virtual change outputs) are permitted. The are defined for the purpose of testing whether or not a HIT adheres to this specification but are not present in the version of the transaction which is broadcast to the network. + +==Non-compliant heterogenous input script transactions== + +If a user wishes to create an output that is larger than half the total size of their spendable outputs, or if their inputs are not distributed in a manner in which the alternate form procedure can be completed, then the user can not create a transaction which is compliant with this procedure. diff --git a/bip-0131.mediawiki b/bip-0131.mediawiki index c30ef54..1efe713 100644 --- a/bip-0131.mediawiki +++ b/bip-0131.mediawiki @@ -77,6 +77,26 @@ the user when their wallet contains many UTXOs that qualify it to benefit from a coalescing transaction. Wallets should not simply replace non-coalescing transactions with coalescing transactions in all instances. +== Isn't this BIP bad because it encourage address re-use? == + +Address re-use comes in two forms: re-use by the ''sender'', and re-use by the ''receiver''. + +Re-use by the sender is basically using the same address for the change output. This is generally considered bad +since people looking through your transaction history can determine who you do business with. When +you generate a new address for every change, your privacy is conserved as it is impossible to know which +output is a recipient, and which output is the change output. This BIP has '''no effect''' on re-use +by the sender. + +On the other hand, address re-use by the ''receiver'' occurs under completely different circumstances. +When you publish an address and have multiple people send to that address, you are engaging in address re-use +from the receiver. This activity has historically been considered bad because it leads to re-using a private key. +When you re-use a private key too many times, you run the risk of an attacker performing statistical analysis +on the multiple signatures, which can lead to an attacker finding out your private key. + +This BIP introduces a way to spend multiple inputs ''without'' re-using the private key. In a sense, this BIP +fixes the problem that makes address re-use bad for the receiver. After this BIP becomes implemented +and deployed, address re-use by the receiver will no longer be considered bad form. + ==Copyright== This document is placed in the public domain. diff --git a/bip-0133.mediawiki b/bip-0133.mediawiki new file mode 100644 index 0000000..7d98f87 --- /dev/null +++ b/bip-0133.mediawiki @@ -0,0 +1,48 @@ +<pre> + BIP: 133 + Title: feefilter message + Author: Alex Morcos <morcos@chaincode.com> + Status: Draft + Type: Standards Track + Created: 2016-02-13 +</pre> + +==Abstract== + +Add a new message, "feefilter", which serves to instruct peers not to send "inv"'s to the node for transactions with fees below the specified fee rate. + +==Motivation== + +The concept of a limited mempool was introduced in Bitcoin Core 0.12 to provide protection against attacks or spam transactions of low fees that are not being mined. A reject filter was also introduced to help prevent repeated requests for the same transaction that might have been recently rejected for insufficient fee. These methods help keep resource utilization on a node from getting out of control. + +However, there are limitations to the effectiveness of these approaches. The reject filter is reset after every block which means transactions that are inv'ed over a longer time period will be rerequested and there is no method to prevent requesting the transaction the first time. Furthermore, inv data is sent at least once either to or from each peer for every transaction accepted to the mempool and there is no mechanism by which to know that an inv sent to a given peer would not result in a getdata request because it represents a transaction with too little fee. + +After receiving a feefilter message, a node can know before sending an inv that a given transaction's fee rate is below the minimum currently required by a given peer, and therefore the node can skip relaying an inv for that transaction to that peer. + +==Specification== + +# The feefilter message is defined as a message containing an int64_t where pchCommand == "feefilter" +# Upon receipt of a "feefilter" message, the node will be permitted, but not required, to filter transaction invs for transactions that fall below the feerate provided in the feefilter message interpreted as satoshis per kilobyte. +# The fee filter is additive with a bloom filter for transactions so if an SPV client were to load a bloom filter and send a feefilter message, transactions would only be relayed if they passed both filters. +# Inv's generated from a mempool message are also subject to a fee filter if it exists. +# Feature discovery is enabled by checking protocol version >= 70013 + +==Considerations== +The propagation efficiency of transactions across the network should not be adversely affected by this change. In general, transactions which are not accepted to a node's mempool are not relayed by this node and the functionality implemented with this message is meant only to filter those transactions. There could be a small number of edge cases where a node's mempool min fee is actually less than the filter value a peer is aware of and transactions with fee rates between these values will now be newly inhibited. + +Feefilter messages are not sent to whitelisted peers if the "-whitelistforcerelay" option is set. In that case, transactions are intended to be relayed even if they are not accepted to the mempool. + +There are privacy concerns with deanonymizing a node by the fact that it is broadcasting identifying information about its mempool min fee. To help ameliorate this concern, the implementation quantizes the filter value broadcast with a small amount of randomness, in addition, the messages are broadcast to different peers at individually randomly distributed times. + +If a node is using prioritisetransaction to accept transactions whose actual fee rates might fall below the node's mempool min fee, it may want to consider disabling the fee filter to make sure it is exposed to all possible txid's. + +==Backward compatibility== + +Older clients remain fully compatible and interoperable after this change. Also, clients implementing this BIP can choose to not send any feefilter messages. + +==Implementation== + +https://github.com/bitcoin/bitcoin/pull/7542 + +==Copyright== +This document is placed in the public domain. diff --git a/bip-0141.mediawiki b/bip-0141.mediawiki index a73cf35..27a59f9 100644 --- a/bip-0141.mediawiki +++ b/bip-0141.mediawiki @@ -48,7 +48,7 @@ A new <code>wtxid</code> is defined: the double SHA256 of the new serialization Format of <code>nVersion</code>, <code>txins</code>, <code>txouts</code>, and <code>nLockTime</code> are same as traditional serialization. -The <code>marker</code> MUST be <code>0x00</code>. +The <code>marker</code> MUST be a 1-byte zero value: <code>0x00</code>. The <code>flag</code> MUST be a 1-byte non-zero value. Currently, <code>0x01</code> MUST be used. @@ -60,28 +60,30 @@ A non-witness program (defined hereinafter) txin MUST be associated with an empt A new block rule is added which requires a commitment to the <code>wtxid</code>. The <code>wtxid</code> of coinbase transaction is assumed to be <code>0x0000....0000</code>. -A witness root hash is calculated with all those <code>wtxid</code> as leaves, in a way similar to the hashMerkleRoot in the block header. +A <code>witness root hash</code> is calculated with all those <code>wtxid</code> as leaves, in a way similar to the <code>hashMerkleRoot</code> in the block header. -The commitment is recorded in a scriptPubKey of the coinbase transaction. It must be at least 38 bytes, with the first 6-byte of <code>0x6a24aa21a9ed</code>, that is: +The commitment is recorded in a <code>scriptPubKey</code> of the coinbase transaction. It must be at least 38 bytes, with the first 6-byte of <code>0x6a24aa21a9ed</code>, that is: 1-byte - OP_RETURN (0x6a) 1-byte - Push the following 36 bytes (0x24) 4-byte - Commitment header (0xaa21a9ed) - 32-byte - Commitment hash: Double-SHA256(witness root hash|witness nonce) + 32-byte - Commitment hash: Double-SHA256(witness root hash|witness reserved value) 39th byte onwards: Optional data with no consensus meaning -and the coinbase's input's witness must consist of a single 32-byte array for the witness nonce. +and the coinbase's input's witness must consist of a single 32-byte array for the <code>witness reserved value</code>. -If there are more than one scriptPubKey matching the pattern, the one with highest output index is assumed to be the commitment. +If there are more than one <code>scriptPubKey</code> matching the pattern, the one with highest output index is assumed to be the commitment. + +If all transactions in a block do not have witness data, the commitment is optional. === Witness program === -A scriptPubKey (or redeemScript as defined in BIP16/P2SH) that consists of a 1-byte push opcode (for 0 to 16) followed by a data push between 2 and 32 bytes gets a new special meaning. The value of the first push is called the "version byte". The following byte vector pushed is called the "witness program". +A <code>scriptPubKey</code> (or <code>redeemScript</code> as defined in BIP16/P2SH) that consists of a 1-byte push opcode (for 0 to 16) followed by a data push between 2 and 40 bytes gets a new special meaning. The value of the first push is called the "version byte". The following byte vector pushed is called the "witness program". There are two cases in which witness validation logic are triggered. Each case determines the location of the witness version byte and program, as well as the form of the scriptSig: -# Triggered by a scriptPubKey that is exactly a push of a version byte, plus a push of a witness program. The scriptSig must be exactly empty. -# Triggered when a scriptPubKey is a P2SH script, and the BIP16 redeemScript pushed in the scriptSig is exactly a push of a version byte plus a push of a witness program. The scriptSig must be exactly a push of the BIP16 redeemScript. +# Triggered by a <code>scriptPubKey</code> that is exactly a push of a version byte, plus a push of a witness program. The scriptSig must be exactly empty or validation fails. (''"native witness program"'') +# Triggered when a <code>scriptPubKey</code> is a P2SH script, and the BIP16 <code>redeemScript</code> pushed in the <code>scriptSig</code> is exactly a push of a version byte plus a push of a witness program. The <code>scriptSig</code> must be exactly a push of the BIP16 <code>redeemScript</code> or validation fails. (''"P2SH witness program"'') If the version byte is 0, and the witness program is 20 bytes: * It is interpreted as a pay-to-witness-public-key-hash (P2WPKH) program. @@ -91,9 +93,9 @@ If the version byte is 0, and the witness program is 20 bytes: If the version byte is 0, and the witness program is 32 bytes: * It is interpreted as a pay-to-witness-script-hash (P2WSH) program. -* The witness must consist of an input stack to feed to the script, followed by a serialized script ("witnessScript"). -* The witnessScript (≤ 10,000 bytes) is popped off the initial witness stack. SHA256 of the witnessScript must match the 32-byte witness program. -* The witnessScript is deserialized, and executed after normal script evaluation with the remaining witness stack (≤ 520 bytes for each stack item). +* The witness must consist of an input stack to feed to the script, followed by a serialized script (<code>witnessScript</code>). +* The <code>witnessScript</code> (≤ 10,000 bytes) is popped off the initial witness stack. SHA256 of the <code>witnessScript</code> must match the 32-byte witness program. +* The <code>witnessScript</code> is deserialized, and executed after normal script evaluation with the remaining witness stack (≤ 520 bytes for each stack item). * The script must not fail, and result in exactly a single TRUE on the stack. If the version byte is 0, but the witness program is neither 20 nor 32 bytes, the script must fail. @@ -106,17 +108,23 @@ If the version byte is 1 to 16, no further interpretation of the witness program Blocks are currently limited to 1,000,000 bytes (1MB) total size. We change this restriction as follows: -''Block cost'' is defined. The cost of each byte in the existing header and transactions is 4, while the cost of each byte in witness data is 1. +''Block cost'' is defined as ''Base size'' * 3 + ''Total size''. (rationale<ref>Rationale of using a single composite constraint, instead of two separate limits such as 1MB base data and 3MB witness data: Using two separate limits would make mining and fee estimation nearly impossible. Miners would need to solve a complex non-linear optimization problem to find the set of transactions that maximize fees given both constraints, and wallets would not be able to know what to pay as it depends on which of the two conditions is most constrained by the time miners try to produce blocks with their transactions in. Another problem with such an approach is freeloading. Once a set of transactions hit the base data 1MB constraint, up to 3MB extra data could be added to the witness by just minimally increasing the fee. The marginal cost for extra witness space effectively becomes zero in that case.</ref>) + +''Base size'' is the block size in bytes with the original transaction serialization without any witness-related data, as seen by a non-upgraded node. -The new rule is total ''block cost'' ≤ 4,000,000. +''Total size'' is the block size in bytes with transactions serialized as described in [[bip-0144.mediawiki|BIP144]], including base data and witness data. + +The new rule is ''block cost'' ≤ 4,000,000. ==== Sigops ==== Sigops per block is currently limited to 20,000. We change this restriction as follows: -''Sigop cost'' is defined. The cost of a sigop in traditional script is 4, while the cost of a sigop in witness program is 1. +Sigops in the current pubkey script, signature script, and P2SH check script are counted at 4 times their previous value. +The sigop limit is likewise quadrupled to ≤ 80,000. -The new rule is total ''sigop cost'' ≤ 80,000. +In addition, opcodes within the witness program are counted identical to as previously within the P2SH check script. +That is, CHECKSIG in a witness program is counted as only 1 sigop, and CHECKMULTISIG in a witness program is counted as 1 to 20 sigops according to the arguments. This rule applies to both native witness program and P2SH witness program. == Examples == @@ -126,8 +134,8 @@ The following example is a version 0 pay-to-witness-public-key-hash (P2WPKH) wit witness: <signature> <pubkey> scriptSig: (empty) - scriptPubKey: 0 <20-byte-hash> - (0x0014{20-byte-hash}) + scriptPubKey: 0 <20-byte-key-hash> + (0x0014{20-byte-key-hash}) The '0' in scriptPubKey indicates the following push is a version 0 witness program. The length of the witness program indicates that it is a P2WPKH type. The witness must consist of exactly 2 items. The HASH160 of the pubkey in witness must match the witness program. @@ -137,6 +145,24 @@ The signature is verified as Comparing with a traditional P2PKH output, the P2WPKH equivalent occupies 3 less bytes in the scriptPubKey, and moves the signature and public key from scriptSig to witness. +=== P2WPKH nested in BIP16 P2SH === + +The following example is the same P2WPKH witness program, but nested in a BIP16 P2SH output. + + witness: <signature> <pubkey> + scriptSig: <0 <20-byte-key-hash>> + (0x160014{20-byte-key-hash}) + scriptPubKey: HASH160 <20-byte-script-hash> EQUAL + (0xA914{20-byte-script-hash}87) + +The only item in scriptSig is hashed with HASH160, compared against the 20-byte-script-hash in scriptPubKey, and interpreted as: + + 0 <20-byte-key-hash> + +The P2WPKH witness program is then executed as described in the previous example. + +Comparing with the previous example, the scriptPubKey is 1 byte bigger and the scriptSig is 23 bytes bigger. Although a nested witness program is less efficient, its payment address is fully transparent and backward compatible for all Bitcoin reference client since version 0.6.0. + === P2WSH witness program === The following example is an 1-of-2 multi-signature version 0 pay-to-witness-script-hash (P2WSH) witness program. @@ -158,13 +184,13 @@ A P2WSH witness program allows arbitrarily large script as the 520-byte push lim The scriptPubKey occupies 34 bytes, as opposed to 23 bytes of BIP16 P2SH. The increased size improves security against possible collision attacks, as 2<sup>80</sup> work is not infeasible anymore (By the end of 2015, 2<sup>84</sup> hashes have been calculated in Bitcoin mining since the creation of Bitcoin). The spending script is same as the one for an equivalent BIP16 P2SH output but is moved to witness. -=== Witness program nested in BIP16 P2SH === +=== P2WSH nested in BIP16 P2SH === The following example is the same 1-of-2 multi-signature P2WSH witness program, but nested in a BIP16 P2SH output. witness: 0 <signature1> <1 <pubkey1> <pubkey2> 2 CHECKMULTISIG> scriptSig: <0 <32-byte-hash>> - (0x0020{32-byte-hash}) + (0x220020{32-byte-hash}) scriptPubKey: HASH160 <20-byte-hash> EQUAL (0xA914{20-byte-hash}87) @@ -174,18 +200,17 @@ The only item in scriptSig is hashed with HASH160, compared against the 20-byte- The P2WSH witness program is then executed as described in the previous example. -Comparing with the previous example, the scriptPubKey is 11 bytes smaller (with reduced security) while witness is the same. However, it also requires 35 bytes in scriptSig, which is not prunable in transmission. Although a nested witness program is less efficient in many ways, its payment address is fully transparent and backward compatible for all Bitcoin reference client since version 0.6.0. +Comparing with the previous example, the scriptPubKey is 11 bytes smaller (with reduced security) while witness is the same. However, it also requires 35 bytes in scriptSig. === Extensible commitment structure === -The new commitment in coinbase transaction is a hash of the witness root hash and a witness nonce. The nonce currently has no consensus meaning, but in the future allows new commitment values for future softforks. For example, if a new consensus-critical commitment is required in the future, the commitment in -coinbase becomes: +The new commitment in coinbase transaction is a hash of the <code>witness root hash</code> and a <code>witness reserved value</code>. The <code>witness reserved value</code> currently has no consensus meaning, but in the future allows new commitment values for future softforks. For example, if a new consensus-critical commitment is required in the future, the commitment in coinbase becomes: - Double-SHA256(Witness root hash|Hash(new commitment|witness nonce)) + Double-SHA256(Witness root hash|Hash(new commitment|witness reserved value)) -For backward compatibility, the Hash(new commitment|witness nonce) will go to the coinbase witness, and the witness nonce will be recorded in another location specified by the future softfork. Any number of new commitment could be added in this way. +For backward compatibility, the <code>Hash(new commitment|witness reserved value)</code> will go to the coinbase witness, and the <code>witness reserved value</code> will be recorded in another location specified by the future softfork. Any number of new commitment could be added in this way. -Any commitments that are not consensus-critical to Bitcoin, such as merge-mining, MUST NOT use the witness nonce to preserve the ability to do upgrades of the Bitcoin consensus protocol. +Any commitments that are not consensus-critical to Bitcoin, such as merge-mining, MUST NOT use the <code>witness reserved value</code> to preserve the ability to do upgrades of the Bitcoin consensus protocol. The optional data space following the commitment also leaves room for metadata of future softforks, and MUST NOT be used for other purpose. @@ -249,33 +274,23 @@ As a soft fork, older software will continue to operate without modification. N == Deployment == -We reuse the double-threshold IsSuperMajority() switchover mechanism used in -BIP65 with the same thresholds, but for nVersion = 5. The new rules are -in effect for every block (at height H) with nVersion = 5 and at least -750 out of 1000 blocks preceding it (with heights H-1000..H-1) also -have nVersion >= 5. Furthermore, when 950 out of the 1000 blocks -preceding a block do have nVersion >= 5, nVersion < 5 blocks become -invalid, and all further blocks enforce the new rules. +This BIP will be deployed by "version bits" BIP9 with the name "segwit" and using bit 1. -(It should be noted that BIP9 involves permanently setting a high-order bit to -1 which results in nVersion >= all prior IsSuperMajority() soft-forks and thus -no bits in nVersion are permanently lost.) +For Bitcoin mainnet, the BIP9 starttime will be midnight TBD UTC (Epoch timestamp TBD) and BIP9 timeout will be midnight TBD UTC (Epoch timestamp TBD). -=== SPV Clients === - -While SPV clients are unable to fully validate blocks, -they are able to validate block headers and, thus, can check block version and proof-of-work. -SPV clients should reject nVersion < 5 blocks if 950 out of 1000 preceding blocks have -nVersion >= 5 to prevent false confirmations from the remaining 5% of -non-upgraded miners when the 95% threshold has been reached. +For Bitcoin testnet, the BIP9 starttime will be midnight 1 May 2016 UTC (Epoch timestamp 1462060800) and BIP9 timeout will be midnight 1 May 2017 UTC (Epoch timestamp 1493596800). == Credits == Special thanks to Gregory Maxwell for originating many of the ideas in this BIP and Luke-Jr for figuring out how to deploy this as a soft fork. +== Footnotes == + +<references /> + == Reference Implementation == -https://github.com/sipa/bitcoin/commits/segwit +https://github.com/bitcoin/bitcoin/pull/7910 == References == diff --git a/bip-0142.mediawiki b/bip-0142.mediawiki index ff831cb..bb60265 100644 --- a/bip-0142.mediawiki +++ b/bip-0142.mediawiki @@ -2,7 +2,7 @@ BIP: 142 Title: Address Format for Segregated Witness Author: Johnson Lau <jl2012@xbt.hk> - Status: Draft + Status: Deferred Type: Standards Track Created: 2015-12-24 </pre> diff --git a/bip-0143.mediawiki b/bip-0143.mediawiki index 72ef22d..892c027 100644 --- a/bip-0143.mediawiki +++ b/bip-0143.mediawiki @@ -1,4 +1,4 @@ -<pre> +<pre> BIP: 143 Title: Transaction Signature Verification for Version 0 Witness Program Author: Johnson Lau <jl2012@xbt.hk> @@ -14,7 +14,7 @@ This proposal defines a new transaction digest algorithm for signature verificat == Motivation == There are 4 ECDSA signature verification codes in the original Bitcoin script system: CHECKSIG, CHECKSIGVERIFY, CHECKMULTISIG, CHECKMULTISIGVERIFY (“sigops”). According to the sighash type (ALL, NONE, SINGLE, ANYONECANPAY), a transaction digest is generated with a double SHA256 of a serialized subset of the transaction, and the signature is verified against this digest with a given public key. The detailed procedure is described in a Bitcoin Wiki article. <ref name=wiki>[https://en.bitcoin.it/wiki/OP_CHECKSIG]</ref> -Unfortunately, there are at least 2 weaknesses in the original transaction digest algorithm: +Unfortunately, there are at least 2 weaknesses in the original SignatureHash transaction digest algorithm: * For the verification of each signature, the amount of data hashing is proportional to the size of the transaction. Therefore, data hashing grows in O(n<sup>2</sup>) as the number of sigops in a transaction increases. While a 1 MB block would normally take 2 seconds to verify with an average computer in 2015, a 1MB transaction with 5569 sigops may take 25 seconds to verify. This could be fixed by optimizing the digest algorithm by introducing some reusable “midstate”, so the time complexity becomes O(n). <ref>[https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2013-2292 CVE-2013-2292]</ref><ref>[https://bitcointalk.org/?topic=140078 New Bitcoin vulnerability: A transaction that takes at least 3 minutes to verify]</ref><ref>[http://rusty.ozlabs.org/?p=522 The Megatransaction: Why Does It Take 25 Seconds?]</ref> * The algorithm does not involve the amount of Bitcoin being spent by the input. This is usually not a problem for online network nodes as they could request for the specified transaction to acquire the output value. For an offline transaction signing device ("cold wallet"), however, the unknowing of input amount makes it impossible to calculate the exact amount being spent and the transaction fee. To cope with this problem a cold wallet must also acquire the full transaction being spent, which could be a big obstacle in the implementation of lightweight, air-gapped wallet. By including the input value of part of the transaction digest, a cold wallet may safely sign a transaction by learning the value from an untrusted source. In the case that a wrong value is provided and signed, the signature would be invalid and no funding might be lost. <ref>[https://bitcointalk.org/index.php?topic=181734.0 SIGHASH_WITHINPUTVALUE: Super-lightweight HW wallets and offline data]</ref> @@ -28,37 +28,42 @@ A new transaction digest algorithm is defined, but only applicable to sigops in 2. hashPrevouts (32-byte hash) 3. hashSequence (32-byte hash) 4. outpoint (32-byte hash + 4-byte little endian) - 5. scriptCode of the input (varInt for the length + script) + 5. scriptCode of the input (serialized as scripts inside CTxOuts) 6. value of the output spent by this input (8-byte little endian) 7. nSequence of the input (4-byte little endian) 8. hashOutputs (32-byte hash) 9. nLocktime of the transaction (4-byte little endian) 10. sighash type of the signature (4-byte little endian) -All components in the original algorithm, including the behavior <code>OP_CODESEPERATOR</code>, remains unchanged. The only difference is the way of serialization and the inclusion of amount being spent. +Semantics of the original sighash types remain unchanged, except the followings: +# The way of serialization is changed; +# All sighash types commit to the amount being spent by the signed input; +# <code>FindAndDelete</code> of the signature is not applied to the <code>scriptCode</code>; +# <code>OP_CODESEPARATOR</code>(s) after the last executed <code>OP_CODESEPARATOR</code> are not removed from the <code>scriptCode</code>; +# <code>SINGLE</code> does not commit to the input index. When <code>ANYONECANPAY</code> is not set, the semantics are unchanged since <code>hashPrevouts</code> and <code>outpoint</code> together implictly commit to the input index. When <code>SINGLE</code> is used with <code>ANYONECANPAY</code>, omission of the index commitment allows permutation of the input-output pairs, as long as each pair is located at an equivalent index. The items 1, 4, 7, 9, 10 have the same meaning as the original algorithm. <ref name=wiki></ref> The item 5: *For P2WPKH witness program, the scriptCode is <code>0x1976a914{20-byte-pubkey-hash}88ac</code>. *For P2WSH witness program, -**if the <code>witnessScript</code> does not contain any <code>OP_CODESEPERATOR</code>, the <code>scriptCode</code> is a <code>varInt</code> for the length of the <code>witnessScript</code>, followed by the <code>witnessScript</code>. -**if the <code>witnessScript</code> contains any <code>OP_CODESEPERATOR</code>, the <code>scriptCode</code> is the evaluated script, with all <code>OP_CODESEPARATOR</code> and everything up to the last <code>OP_CODESEPARATOR</code> before the signature checking opcode being executed removed, and prepended by a <code>varInt</code> for the length of the truncated script. +**if the <code>witnessScript</code> does not contain any <code>OP_CODESEPARATOR</code>, the <code>scriptCode</code> is the <code>witnessScript</code> serialized as scripts inside CTxOuts. +**if the <code>witnessScript</code> contains any <code>OP_CODESEPARATOR</code>, the <code>scriptCode</code> is the evaluated script, with everything up to and including the last executed <code>OP_CODESEPARATOR</code> before the signature checking opcode being executed removed, serialized as scripts inside CTxOuts. The item 6 is a 8-byte value of the amount of bitcoin spent in this input. <code>hashPrevouts</code>: -*If the ANYONECANPAY flag is not set, hashPrevouts is the double SHA256 of the serialization of all input outpoints; +*If the <code>ANYONECANPAY</code> flag is not set, <code>hashPrevouts</code> is the double SHA256 of the serialization of all input outpoints; *Otherwise, <code>hashPrevouts</code> is a <code>uint256</code> of <code>0x0000......0000</code>. <code>hashSequence</code>: -*If none of the ANYONECANPAY, SINGLE, NONE sighash type is set, hashSequence is the double SHA256 of the serialization of nSequence of all inputs; +*If none of the <code>ANYONECANPAY</code>, <code>SINGLE</code>, <code>NONE</code> sighash type is set, <code>hashSequence</code> is the double SHA256 of the serialization of <code>nSequence</code> of all inputs; *Otherwise, <code>hashSequence</code> is a <code>uint256</code> of <code>0x0000......0000</code>. <code>hashOutputs</code>: -*If the sighash type is neither SINGLE nor NONE, hashOutputs is the double SHA256 of the serialization of all output value (8-byte little endian) with scriptPubKey (<code>varInt</code> for the length + script); -*If sighash type is SINGLE and the input index is not greater than the number of outputs, <code>hashOutputs</code> is the double SHA256 of the output value with <code>scriptPubKey</code> of the same index as the input; -*Otherwise, <code>hashOutputs</code> is a <code>uint256</code> of <code>0x0000......0000</code>. +*If the sighash type is neither <code>SINGLE</code> nor <code>NONE</code>, <code>hashOutputs</code> is the double SHA256 of the serialization of all output amount (8-byte little endian) with <code>scriptPubKey</code> (serialized as scripts inside CTxOuts); +*If sighash type is <code>SINGLE</code> and the input index is smaller than the number of outputs, <code>hashOutputs</code> is the double SHA256 of the output amount with <code>scriptPubKey</code> of the same index as the input; +*Otherwise, <code>hashOutputs</code> is a <code>uint256</code> of <code>0x0000......0000</code>.<ref>In the original algorithm, a <code>uint256</code> of <code>0x0000......0001</code> is commited if the input index for a <code>SINGLE</code> signature is greater than or equal to the number of outputs. In this BIP a <code>0x0000......0000</code> is commited, without changing the semantics.</ref> The <code>hashPrevouts</code>, <code>hashSequence</code>, and <code>hashOutputs</code> calculated in an earlier verification may be reused in other inputs of the same transaction, so that the time complexity of the whole hashing process reduces from O(n<sup>2</sup>) to O(n). @@ -121,7 +126,7 @@ Refer to the reference implementation, reproduced below, for the precise algorit </source> == Example == - +=== Native P2WPKH === The following is an unsigned transaction: 0100000002fff7f7881a8099afa6940d42d1e7f6362bec38171ea3edf433541db4e4ad969f0000000000eeffffffef51e1b804cc89d182d279655c3aa89e815b1b309fe287d9b2b55d57b90ec68a0100000000ffffffff02202cb206000000001976a9148280b37df378db99f66f85c95a783a76ac7a6d5988ac9093510d000000001976a9143bde42dbee7e4dbe6a21b2d50ce2f0167faa815988ac11000000 @@ -134,10 +139,13 @@ Refer to the reference implementation, reproduced below, for the precise algorit nLockTime: 11000000 The first input comes from an ordinary P2PK: - scriptPubKey: 2103c9f4836b9a4f77fc0d81f7bcb01b7f1b35916864b9476c241ce9fc198bd25432ac value: 6.25 + scriptPubKey : 2103c9f4836b9a4f77fc0d81f7bcb01b7f1b35916864b9476c241ce9fc198bd25432ac value: 6.25 + private key : bbc27228ddcb9209d7fd6f36b02f7dfa6252af40bb2f1cbc7a557da8027ff866 The second input comes from a P2WPKH witness program: - scriptPubKey: 00141d0f172a0ecb48aee1be1f2687d2963ae33f71a1, value: 6 + scriptPubKey : 00141d0f172a0ecb48aee1be1f2687d2963ae33f71a1, value: 6 + private key : 619c335025c7f4012e556c2a58b2506e30b8511b53ade95ea316fd8c3286feb9 + public key : 025476c2e83188368da1ff3e292e7acafcdb3566bb0ad253f62fc70f07aeee6357 To sign it with a nHashType of 1 (SIGHASH_ALL): @@ -182,18 +190,355 @@ Refer to the reference implementation, reproduced below, for the precise algorit 02 47304402203609e17b84f6a7d30c80bfa610b5b4542f32a8a0d5447a12fb1366d7f01cc44a0220573a954c4518331561406f90300e8f3358f51928d43c212a8caed02de67eebee01 21025476c2e83188368da1ff3e292e7acafcdb3566bb0ad253f62fc70f07aeee6357 nLockTime: 11000000 +=== P2SH-P2WPKH === + + + The following is an unsigned transaction: 0100000001db6b1b20aa0fd7b23880be2ecbd4a98130974cf4748fb66092ac4d3ceb1a54770100000000feffffff02b8b4eb0b000000001976a914a457b684d7f0d539a46a45bbc043f35b59d0d96388ac0008af2f000000001976a914fd270b1ee6abcaea97fea7ad0402e8bd8ad6d77c88ac92040000 + + nVersion: 01000000 + txin: 01 db6b1b20aa0fd7b23880be2ecbd4a98130974cf4748fb66092ac4d3ceb1a5477 01000000 00 feffffff + txout: 02 b8b4eb0b00000000 1976a914a457b684d7f0d539a46a45bbc043f35b59d0d96388ac + 0008af2f00000000 1976a914fd270b1ee6abcaea97fea7ad0402e8bd8ad6d77c88ac + nLockTime: 92040000 + + The input comes from a P2SH-P2WPKH witness program: + scriptPubKey : a9144733f37cf4db86fbc2efed2500b4f4e49f31202387, value: 10 + redeemScript : 001479091972186c449eb1ded22b78e40d009bdf0089 + private key : eb696a065ef48a2192da5b28b694f87544b30fae8327c4510137a922f32c6dcf + public key : 03ad1d8e89212f0b92c74d23bb710c00662ad1470198ac48c43f7d6f93a2a26873 + + To sign it with a nHashType of 1 (SIGHASH_ALL): + + hashPrevouts: + dSHA256(db6b1b20aa0fd7b23880be2ecbd4a98130974cf4748fb66092ac4d3ceb1a547701000000) + = b0287b4a252ac05af83d2dcef00ba313af78a3e9c329afa216eb3aa2a7b4613a + + hashSequence: + dSHA256(feffffff) + = 18606b350cd8bf565266bc352f0caddcf01e8fa789dd8a15386327cf8cabe198 + + hashOutputs: + dSHA256(b8b4eb0b000000001976a914a457b684d7f0d539a46a45bbc043f35b59d0d96388ac0008af2f000000001976a914fd270b1ee6abcaea97fea7ad0402e8bd8ad6d77c88ac) + = de984f44532e2173ca0d64314fcefe6d30da6f8cf27bafa706da61df8a226c83 + + hash preimage: 01000000b0287b4a252ac05af83d2dcef00ba313af78a3e9c329afa216eb3aa2a7b4613a18606b350cd8bf565266bc352f0caddcf01e8fa789dd8a15386327cf8cabe198db6b1b20aa0fd7b23880be2ecbd4a98130974cf4748fb66092ac4d3ceb1a5477010000001976a91479091972186c449eb1ded22b78e40d009bdf008988ac00ca9a3b00000000feffffffde984f44532e2173ca0d64314fcefe6d30da6f8cf27bafa706da61df8a226c839204000001000000 + + nVersion: 01000000 + hashPrevouts: b0287b4a252ac05af83d2dcef00ba313af78a3e9c329afa216eb3aa2a7b4613a + hashSequence: 18606b350cd8bf565266bc352f0caddcf01e8fa789dd8a15386327cf8cabe198 + outpoint: db6b1b20aa0fd7b23880be2ecbd4a98130974cf4748fb66092ac4d3ceb1a547701000000 + scriptCode: 1976a91479091972186c449eb1ded22b78e40d009bdf008988ac + amount: 00ca9a3b00000000 + nSequence: feffffff + hashOutputs: de984f44532e2173ca0d64314fcefe6d30da6f8cf27bafa706da61df8a226c83 + nLockTime: 92040000 + nHashType: 01000000 + + sigHash: 64f3b0f4dd2bb3aa1ce8566d220cc74dda9df97d8490cc81d89d735c92e59fb6 + signature: 3044022047ac8e878352d3ebbde1c94ce3a10d057c24175747116f8288e5d794d12d482f0220217f36a485cae903c713331d877c1f64677e3622ad4010726870540656fe9dcb01 + + The serialized signed transaction is: 01000000000101db6b1b20aa0fd7b23880be2ecbd4a98130974cf4748fb66092ac4d3ceb1a5477010000001716001479091972186c449eb1ded22b78e40d009bdf0089feffffff02b8b4eb0b000000001976a914a457b684d7f0d539a46a45bbc043f35b59d0d96388ac0008af2f000000001976a914fd270b1ee6abcaea97fea7ad0402e8bd8ad6d77c88ac02473044022047ac8e878352d3ebbde1c94ce3a10d057c24175747116f8288e5d794d12d482f0220217f36a485cae903c713331d877c1f64677e3622ad4010726870540656fe9dcb012103ad1d8e89212f0b92c74d23bb710c00662ad1470198ac48c43f7d6f93a2a2687392040000 + nVersion: 01000000 + marker: 00 + flag: 01 + txin: 01 db6b1b20aa0fd7b23880be2ecbd4a98130974cf4748fb66092ac4d3ceb1a5477 01000000 1716001479091972186c449eb1ded22b78e40d009bdf0089 feffffff + txout: 02 b8b4eb0b00000000 1976a914a457b684d7f0d539a46a45bbc043f35b59d0d96388ac + 0008af2f00000000 1976a914fd270b1ee6abcaea97fea7ad0402e8bd8ad6d77c88ac + witness 02 473044022047ac8e878352d3ebbde1c94ce3a10d057c24175747116f8288e5d794d12d482f0220217f36a485cae903c713331d877c1f64677e3622ad4010726870540656fe9dcb01 2103ad1d8e89212f0b92c74d23bb710c00662ad1470198ac48c43f7d6f93a2a26873 + nLockTime: 92040000 + +=== Native P2WSH === + +This example shows how <code>OP_CODESEPARATOR</code> and out-of-range <code>SIGHASH_SINGLE</code> are processed: + + + + The following is an unsigned transaction: + 0100000002fe3dc9208094f3ffd12645477b3dc56f60ec4fa8e6f5d67c565d1c6b9216b36e0000000000ffffffff0815cf020f013ed6cf91d29f4202e8a58726b1ac6c79da47c23d1bee0a6925f80000000000ffffffff0100f2052a010000001976a914a30741f8145e5acadf23f751864167f32e0963f788ac00000000 + + nVersion: 01000000 + txin: 02 fe3dc9208094f3ffd12645477b3dc56f60ec4fa8e6f5d67c565d1c6b9216b36e 00000000 00 ffffffff + 0815cf020f013ed6cf91d29f4202e8a58726b1ac6c79da47c23d1bee0a6925f8 00000000 00 ffffffff + txout: 01 00f2052a01000000 1976a914a30741f8145e5acadf23f751864167f32e0963f788ac + nLockTime: 00000000 + + The first input comes from an ordinary P2PK: + scriptPubKey: 21036d5c20fa14fb2f635474c1dc4ef5909d4568e5569b79fc94d3448486e14685f8ac value: 1.5625 + private key: b8f28a772fccbf9b4f58a4f027e07dc2e35e7cd80529975e292ea34f84c4580c + signature: 304402200af4e47c9b9629dbecc21f73af989bdaa911f7e6f6c2e9394588a3aa68f81e9902204f3fcf6ade7e5abb1295b6774c8e0abd94ae62217367096bc02ee5e435b67da201 (SIGHASH_ALL) + + The second input comes from a native P2WSH witness program: + scriptPubKey : 00205d1b56b63d714eebe542309525f484b7e9d6f686b3781b6f61ef925d66d6f6a0, value: 49 + witnessScript: 21026dccc749adc2a9d0d89497ac511f760f45c47dc5ed9cf352a58ac706453880aeadab210255a9626aebf5e29c0e6538428ba0d1dcf6ca98ffdf086aa8ced5e0d0215ea465ac + <026dccc749adc2a9d0d89497ac511f760f45c47dc5ed9cf352a58ac706453880ae> CHECKSIGVERIFY CODESEPERATOR <0255a9626aebf5e29c0e6538428ba0d1dcf6ca98ffdf086aa8ced5e0d0215ea465> CHECKSIG + + To sign it with a nHashType of 3 (SIGHASH_SINGLE): + + hashPrevouts: + dSHA256(fe3dc9208094f3ffd12645477b3dc56f60ec4fa8e6f5d67c565d1c6b9216b36e000000000815cf020f013ed6cf91d29f4202e8a58726b1ac6c79da47c23d1bee0a6925f800000000) + = ef546acf4a020de3898d1b8956176bb507e6211b5ed3619cd08b6ea7e2a09d41 + + nVersion: 01000000 + hashPrevouts: ef546acf4a020de3898d1b8956176bb507e6211b5ed3619cd08b6ea7e2a09d41 + hashSequence: 0000000000000000000000000000000000000000000000000000000000000000 + outpoint: 0815cf020f013ed6cf91d29f4202e8a58726b1ac6c79da47c23d1bee0a6925f800000000 + scriptCode: (see below) + amount: 0011102401000000 + nSequence: ffffffff + hashOutputs: 0000000000000000000000000000000000000000000000000000000000000000 (this is the second input but there is only one output) + nLockTime: 00000000 + nHashType: 03000000 + + scriptCode: 4721026dccc749adc2a9d0d89497ac511f760f45c47dc5ed9cf352a58ac706453880aeadab210255a9626aebf5e29c0e6538428ba0d1dcf6ca98ffdf086aa8ced5e0d0215ea465ac + ^^ + (please note that the not-yet-exectued OP_CODESEPARATOR is not removed from the scriptCode) + preimage: 01000000ef546acf4a020de3898d1b8956176bb507e6211b5ed3619cd08b6ea7e2a09d4100000000000000000000000000000000000000000000000000000000000000000815cf020f013ed6cf91d29f4202e8a58726b1ac6c79da47c23d1bee0a6925f8000000004721026dccc749adc2a9d0d89497ac511f760f45c47dc5ed9cf352a58ac706453880aeadab210255a9626aebf5e29c0e6538428ba0d1dcf6ca98ffdf086aa8ced5e0d0215ea465ac0011102401000000ffffffff00000000000000000000000000000000000000000000000000000000000000000000000003000000 + sigHash: 82dde6e4f1e94d02c2b7ad03d2115d691f48d064e9d52f58194a6637e4194391 + public key: 026dccc749adc2a9d0d89497ac511f760f45c47dc5ed9cf352a58ac706453880ae + private key: 8e02b539b1500aa7c81cf3fed177448a546f19d2be416c0c61ff28e577d8d0cd + signature: 3044022027dc95ad6b740fe5129e7e62a75dd00f291a2aeb1200b84b09d9e3789406b6c002201a9ecd315dd6a0e632ab20bbb98948bc0c6fb204f2c286963bb48517a7058e2703 + + scriptCode: 23210255a9626aebf5e29c0e6538428ba0d1dcf6ca98ffdf086aa8ced5e0d0215ea465ac + (everything up to the last executed OP_CODESEPARATOR, including that OP_CODESEPARATOR, are removed) + preimage: 01000000ef546acf4a020de3898d1b8956176bb507e6211b5ed3619cd08b6ea7e2a09d4100000000000000000000000000000000000000000000000000000000000000000815cf020f013ed6cf91d29f4202e8a58726b1ac6c79da47c23d1bee0a6925f80000000023210255a9626aebf5e29c0e6538428ba0d1dcf6ca98ffdf086aa8ced5e0d0215ea465ac0011102401000000ffffffff00000000000000000000000000000000000000000000000000000000000000000000000003000000 + sigHash: fef7bd749cce710c5c052bd796df1af0d935e59cea63736268bcbe2d2134fc47 + public key: 0255a9626aebf5e29c0e6538428ba0d1dcf6ca98ffdf086aa8ced5e0d0215ea465 + private key: 86bf2ed75935a0cbef03b89d72034bb4c189d381037a5ac121a70016db8896ec + signature: 304402200de66acf4527789bfda55fc5459e214fa6083f936b430a762c629656216805ac0220396f550692cd347171cbc1ef1f51e15282e837bb2b30860dc77c8f78bc8501e503 + + The serialized signed transaction is: 01000000000102fe3dc9208094f3ffd12645477b3dc56f60ec4fa8e6f5d67c565d1c6b9216b36e000000004847304402200af4e47c9b9629dbecc21f73af989bdaa911f7e6f6c2e9394588a3aa68f81e9902204f3fcf6ade7e5abb1295b6774c8e0abd94ae62217367096bc02ee5e435b67da201ffffffff0815cf020f013ed6cf91d29f4202e8a58726b1ac6c79da47c23d1bee0a6925f80000000000ffffffff0100f2052a010000001976a914a30741f8145e5acadf23f751864167f32e0963f788ac000347304402200de66acf4527789bfda55fc5459e214fa6083f936b430a762c629656216805ac0220396f550692cd347171cbc1ef1f51e15282e837bb2b30860dc77c8f78bc8501e503473044022027dc95ad6b740fe5129e7e62a75dd00f291a2aeb1200b84b09d9e3789406b6c002201a9ecd315dd6a0e632ab20bbb98948bc0c6fb204f2c286963bb48517a7058e27034721026dccc749adc2a9d0d89497ac511f760f45c47dc5ed9cf352a58ac706453880aeadab210255a9626aebf5e29c0e6538428ba0d1dcf6ca98ffdf086aa8ced5e0d0215ea465ac00000000 + + +This example shows how unexecuted <code>OP_CODESEPARATOR</code> is processed, and <code>SINGLE|ANYONECANPAY</code> does not commit to the input index: + + + + The following is an unsigned transaction: + 0100000002e9b542c5176808107ff1df906f46bb1f2583b16112b95ee5380665ba7fcfc0010000000000ffffffff80e68831516392fcd100d186b3c2c7b95c80b53c77e77c35ba03a66b429a2a1b0000000000ffffffff0280969800000000001976a914de4b231626ef508c9a74a8517e6783c0546d6b2888ac80969800000000001976a9146648a8cd4531e1ec47f35916de8e259237294d1e88ac00000000 + + nVersion: 01000000 + txin: 02 e9b542c5176808107ff1df906f46bb1f2583b16112b95ee5380665ba7fcfc001 00000000 00 ffffffff + 80e68831516392fcd100d186b3c2c7b95c80b53c77e77c35ba03a66b429a2a1b 00000000 00 ffffffff + txout: 02 8096980000000000 1976a914de4b231626ef508c9a74a8517e6783c0546d6b2888ac + 8096980000000000 1976a9146648a8cd4531e1ec47f35916de8e259237294d1e88ac + nLockTime: 00000000 + + The first input comes from a native P2WSH witness program: + scriptPubKey: 0020ba468eea561b26301e4cf69fa34bde4ad60c81e70f059f045ca9a79931004a4d value: 0.16777215 + witnessScript:0063ab68210392972e2eb617b2388771abe27235fd5ac44af8e61693261550447a4c3e39da98ac + 0 IF CODESEPERATOR ENDIF <0392972e2eb617b2388771abe27235fd5ac44af8e61693261550447a4c3e39da98> CHECKSIG + + The second input comes from a native P2WSH witness program: + scriptPubKey: 0020d9bbfbe56af7c4b7f960a70d7ea107156913d9e5a26b0a71429df5e097ca6537 value: 0.16777215 + witnessScript:5163ab68210392972e2eb617b2388771abe27235fd5ac44af8e61693261550447a4c3e39da98ac + 1 IF CODESEPERATOR ENDIF <0392972e2eb617b2388771abe27235fd5ac44af8e61693261550447a4c3e39da98> CHECKSIG + + To sign it with a nHashType of 0x83 (SINGLE|ANYONECANPAY): + + nVersion: 01000000 + hashPrevouts: 0000000000000000000000000000000000000000000000000000000000000000 + hashSequence: 0000000000000000000000000000000000000000000000000000000000000000 + outpoint: (see below) + scriptCode: (see below) + amount: ffffff0000000000 + nSequence: ffffffff + hashOutputs: (see below) + nLockTime: 00000000 + nHashType: 83000000 + + outpoint: e9b542c5176808107ff1df906f46bb1f2583b16112b95ee5380665ba7fcfc00100000000 + scriptCode: 270063ab68210392972e2eb617b2388771abe27235fd5ac44af8e61693261550447a4c3e39da98ac + (since the OP_CODESEPARATOR is not executed, nothing is removed from the scriptCode) + hashOutputs: b258eaf08c39fbe9fbac97c15c7e7adeb8df142b0df6f83e017f349c2b6fe3d2 + preimage: 0100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000e9b542c5176808107ff1df906f46bb1f2583b16112b95ee5380665ba7fcfc00100000000270063ab68210392972e2eb617b2388771abe27235fd5ac44af8e61693261550447a4c3e39da98acffffff0000000000ffffffffb258eaf08c39fbe9fbac97c15c7e7adeb8df142b0df6f83e017f349c2b6fe3d20000000083000000 + sigHash: e9071e75e25b8a1e298a72f0d2e9f4f95a0f5cdf86a533cda597eb402ed13b3a + public key: 0392972e2eb617b2388771abe27235fd5ac44af8e61693261550447a4c3e39da98 + private key: f52b3484edd96598e02a9c89c4492e9c1e2031f471c49fd721fe68b3ce37780d + signature: 3045022100f6a10b8604e6dc910194b79ccfc93e1bc0ec7c03453caaa8987f7d6c3413566002206216229ede9b4d6ec2d325be245c5b508ff0339bf1794078e20bfe0babc7ffe683 + + outpoint: 80e68831516392fcd100d186b3c2c7b95c80b53c77e77c35ba03a66b429a2a1b00000000 + scriptCode: 2468210392972e2eb617b2388771abe27235fd5ac44af8e61693261550447a4c3e39da98ac + (everything up to the last executed OP_CODESEPARATOR, including that OP_CODESEPARATOR, are removed) + hashOutputs: 91ea93dd77f702b738ebdbf3048940a98310e869a7bb8fa2c6cb3312916947ca + preimage: 010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000080e68831516392fcd100d186b3c2c7b95c80b53c77e77c35ba03a66b429a2a1b000000002468210392972e2eb617b2388771abe27235fd5ac44af8e61693261550447a4c3e39da98acffffff0000000000ffffffff91ea93dd77f702b738ebdbf3048940a98310e869a7bb8fa2c6cb3312916947ca0000000083000000 + sigHash: cd72f1f1a433ee9df816857fad88d8ebd97e09a75cd481583eb841c330275e54 + public key: 0392972e2eb617b2388771abe27235fd5ac44af8e61693261550447a4c3e39da98 + private key: f52b3484edd96598e02a9c89c4492e9c1e2031f471c49fd721fe68b3ce37780d + signature: 30440220032521802a76ad7bf74d0e2c218b72cf0cbc867066e2e53db905ba37f130397e02207709e2188ed7f08f4c952d9d13986da504502b8c3be59617e043552f506c46ff83 + + The serialized signed transaction is: + 01000000000102e9b542c5176808107ff1df906f46bb1f2583b16112b95ee5380665ba7fcfc0010000000000ffffffff80e68831516392fcd100d186b3c2c7b95c80b53c77e77c35ba03a66b429a2a1b0000000000ffffffff0280969800000000001976a914de4b231626ef508c9a74a8517e6783c0546d6b2888ac80969800000000001976a9146648a8cd4531e1ec47f35916de8e259237294d1e88ac02483045022100f6a10b8604e6dc910194b79ccfc93e1bc0ec7c03453caaa8987f7d6c3413566002206216229ede9b4d6ec2d325be245c5b508ff0339bf1794078e20bfe0babc7ffe683270063ab68210392972e2eb617b2388771abe27235fd5ac44af8e61693261550447a4c3e39da98ac024730440220032521802a76ad7bf74d0e2c218b72cf0cbc867066e2e53db905ba37f130397e02207709e2188ed7f08f4c952d9d13986da504502b8c3be59617e043552f506c46ff83275163ab68210392972e2eb617b2388771abe27235fd5ac44af8e61693261550447a4c3e39da98ac00000000 + nVersion: 01000000 + marker: 00 + flag: 01 + txin: 02 e9b542c5176808107ff1df906f46bb1f2583b16112b95ee5380665ba7fcfc001 00000000 00 ffffffff + 80e68831516392fcd100d186b3c2c7b95c80b53c77e77c35ba03a66b429a2a1b 00000000 00 ffffffff + txout: 02 8096980000000000 1976a914de4b231626ef508c9a74a8517e6783c0546d6b2888ac + 8096980000000000 1976a9146648a8cd4531e1ec47f35916de8e259237294d1e88ac + witness 02 483045022100f6a10b8604e6dc910194b79ccfc93e1bc0ec7c03453caaa8987f7d6c3413566002206216229ede9b4d6ec2d325be245c5b508ff0339bf1794078e20bfe0babc7ffe683 270063ab68210392972e2eb617b2388771abe27235fd5ac44af8e61693261550447a4c3e39da98ac + 02 4730440220032521802a76ad7bf74d0e2c218b72cf0cbc867066e2e53db905ba37f130397e02207709e2188ed7f08f4c952d9d13986da504502b8c3be59617e043552f506c46ff83 275163ab68210392972e2eb617b2388771abe27235fd5ac44af8e61693261550447a4c3e39da98ac + nLockTime: 00000000 + + Since SINGLE|ANYONECANPAY does not commit to the input index, the signatures are still valid when the the input-output pairs are swapped: + 0100000000010280e68831516392fcd100d186b3c2c7b95c80b53c77e77c35ba03a66b429a2a1b0000000000ffffffffe9b542c5176808107ff1df906f46bb1f2583b16112b95ee5380665ba7fcfc0010000000000ffffffff0280969800000000001976a9146648a8cd4531e1ec47f35916de8e259237294d1e88ac80969800000000001976a914de4b231626ef508c9a74a8517e6783c0546d6b2888ac024730440220032521802a76ad7bf74d0e2c218b72cf0cbc867066e2e53db905ba37f130397e02207709e2188ed7f08f4c952d9d13986da504502b8c3be59617e043552f506c46ff83275163ab68210392972e2eb617b2388771abe27235fd5ac44af8e61693261550447a4c3e39da98ac02483045022100f6a10b8604e6dc910194b79ccfc93e1bc0ec7c03453caaa8987f7d6c3413566002206216229ede9b4d6ec2d325be245c5b508ff0339bf1794078e20bfe0babc7ffe683270063ab68210392972e2eb617b2388771abe27235fd5ac44af8e61693261550447a4c3e39da98ac00000000 + nVersion: 01000000 + marker: 00 + flag: 01 + txin: 02 80e68831516392fcd100d186b3c2c7b95c80b53c77e77c35ba03a66b429a2a1b 00000000 00 ffffffff + e9b542c5176808107ff1df906f46bb1f2583b16112b95ee5380665ba7fcfc001 00000000 00 ffffffff + txout: 02 8096980000000000 1976a9146648a8cd4531e1ec47f35916de8e259237294d1e88ac + 8096980000000000 1976a914de4b231626ef508c9a74a8517e6783c0546d6b2888ac + witness 02 4730440220032521802a76ad7bf74d0e2c218b72cf0cbc867066e2e53db905ba37f130397e02207709e2188ed7f08f4c952d9d13986da504502b8c3be59617e043552f506c46ff83 275163ab68210392972e2eb617b2388771abe27235fd5ac44af8e61693261550447a4c3e39da98ac + 02 483045022100f6a10b8604e6dc910194b79ccfc93e1bc0ec7c03453caaa8987f7d6c3413566002206216229ede9b4d6ec2d325be245c5b508ff0339bf1794078e20bfe0babc7ffe683 270063ab68210392972e2eb617b2388771abe27235fd5ac44af8e61693261550447a4c3e39da98ac + nLockTime: 00000000 + +=== P2SH-P2WSH === + +This example is a P2SH-P2WSH 6-of-6 multisig witness program signed with 6 different <code>SIGHASH</code> types. + + + + The following is an unsigned transaction: 010000000136641869ca081e70f394c6948e8af409e18b619df2ed74aa106c1ca29787b96e0100000000ffffffff0200e9a435000000001976a914389ffce9cd9ae88dcc0631e88a821ffdbe9bfe2688acc0832f05000000001976a9147480a33f950689af511e6e84c138dbbd3c3ee41588ac00000000 + + nVersion: 01000000 + txin: 01 36641869ca081e70f394c6948e8af409e18b619df2ed74aa106c1ca29787b96e 01000000 00 ffffffff + txout: 02 00e9a43500000000 1976a914389ffce9cd9ae88dcc0631e88a821ffdbe9bfe2688ac + c0832f0500000000 1976a9147480a33f950689af511e6e84c138dbbd3c3ee41588ac + nLockTime: 00000000 + + The input comes from a P2SH-P2WSH 6-of-6 multisig witness program: + scriptPubKey : a9149993a429037b5d912407a71c252019287b8d27a587, value: 9.87654321 + redeemScript : 0020a16b5755f7f6f96dbd65f5f0d6ab9418b89af4b1f14a1bb8a09062c35f0dcb54 + witnessScript: 56210307b8ae49ac90a048e9b53357a2354b3334e9c8bee813ecb98e99a7e07e8c3ba32103b28f0c28bfab54554ae8c658ac5c3e0ce6e79ad336331f78c428dd43eea8449b21034b8113d703413d57761b8b9781957b8c0ac1dfe69f492580ca4195f50376ba4a21033400f6afecb833092a9a21cfdf1ed1376e58c5d1f47de74683123987e967a8f42103a6d48b1131e94ba04d9737d61acdaa1322008af9602b3b14862c07a1789aac162102d8b661b0b3302ee2f162b09e07a55ad5dfbe673a9f01d9f0c19617681024306b56ae + + hashPrevouts: + dSHA256(36641869ca081e70f394c6948e8af409e18b619df2ed74aa106c1ca29787b96e01000000) + = 74afdc312af5183c4198a40ca3c1a275b485496dd3929bca388c4b5e31f7aaa0 + + hashSequence: + dSHA256(ffffffff) + = 3bb13029ce7b1f559ef5e747fcac439f1455a2ec7c5f09b72290795e70665044 + + hashOutputs for ALL: + dSHA256(00e9a435000000001976a914389ffce9cd9ae88dcc0631e88a821ffdbe9bfe2688acc0832f05000000001976a9147480a33f950689af511e6e84c138dbbd3c3ee41588ac) + = bc4d309071414bed932f98832b27b4d76dad7e6c1346f487a8fdbb8eb90307cc + + hashOutputs for SINGLE: + dSHA256(00e9a435000000001976a914389ffce9cd9ae88dcc0631e88a821ffdbe9bfe2688ac) + = 9efe0c13a6b16c14a41b04ebe6a63f419bdacb2f8705b494a43063ca3cd4f708 + + hash preimage for ALL: 0100000074afdc312af5183c4198a40ca3c1a275b485496dd3929bca388c4b5e31f7aaa03bb13029ce7b1f559ef5e747fcac439f1455a2ec7c5f09b72290795e7066504436641869ca081e70f394c6948e8af409e18b619df2ed74aa106c1ca29787b96e01000000cf56210307b8ae49ac90a048e9b53357a2354b3334e9c8bee813ecb98e99a7e07e8c3ba32103b28f0c28bfab54554ae8c658ac5c3e0ce6e79ad336331f78c428dd43eea8449b21034b8113d703413d57761b8b9781957b8c0ac1dfe69f492580ca4195f50376ba4a21033400f6afecb833092a9a21cfdf1ed1376e58c5d1f47de74683123987e967a8f42103a6d48b1131e94ba04d9737d61acdaa1322008af9602b3b14862c07a1789aac162102d8b661b0b3302ee2f162b09e07a55ad5dfbe673a9f01d9f0c19617681024306b56aeb168de3a00000000ffffffffbc4d309071414bed932f98832b27b4d76dad7e6c1346f487a8fdbb8eb90307cc0000000001000000 + nVersion: 01000000 + hashPrevouts: 74afdc312af5183c4198a40ca3c1a275b485496dd3929bca388c4b5e31f7aaa0 + hashSequence: 3bb13029ce7b1f559ef5e747fcac439f1455a2ec7c5f09b72290795e70665044 + outpoint: 36641869ca081e70f394c6948e8af409e18b619df2ed74aa106c1ca29787b96e01000000 + scriptCode: cf56210307b8ae49ac90a048e9b53357a2354b3334e9c8bee813ecb98e99a7e07e8c3ba32103b28f0c28bfab54554ae8c658ac5c3e0ce6e79ad336331f78c428dd43eea8449b21034b8113d703413d57761b8b9781957b8c0ac1dfe69f492580ca4195f50376ba4a21033400f6afecb833092a9a21cfdf1ed1376e58c5d1f47de74683123987e967a8f42103a6d48b1131e94ba04d9737d61acdaa1322008af9602b3b14862c07a1789aac162102d8b661b0b3302ee2f162b09e07a55ad5dfbe673a9f01d9f0c19617681024306b56ae + amount: b168de3a00000000 + nSequence: ffffffff + hashOutputs: bc4d309071414bed932f98832b27b4d76dad7e6c1346f487a8fdbb8eb90307cc + nLockTime: 00000000 + nHashType: 01000000 + sigHash: 185c0be5263dce5b4bb50a047973c1b6272bfbd0103a89444597dc40b248ee7c + public key: 0307b8ae49ac90a048e9b53357a2354b3334e9c8bee813ecb98e99a7e07e8c3ba3 + private key: 730fff80e1413068a05b57d6a58261f07551163369787f349438ea38ca80fac6 + signature: 304402206ac44d672dac41f9b00e28f4df20c52eeb087207e8d758d76d92c6fab3b73e2b0220367750dbbe19290069cba53d096f44530e4f98acaa594810388cf7409a1870ce01 + + hash preimage for NONE: 0100000074afdc312af5183c4198a40ca3c1a275b485496dd3929bca388c4b5e31f7aaa0000000000000000000000000000000000000000000000000000000000000000036641869ca081e70f394c6948e8af409e18b619df2ed74aa106c1ca29787b96e01000000cf56210307b8ae49ac90a048e9b53357a2354b3334e9c8bee813ecb98e99a7e07e8c3ba32103b28f0c28bfab54554ae8c658ac5c3e0ce6e79ad336331f78c428dd43eea8449b21034b8113d703413d57761b8b9781957b8c0ac1dfe69f492580ca4195f50376ba4a21033400f6afecb833092a9a21cfdf1ed1376e58c5d1f47de74683123987e967a8f42103a6d48b1131e94ba04d9737d61acdaa1322008af9602b3b14862c07a1789aac162102d8b661b0b3302ee2f162b09e07a55ad5dfbe673a9f01d9f0c19617681024306b56aeb168de3a00000000ffffffff00000000000000000000000000000000000000000000000000000000000000000000000002000000 + nVersion: 01000000 + hashPrevouts: 74afdc312af5183c4198a40ca3c1a275b485496dd3929bca388c4b5e31f7aaa0 + hashSequence: 0000000000000000000000000000000000000000000000000000000000000000 + outpoint: 36641869ca081e70f394c6948e8af409e18b619df2ed74aa106c1ca29787b96e01000000 + scriptCode: cf56210307b8ae49ac90a048e9b53357a2354b3334e9c8bee813ecb98e99a7e07e8c3ba32103b28f0c28bfab54554ae8c658ac5c3e0ce6e79ad336331f78c428dd43eea8449b21034b8113d703413d57761b8b9781957b8c0ac1dfe69f492580ca4195f50376ba4a21033400f6afecb833092a9a21cfdf1ed1376e58c5d1f47de74683123987e967a8f42103a6d48b1131e94ba04d9737d61acdaa1322008af9602b3b14862c07a1789aac162102d8b661b0b3302ee2f162b09e07a55ad5dfbe673a9f01d9f0c19617681024306b56ae + amount: b168de3a00000000 + nSequence: ffffffff + hashOutputs: 0000000000000000000000000000000000000000000000000000000000000000 + nLockTime: 00000000 + nHashType: 02000000 + sigHash: e9733bc60ea13c95c6527066bb975a2ff29a925e80aa14c213f686cbae5d2f36 + public key: 03b28f0c28bfab54554ae8c658ac5c3e0ce6e79ad336331f78c428dd43eea8449b + private key: 11fa3d25a17cbc22b29c44a484ba552b5a53149d106d3d853e22fdd05a2d8bb3 + signature: 3044022068c7946a43232757cbdf9176f009a928e1cd9a1a8c212f15c1e11ac9f2925d9002205b75f937ff2f9f3c1246e547e54f62e027f64eefa2695578cc6432cdabce271502 + + hash preimage for SINGLE: 0100000074afdc312af5183c4198a40ca3c1a275b485496dd3929bca388c4b5e31f7aaa0000000000000000000000000000000000000000000000000000000000000000036641869ca081e70f394c6948e8af409e18b619df2ed74aa106c1ca29787b96e01000000cf56210307b8ae49ac90a048e9b53357a2354b3334e9c8bee813ecb98e99a7e07e8c3ba32103b28f0c28bfab54554ae8c658ac5c3e0ce6e79ad336331f78c428dd43eea8449b21034b8113d703413d57761b8b9781957b8c0ac1dfe69f492580ca4195f50376ba4a21033400f6afecb833092a9a21cfdf1ed1376e58c5d1f47de74683123987e967a8f42103a6d48b1131e94ba04d9737d61acdaa1322008af9602b3b14862c07a1789aac162102d8b661b0b3302ee2f162b09e07a55ad5dfbe673a9f01d9f0c19617681024306b56aeb168de3a00000000ffffffff9efe0c13a6b16c14a41b04ebe6a63f419bdacb2f8705b494a43063ca3cd4f7080000000003000000 + nVersion: 01000000 + hashPrevouts: 74afdc312af5183c4198a40ca3c1a275b485496dd3929bca388c4b5e31f7aaa0 + hashSequence: 0000000000000000000000000000000000000000000000000000000000000000 + outpoint: 36641869ca081e70f394c6948e8af409e18b619df2ed74aa106c1ca29787b96e01000000 + scriptCode: cf56210307b8ae49ac90a048e9b53357a2354b3334e9c8bee813ecb98e99a7e07e8c3ba32103b28f0c28bfab54554ae8c658ac5c3e0ce6e79ad336331f78c428dd43eea8449b21034b8113d703413d57761b8b9781957b8c0ac1dfe69f492580ca4195f50376ba4a21033400f6afecb833092a9a21cfdf1ed1376e58c5d1f47de74683123987e967a8f42103a6d48b1131e94ba04d9737d61acdaa1322008af9602b3b14862c07a1789aac162102d8b661b0b3302ee2f162b09e07a55ad5dfbe673a9f01d9f0c19617681024306b56ae + amount: b168de3a00000000 + nSequence: ffffffff + hashOutputs: 9efe0c13a6b16c14a41b04ebe6a63f419bdacb2f8705b494a43063ca3cd4f708 + nLockTime: 00000000 + nHashType: 03000000 + sigHash: 1e1f1c303dc025bd664acb72e583e933fae4cff9148bf78c157d1e8f78530aea + public key: 034b8113d703413d57761b8b9781957b8c0ac1dfe69f492580ca4195f50376ba4a + private key: 77bf4141a87d55bdd7f3cd0bdccf6e9e642935fec45f2f30047be7b799120661 + signature: 3044022059ebf56d98010a932cf8ecfec54c48e6139ed6adb0728c09cbe1e4fa0915302e022007cd986c8fa870ff5d2b3a89139c9fe7e499259875357e20fcbb15571c76795403 + + hash preimage for ALL|ANYONECANPAY: 010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000036641869ca081e70f394c6948e8af409e18b619df2ed74aa106c1ca29787b96e01000000cf56210307b8ae49ac90a048e9b53357a2354b3334e9c8bee813ecb98e99a7e07e8c3ba32103b28f0c28bfab54554ae8c658ac5c3e0ce6e79ad336331f78c428dd43eea8449b21034b8113d703413d57761b8b9781957b8c0ac1dfe69f492580ca4195f50376ba4a21033400f6afecb833092a9a21cfdf1ed1376e58c5d1f47de74683123987e967a8f42103a6d48b1131e94ba04d9737d61acdaa1322008af9602b3b14862c07a1789aac162102d8b661b0b3302ee2f162b09e07a55ad5dfbe673a9f01d9f0c19617681024306b56aeb168de3a00000000ffffffffbc4d309071414bed932f98832b27b4d76dad7e6c1346f487a8fdbb8eb90307cc0000000081000000 + nVersion: 01000000 + hashPrevouts: 0000000000000000000000000000000000000000000000000000000000000000 + hashSequence: 0000000000000000000000000000000000000000000000000000000000000000 + outpoint: 36641869ca081e70f394c6948e8af409e18b619df2ed74aa106c1ca29787b96e01000000 + scriptCode: cf56210307b8ae49ac90a048e9b53357a2354b3334e9c8bee813ecb98e99a7e07e8c3ba32103b28f0c28bfab54554ae8c658ac5c3e0ce6e79ad336331f78c428dd43eea8449b21034b8113d703413d57761b8b9781957b8c0ac1dfe69f492580ca4195f50376ba4a21033400f6afecb833092a9a21cfdf1ed1376e58c5d1f47de74683123987e967a8f42103a6d48b1131e94ba04d9737d61acdaa1322008af9602b3b14862c07a1789aac162102d8b661b0b3302ee2f162b09e07a55ad5dfbe673a9f01d9f0c19617681024306b56ae + amount: b168de3a00000000 + nSequence: ffffffff + hashOutputs: bc4d309071414bed932f98832b27b4d76dad7e6c1346f487a8fdbb8eb90307cc + nLockTime: 00000000 + nHashType: 81000000 + sigHash: 2a67f03e63a6a422125878b40b82da593be8d4efaafe88ee528af6e5a9955c6e + public key: 033400f6afecb833092a9a21cfdf1ed1376e58c5d1f47de74683123987e967a8f4 + private key: 14af36970f5025ea3e8b5542c0f8ebe7763e674838d08808896b63c3351ffe49 + signature: 3045022100fbefd94bd0a488d50b79102b5dad4ab6ced30c4069f1eaa69a4b5a763414067e02203156c6a5c9cf88f91265f5a942e96213afae16d83321c8b31bb342142a14d16381 + + hash preimage for NONE|ANYONECANPAY: 010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000036641869ca081e70f394c6948e8af409e18b619df2ed74aa106c1ca29787b96e01000000cf56210307b8ae49ac90a048e9b53357a2354b3334e9c8bee813ecb98e99a7e07e8c3ba32103b28f0c28bfab54554ae8c658ac5c3e0ce6e79ad336331f78c428dd43eea8449b21034b8113d703413d57761b8b9781957b8c0ac1dfe69f492580ca4195f50376ba4a21033400f6afecb833092a9a21cfdf1ed1376e58c5d1f47de74683123987e967a8f42103a6d48b1131e94ba04d9737d61acdaa1322008af9602b3b14862c07a1789aac162102d8b661b0b3302ee2f162b09e07a55ad5dfbe673a9f01d9f0c19617681024306b56aeb168de3a00000000ffffffff00000000000000000000000000000000000000000000000000000000000000000000000082000000 + nVersion: 01000000 + hashPrevouts: 0000000000000000000000000000000000000000000000000000000000000000 + hashSequence: 0000000000000000000000000000000000000000000000000000000000000000 + outpoint: 36641869ca081e70f394c6948e8af409e18b619df2ed74aa106c1ca29787b96e01000000 + scriptCode: cf56210307b8ae49ac90a048e9b53357a2354b3334e9c8bee813ecb98e99a7e07e8c3ba32103b28f0c28bfab54554ae8c658ac5c3e0ce6e79ad336331f78c428dd43eea8449b21034b8113d703413d57761b8b9781957b8c0ac1dfe69f492580ca4195f50376ba4a21033400f6afecb833092a9a21cfdf1ed1376e58c5d1f47de74683123987e967a8f42103a6d48b1131e94ba04d9737d61acdaa1322008af9602b3b14862c07a1789aac162102d8b661b0b3302ee2f162b09e07a55ad5dfbe673a9f01d9f0c19617681024306b56ae + amount: b168de3a00000000 + nSequence: ffffffff + hashOutputs: 0000000000000000000000000000000000000000000000000000000000000000 + nLockTime: 00000000 + nHashType: 82000000 + sigHash: 781ba15f3779d5542ce8ecb5c18716733a5ee42a6f51488ec96154934e2c890a + public key: 03a6d48b1131e94ba04d9737d61acdaa1322008af9602b3b14862c07a1789aac16 + private key: fe9a95c19eef81dde2b95c1284ef39be497d128e2aa46916fb02d552485e0323 + signature: 3045022100a5263ea0553ba89221984bd7f0b13613db16e7a70c549a86de0cc0444141a407022005c360ef0ae5a5d4f9f2f87a56c1546cc8268cab08c73501d6b3be2e1e1a8a0882 + + hash preimage for SINGLE|ANYONECANPAY: 010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000036641869ca081e70f394c6948e8af409e18b619df2ed74aa106c1ca29787b96e01000000cf56210307b8ae49ac90a048e9b53357a2354b3334e9c8bee813ecb98e99a7e07e8c3ba32103b28f0c28bfab54554ae8c658ac5c3e0ce6e79ad336331f78c428dd43eea8449b21034b8113d703413d57761b8b9781957b8c0ac1dfe69f492580ca4195f50376ba4a21033400f6afecb833092a9a21cfdf1ed1376e58c5d1f47de74683123987e967a8f42103a6d48b1131e94ba04d9737d61acdaa1322008af9602b3b14862c07a1789aac162102d8b661b0b3302ee2f162b09e07a55ad5dfbe673a9f01d9f0c19617681024306b56aeb168de3a00000000ffffffff9efe0c13a6b16c14a41b04ebe6a63f419bdacb2f8705b494a43063ca3cd4f7080000000083000000 + nVersion: 01000000 + hashPrevouts: 0000000000000000000000000000000000000000000000000000000000000000 + hashSequence: 0000000000000000000000000000000000000000000000000000000000000000 + outpoint: 36641869ca081e70f394c6948e8af409e18b619df2ed74aa106c1ca29787b96e01000000 + scriptCode: cf56210307b8ae49ac90a048e9b53357a2354b3334e9c8bee813ecb98e99a7e07e8c3ba32103b28f0c28bfab54554ae8c658ac5c3e0ce6e79ad336331f78c428dd43eea8449b21034b8113d703413d57761b8b9781957b8c0ac1dfe69f492580ca4195f50376ba4a21033400f6afecb833092a9a21cfdf1ed1376e58c5d1f47de74683123987e967a8f42103a6d48b1131e94ba04d9737d61acdaa1322008af9602b3b14862c07a1789aac162102d8b661b0b3302ee2f162b09e07a55ad5dfbe673a9f01d9f0c19617681024306b56ae + amount: b168de3a00000000 + nSequence: ffffffff + hashOutputs: 9efe0c13a6b16c14a41b04ebe6a63f419bdacb2f8705b494a43063ca3cd4f708 + nLockTime: 00000000 + nHashType: 83000000 + sigHash: 511e8e52ed574121fc1b654970395502128263f62662e076dc6baf05c2e6a99b + public key: 02d8b661b0b3302ee2f162b09e07a55ad5dfbe673a9f01d9f0c19617681024306b + private key: 428a7aee9f0c2af0cd19af3cf1c78149951ea528726989b2e83e4778d2c3f890 + signature: 30440220525406a1482936d5a21888260dc165497a90a15669636d8edca6b9fe490d309c022032af0c646a34a44d1f4576bf6a4a74b67940f8faa84c7df9abe12a01a11e2b4783 + + The serialized signed transaction is: 0100000000010136641869ca081e70f394c6948e8af409e18b619df2ed74aa106c1ca29787b96e0100000023220020a16b5755f7f6f96dbd65f5f0d6ab9418b89af4b1f14a1bb8a09062c35f0dcb54ffffffff0200e9a435000000001976a914389ffce9cd9ae88dcc0631e88a821ffdbe9bfe2688acc0832f05000000001976a9147480a33f950689af511e6e84c138dbbd3c3ee41588ac080047304402206ac44d672dac41f9b00e28f4df20c52eeb087207e8d758d76d92c6fab3b73e2b0220367750dbbe19290069cba53d096f44530e4f98acaa594810388cf7409a1870ce01473044022068c7946a43232757cbdf9176f009a928e1cd9a1a8c212f15c1e11ac9f2925d9002205b75f937ff2f9f3c1246e547e54f62e027f64eefa2695578cc6432cdabce271502473044022059ebf56d98010a932cf8ecfec54c48e6139ed6adb0728c09cbe1e4fa0915302e022007cd986c8fa870ff5d2b3a89139c9fe7e499259875357e20fcbb15571c76795403483045022100fbefd94bd0a488d50b79102b5dad4ab6ced30c4069f1eaa69a4b5a763414067e02203156c6a5c9cf88f91265f5a942e96213afae16d83321c8b31bb342142a14d16381483045022100a5263ea0553ba89221984bd7f0b13613db16e7a70c549a86de0cc0444141a407022005c360ef0ae5a5d4f9f2f87a56c1546cc8268cab08c73501d6b3be2e1e1a8a08824730440220525406a1482936d5a21888260dc165497a90a15669636d8edca6b9fe490d309c022032af0c646a34a44d1f4576bf6a4a74b67940f8faa84c7df9abe12a01a11e2b4783cf56210307b8ae49ac90a048e9b53357a2354b3334e9c8bee813ecb98e99a7e07e8c3ba32103b28f0c28bfab54554ae8c658ac5c3e0ce6e79ad336331f78c428dd43eea8449b21034b8113d703413d57761b8b9781957b8c0ac1dfe69f492580ca4195f50376ba4a21033400f6afecb833092a9a21cfdf1ed1376e58c5d1f47de74683123987e967a8f42103a6d48b1131e94ba04d9737d61acdaa1322008af9602b3b14862c07a1789aac162102d8b661b0b3302ee2f162b09e07a55ad5dfbe673a9f01d9f0c19617681024306b56ae00000000 + + The new serialization format is described in BIP144 <ref>[[bip-0144.mediawiki|BIP144: Segregated Witness (Peer Services)]]</ref> + == Deployment == This proposal is deployed with Segregated Witness softfork (BIP 141) == Backward compatibility == -As a soft fork, older software will continue to operate without modification. Non-upgraded nodes, however, will not see nor validate the witness data and will consider all witness programs, inculding the redefined sigops, as anyone-can-spend scripts. +As a soft fork, older software will continue to operate without modification. Non-upgraded nodes, however, will not see nor validate the witness data and will consider all witness programs, including the redefined sigops, as anyone-can-spend scripts. == Reference Implementation == -https://github.com/sipa/bitcoin/commits/segwit +https://github.com/bitcoin/bitcoin/pull/7910 == References == diff --git a/bip-0144.mediawiki b/bip-0144.mediawiki index 2da222c..f2c8857 100644 --- a/bip-0144.mediawiki +++ b/bip-0144.mediawiki @@ -86,9 +86,10 @@ Currently, the only witness objects type supported are script witnesses which co * '''Rationale for the 0x01 flag byte in between''': this will allow us to easily add more extra non-committed data to transactions (like txouts being spent, ...). It can be interpreted as a bitvector. === Handshake === -A new message 'havewitness' is sent after receiving 'verack' to -indicate that a node can provide witness if requested (similar to -'sendheaders') (Note: it might be better to signal this with a services bit in the version message) +A node will signal that it can provide witnesses using the following service bit + + NODE_WITNESS = (1 << 3) + === Hashes === Transaction hashes used in the transaction merkle tree and txin outpoints are always computed using the old non-witness diff --git a/bip-0144/witnesstx.png b/bip-0144/witnesstx.png Binary files differindex 5fd8afc..cb02a63 100644 --- a/bip-0144/witnesstx.png +++ b/bip-0144/witnesstx.png diff --git a/bip-0145.mediawiki b/bip-0145.mediawiki index b90725e..b04c9e6 100644 --- a/bip-0145.mediawiki +++ b/bip-0145.mediawiki @@ -15,20 +15,20 @@ This BIP describes modifications to the getblocktemplate JSON-RPC call ([[bip-00 ===Block Template=== -The template Object is revised to include these keys: +The template Object is revised to include a new key: {| class="wikitable" !colspan=4| template |- ! Key !! Required !! Type !! Description |- -| costlimit || {{No}} || Number || total cost allowed in blocks -|- -| sigoplimit || {{No}} || Number || total sigop cost allowed in blocks divided by 4 -|- -| version || {{Yes}} || Number || block version; clients MUST understand the implications of the version they use (eg, comply with [[bip-0141.mediawiki|BIP 141]] for version 5) +| costlimit || No || Number || total cost allowed in blocks |} +The '!' rule prefix MUST be enabled on the "segwit" rule for templates including transactions with witness data. +In particular, note that even if the client's "rules" list lacks "segwit", server MAY support old miners by producing a witness-free template and omitting the '!' rule prefix for "segwit" in the template's "rules" list. +If the GBT server does not support producing witness-free templates after its activation, it must also use the '!' rule prefix in the "vbavailable" list prior to activation. + ====Transactions Object Format==== The Objects listed in the response's "transactions" key is revised to include these keys: @@ -45,7 +45,11 @@ The Objects listed in the response's "transactions" key is revised to include th | hash || String || reversed hash of complete transaction (with witness data included) encoded in hexadecimal |} -Transactions with witness data may only be included if the template's block version is at least 5. +Transactions with witness data may only be included if the template's "rules" list (see [[bip-0009.mediawiki#getblocktemplate_changes|BIP 9]]) includes "segwit". + +===Sigops=== + +For templates with "segwit" enabled as a rule, the "sigoplimit" and "sigops" keys must use the new values as calculated in [[bip-0141.mediawiki#Sigops|BIP 141]]. ===Block Assembly with Witness Transactions=== @@ -87,6 +91,7 @@ Why shouldn't the server simply add the commitment upfront in the "coinbasetxn", * [https://github.com/bitcoin/bitcoin/pull/7404/files Bitcoin Core] ==See Also== +* [[bip-0009.mediawiki|BIP 9: Version bits with timeout and delay]] * [[bip-0022.mediawiki|BIP 22: getblocktemplate - Fundamentals]] * [[bip-0023.mediawiki|BIP 23: getblocktemplate - Pooled Mining]] * [[bip-0141.mediawiki|BIP 141: Segregated Witness (Consensus layer)]] diff --git a/bip-0151.mediawiki b/bip-0151.mediawiki new file mode 100644 index 0000000..cc79712 --- /dev/null +++ b/bip-0151.mediawiki @@ -0,0 +1,184 @@ +<pre> + BIP: 151 + Title: Peer-to-Peer Communication Encryption + Author: Jonas Schnelli <dev@jonasschnelli.ch> + Status: Draft + Type: Standards Track + Created: 2016-03-23 +</pre> + +== Abstract == + +This BIP describes an alternative way that a peer can encrypt their communication between a selective subset of remote peers. + +== Motivation == + + +The Bitcoin network does not encrypt communication between peers today. This opens up security issues (eg: traffic manipulation by others) and allows for mass surveillance / analysis of bitcoin users. Mostly this is negligible because of the nature of Bitcoins trust model, however for SPV nodes this can have significant privacy impacts [1] and could reduce the censorship-resistance of a peer. + +Encrypting peer traffic will make analysis and specific user targeting much more difficult than it currently is. Today it's trivial for a network provider or any other men-in-the-middle to identify a Bitcoin user and its controlled addresses/keys (and link with his Google profile, etc.). Just created and broadcasted transactions will reveal the amount and the payee to the network provider. + +This BIP also describes a way that data manipulation (blocking commands by a intercepting TCP/IP node) would be identifiable by the communicating peers. + +Analyzing the type of p2p communication would still be possible because of the characteristics (size, sending-interval, etc.) of the encrypted messages. + +Encrypting traffic between peers is already possible with VPN, tor, stunnel, curveCP or any other encryption mechanism on a deeper OSI level, however, most mechanism are not practical for SPV or other DHCP/NAT environment and will require significant knowhow in how to setup such a secure channel. + +== Specification == + +A peer that supports encryption must accept encryption requests from all peers. + +A independent ECDH negotiation for both communication directions is required and therefore a bidirectional communication will use two symmetric cipher keys (one per direction). + +Both peers must only send encrypted messages after a successful ECDH negotiation in ''both directions''. + +Encryption initialization must happen before sending any other messages to the responding peer (<code>encinit</code> message after a <code>version</code> message must be ignored). + +=== Symmetric Encryption Cipher Keys === + +The symmetric encryption cipher keys will be calculated with ECDH/HKDF by sharing the pubkeys of a ephemeral key. Once the ECDH secret is calculated on each side, the symmetric encryption cipher keys must be derived with HKDF [2] after the following specification: + +1. HKDF extraction +<code>PRK = HKDF_EXTRACT(hash=SHA256, salt="bitcoinechd", ikm=ecdh_secret|cipher-type)</code>. + +2. Derive Key1 +<code>K_1 = HKDF_EXPAND(prk=PRK, hash=SHA256, info="BitcoinK1", L=32)</code> + +3. Derive Key2 +<code>K_2 = HKDF_EXPAND(prk=PRK, hash=SHA256, info="BitcoinK2", L=32)</code> + +It is important to include the cipher-type into the symmetric cipher key derivation to avoid weak-cipher-attacks. + +=== Session ID === + +Both sides must also calculate the 256bit session-id using <code>SID = HKDF_EXPAND(prk=PRK, hash=SHA256, info="BitcoinSessionID", L=32)</code>. The session-id can be used for linking the encryption-session to an identity check. + +=== The <code>encinit</code> message type === + +To request encrypted communication, the requesting peer generates an EC ephemeral-session-keypair and sends an <code>encinit</code> message to the responding peer and waits for a <code>encack</code> message. The responding node must do the same <code>encinit</code>/<code>encack</code> interaction for the opposite communication direction. + +{|class="wikitable" +! Field Size !! Description !! Data type !! Comments +|- +| 33bytes || ephemeral-pubkey || comp.-pubkey || The session pubkey from the requesting peer +|- +| 1bytes || symmetric key cipher type || int8 || symmetric key cipher type to use +|} + +Possible symmetric key ciphers types +{|class="wikitable" +! Number !! symmetric key ciphers type +|- +| 0 || chacha20-poly1305@openssh.com +|} + +=== ChaCha20-Poly1305 Cipher Suite === + +ChaCha20 is a stream cipher designed by Daniel Bernstein [3]. It operates by permuting 128 fixed bits, 128 or 256 bits of key, +a 64 bit nonce and a 64 bit counter into 64 bytes of output. This output is used as a keystream, with any unused bytes simply discarded. + +Poly1305, also by Daniel Bernstein [4], is a one-time Carter-Wegman MAC that computes a 128 bit integrity tag given a message and a single-use +256 bit secret key. + +The chacha20-poly1305@openssh.com specified and defined by openssh [5] combines these two primitives into an authenticated encryption mode. The construction used is based on that proposed for TLS by Adam Langley [6], but differs in the layout of data passed to the MAC and in the addition of encyption of the packet lengths. + +<code>K_1</code> must be used to only encrypt the payload size of the encrypted message to avoid leaking information by revealing the message size. + +<code>K_2</code> must be used in conjunction with poly1305 to build an AEAD. + +Optimized implementations of ChaCha20-Poly1305 are very fast in general, therefore it is very likely that encrypted messages require less CPU cycles per bytes then the current unencrypted p2p message format. A quick analysis by Pieter Wuille of the current ''standard implementations'' has shown that SHA256 requires more CPU cycles per byte then ChaCha20 & Poly1304. + +=== The <code>encack</code> message type === + +The responding peer accepts the encryption request by sending a <code>encack</code> message. + +{|class="wikitable" +! Field Size !! Description !! Data type !! Comments +|- +| 33bytes || ephemeral-pubkey || comp.-pubkey || The session pubkey from the responding peer +|} + +At this point, the shared secret key for the symmetric key cipher must be calculated by using ECDH (own privkey x remote pub key). +Private keys will never be transmitted. The shared secret can only be calculated if an attacker knows at least one private key and the remote peer's public key. + +* '''The <code>encinit</code>/<code>encack</code> interaction must be done from both sides.''' +* Each communication direction uses its own secret key for the symmetric cipher. +* The second <code>encinit</code> request (from the responding peer) must use the same symmetric cipher type. +* All unencrypted messages before the second <code>encack</code> response (from the responding peer) must be ignored. +* After a successful <code>encinit</code>/<code>encack</code> interaction, the "encrypted messages structure" must be used. Non-encrypted messages from the requesting peer must lead to a connection termination. + +After a successful <code>encinit</code>/<code>encack</code> interaction from both sides, the messages format must use the "encrypted messages structure". Non-encrypted messages from the requesting peer must lead to a connection termination (can be detected by the 4 byte network magic in the unencrypted message structure). + +=== Encrypted Messages Structure === + +{|class="wikitable" +! Field Size !! Description !! Data type !! Comments +|- +| 4 || length || uint32_t || Length of ciphertext payload in number of bytes +|- +| ? || ciphertext payload || ? || One or many ciphertext command & message data +|- +| 16 || MAC tag || ? || 128bit MAC-tag +|} + +Encrypted messages do not have the 4byte network magic. + +The maximum message length needs to be chosen carefully. The 4 byte length field can lead to a required message buffer of 4 GiB. +Processing the message before the authentication succeeds must not be done. + +The 4byte sha256 checksum is no longer required because the AEAD. + +Both peers need to track the message number (int64) of sent messages to the remote peer for building a symmetric cipher IV. Padding might be required (96bit IVs). + +The encrypted payload will result decrypted in one or many unencrypted messages: + +{|class="wikitable" +! Field Size !! Description !! Data type !! Comments +|- +| ? || command || varlen || ASCII string identifying the packet content, we are using varlen in the encrypted messages. +|- +| 4 || length || uint32_t || Length of plaintext payload +|- +| ? || payload || ? || The actual data +|} +If more data is present, another message must be deserialized. There is no explicit amount-of-messages integer. + + +=== Re-Keying === + +A responding peer can inform the requesting peer over a re-keying with a <code>encack</code> message containing 33byte of zeros to indicate that all encrypted message following after this <code>encack</code> message will be encrypted with ''the next symmetric cipher key''. + +The new symmetric cipher key will be calculated by <code>SHA256(SHA256(old_symetric_cipher_key))</code>. + +Re-Keying interval is a peer policy with a minimum timespan of 10 seconds. + +The Re-Keying must be done after every 1GB of data sent or received (recommended by RFC4253 SSH Transport). + +=== Risks === + +The encryption does not include an identity authentication scheme. This BIP does not cover a proposal to avoid MITM attacks during the encryption initialization. + +Identity authentication will be covered in another BIP and will presume communication encryption after this BIP. + +== Compatibility == + +This proposal is backward compatible. Non-supporting peers will ignore the <code>encinit</code> messages. + +== Reference implementation == + +== References == + +* [1] http://e-collection.library.ethz.ch/eserv/eth:48205/eth-48205-01.pdf +* [2] HKDF (RFC 5869) https://tools.ietf.org/html/rfc5869 +* [3] ChaCha20 http://cr.yp.to/chacha/chacha-20080128.pdf +* [4] Poly1305 http://cr.yp.to/mac/poly1305-20050329.pdf +* [5] https://github.com/openssh/openssh-portable/blob/05855bf2ce7d5cd0a6db18bc0b4214ed5ef7516d/PROTOCOL.chacha20poly1305 +* [6] "ChaCha20 and Poly1305 based Cipher Suites for TLS", Adam Langley http://tools.ietf.org/html/draft-agl-tls-chacha20poly1305-03 + +== Acknowledgements == +* Pieter Wuille and Gregory Maxwell for most of the ideas in this BIP. + +== Copyright == +This work is placed in the public domain. + + diff --git a/bip-0152.mediawiki b/bip-0152.mediawiki new file mode 100644 index 0000000..cfae0c0 --- /dev/null +++ b/bip-0152.mediawiki @@ -0,0 +1,208 @@ +<pre> + BIP: 152 + Title: Compact Block Relay + Author: Matt Corallo <bip152@bluematt.me> + Status: Draft + Type: Standards Track + Created: 2016-04-27 +</pre> + +==Abstract== + +Compact blocks on the wire as a way to save bandwidth for nodes on the P2P network. + +The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119. + +==Motivation== + +Historically, the Bitcoin P2P protocol has not been very bandwidth efficient for block relay. Every transaction in a block is included when relayed, even though a large number of the transactions in a given block are already available to nodes before the block is relayed. This causes moderate inbound bandwidth spikes for nodes when receiving blocks, but can cause very significant outbound bandwidth spikes for some nodes which receive a block before their peers. When such spikes occur, buffer bloat can make consumer-grade internet connections temporarily unusable, and can delay the relay of blocks to remote peers who may choose to wait instead of redundantly requesting the same block from other, less congested, peers. + +Thus, decreasing the bandwidth used during block relay is very useful for many individuals running nodes. + +While the goal of this work is explicitly not to reduce block transfer latency, it does, as a side effect reduce block transfer latencies in some rather significant ways. Additionally, this work forms a foundation for future work explicitly targeting low-latency block transfer. + +==Specification== + +===Intended Protocol Flow=== +<img src=bip-0152/protocol-flow.png></img> + +The protocol is intended to be used in two ways, depending on the peers and bandwidth available, as discussed [[#Implementation_Details|later]]. The "high-bandwidth" mode, which nodes may only enable for a few of their peers, is enabled by setting the first boolean to 1 in a <code>sendcmpct</code> message. In this mode, peers send new block announcements with the short transaction IDs already (via a <code>cmpctblock</code> message), possibly even before fully validating the block (as indicated by the grey box in the image above). In some cases no further round-trip is needed, and the receiver can reconstruct the block and process it as usual immediately. When some transactions were not available from local sources (ie mempool), a <code>getblocktxn</code>/<code>blocktxn</code> roundtrip is necessary, bringing the best-case latency to the same 1.5*RTT minimum time that nodes take today, though with significantly less bandwidth usage. + +The "low-bandwidth" mode is enabled by setting the first boolean to 0 in a <code>sendcmpct</code> message. In this mode, peers send new block announcements with the usual inv/headers announcements (as per BIP130, and after fully validating the block). The receiving peer may then request the block using a MSG_CMPCT_BLOCK <code>getdata</code> request, which will receive a response of the header and short transaction IDs. In some cases no further round-trip is needed, and the receiver can reconstruct the block and process it as usual, taking the same 1.5*RTT minimum time that nodes take today, though with significantly less bandwidth usage. When some transactions were not available from local sources (ie mempool), a <code>getblocktxn</code>/<code>blocktxn</code> roundtrip is necessary, bringing the latency to at least 2.5*RTT in this case, again with significantly less bandwidth usage than today. Because TCP often exhibits worse transfer latency for larger data sizes (as a multiple of RTT), total latency is expected to be reduced even when the full 2.5*RTT transfer mechanism is used. + +===New data structures=== +Several new data structures are added to the P2P network to relay compact blocks: PrefilledTransaction, HeaderAndShortIDs, BlockTransactionsRequest, and BlockTransactions. + +For the purposes of this section, CompactSize refers to the variable-length integer encoding used across the existing P2P protocol to encode array lengths, among other things, in 1, 3, 5 or 9 bytes. Only CompactSize encodings which are minimally-encoded (ie the shortest length possible) are used by this spec. Any other CompactSize encodings are left with undefined behavior. + +Several uses of CompactSize below are "differentially encoded". For these, instead of using raw indexes, the number encoded is the difference between the current index and the previous index, minus one. For example, a first index of 0 implies a real index of 0, a second index of 0 thereafter refers to a real index of 1, etc. + +====PrefilledTransaction==== +A PrefilledTransaction structure is used in HeaderAndShortIDs to provide a list of a few transactions explicitly. + +{| +|Field Name||Type||Size||Encoding||Purpose +|- +|index||CompactSize||1, 3 bytes||Compact Size, differentially encoded since the last PrefilledTransaction in a list||The index into the block at which this transaction is +|- +|tx||Transaction||variable||As encoded in "tx" messages||The transaction which is in the block at index index. +|} + +====HeaderAndShortIDs==== +A HeaderAndShortIDs structure is used to relay a block header, the short transactions IDs used for matching already-available transactions, and a select few transactions which we expect a peer may be missing. + +{| +|Field Name||Type||Size||Encoding||Purpose +|- +|header||Block header||80 bytes||First 80 bytes of the block as defined by the encoding used by "block" messages||The header of the block being provided +|- +|nonce||uint64_t||8 bytes||Little Endian||A nonce for use in short transaction ID calculations +|- +|shortids_length||CompactSize||1 or 3 bytes||As used to encode array lengths elsewhere||||The number of short transaction IDs in shortids (ie block tx count - prefilledtxn_length) +|- +|shortids||List of 6-byte integers||6*shortids_length bytes||Little Endian||The short transaction IDs calculated from the transactions which were not provided explicitly in prefilledtxn +|- +|prefilledtxn_length||CompactSize||1 or 3 bytes||As used to encode array lengths elsewhere||||The number of prefilled transactions in prefilledtxn (ie block tx count - shortids_length) +|- +|prefilledtxn||List of PrefilledTransactions||variable size*prefilledtxn_length||As defined by PrefilledTransaction definition, above||Used to provide the coinbase transaction and a select few which we expect a peer may be missing +|} + +====BlockTransactionsRequest==== +A BlockTransactionsRequest structure is used to list transaction indexes in a block being requested. + +{| +|Field Name||Type||Size||Encoding||Purpose +|- +|blockhash||Binary blob||32 bytes||The output from a double-SHA256 of the block header, as used elsewhere||The blockhash of the block which the transactions being requested are in +|- +|indexes_length||CompactSize||1 or 3 bytes||As used to encode array lengths elsewhere||||The number of transactions being requested +|- +|indexes||List of CompactSizes||1 or 3 bytes*indexes_length||Differentially encoded||The indexes of the transactions being requested in the block +|} + +====BlockTransactions==== +A BlockTransactions structure is used to provide some of the transactions in a block, as requested. + +{| +|Field Name||Type||Size||Encoding||Purpose +|- +|blockhash||Binary blob||32 bytes||The output from a double-SHA256 of the block header, as used elsewhere||The blockhash of the block which the transactions being provided are in +|- +|transactions_length||CompactSize||1 or 3 bytes||As used to encode array lengths elsewhere||||The number of transactions provided +|- +|transactions||List of Transactions||variable||As encoded in "tx" messages||The transactions provided +|} + +====Short transaction IDs==== +Short transaction IDs are used to represent a transaction without sending a full 256-bit hash. They are calculated by: +# single-SHA256 hashing the block header with the nonce appended (in little-endian) +# Running SipHash-2-4 with the input being the transaction ID and the keys (k0/k1) set to the first two little-endian 64-bit integers from the above hash, respectively. +# Dropping the 2 most significant bytes from the SipHash output to make it 6 bytes. + +===New messages=== +A new inv type (MSG_CMPCT_BLOCK == 4) and several new protocol messages are added: sendcmpct, cmpctblock, getblocktxn, and blocktxn. + +====sendcmpct==== +# The sendcmpct message is defined as a message containing a 1-byte integer followed by a 8-byte integer where pchCommand == "sendcmpct". +# The first integer SHALL be interpreted as a boolean (and MUST have a value of either 1 or 0) +# The second integer SHALL be interpreted as a little-endian version number. Nodes sending a sendcmpct message MUST currently set this value to 1. +# Upon receipt of a "sendcmpct" message with the first and second integers set to 1, the node SHOULD announce new blocks by sending a cmpctblock message. +# Upon receipt of a "sendcmpct" message with the first integer set to 0, the node SHOULD NOT announce new blocks by sending a cmpctblock message, but SHOULD announce new blocks by sending invs or headers, as defined by BIP130. +# Upon receipt of a "sendcmpct" message with the second integer set to something other than 1, nodes MUST treat the peer as if they had not received the message (as it indicates the peer will provide an unexpected encoding in cmpctblock, and/or other, messages). This allows future versions to send duplicate sendcmpct messages with different versions as a part of a version handshake for future versions. +# Nodes SHOULD check for a protocol version of >= 70014 before sending sendcmpct messages. +# Nodes MUST NOT send a request for a MSG_CMPCT_BLOCK object to a peer before having received a sendcmpct message from that peer. + +====MSG_CMPCT_BLOCK==== +# getdata messages may now contain requests for MSG_CMPCT_BLOCK objects. +# Upon receipt of a getdata containing a request for a MSG_CMPCT_BLOCK object with the hash of a block which was recently announced and is close to the tip of the best chain of the receiver and after having sent the requesting peer a sendcmpct message, nodes MUST respond with a cmpctblock message containing appropriate data representing the block being requested. +# Upon receipt of a getdata containing a request for a MSG_CMPCT_BLOCK object for which a cmpctblock message is not sent in response, a block message containing the requested block in non-compact form MUST be sent. +# MSG_CMPCT_BLOCK inv objects MUST NOT appear anywhere except for in getdata messages. + +====cmpctblock==== +# The cmpctblock message is defined as as a message containing a serialized HeaderAndShortIDs message and pchCommand == "cmpctblock". +# Upon receipt of a cmpctblock message after sending a sendcmpct message, nodes SHOULD calculate the short transaction ID for each unconfirmed transaction they have available (ie in their mempool) and compare each to each short transaction ID in the cmpctblock message. +# After finding already-available transactions, nodes which do not have all transactions available to reconstruct the full block SHOULD request the missing transactions using a getblocktxn message. +# A node MUST NOT send a cmpctblock message unless they are able to respond to a getblocktxn message which requests every transaction in the block. +# A node MUST NOT send a cmpctblock message without having validated that the header properly commits to each transaction in the block, and properly builds on top of the existing chain with a valid proof-of-work. A node MAY send a cmpctblock before validating that each transaction in the block validly spends existing UTXO set entries. + +====getblocktxn==== +# The getblocktxn message is defined as as a message containing a serialized BlockTransactionsRequest message and pchCommand == "getblocktxn". +# Upon receipt of a properly-formatted getblocktxnmessage, nodes which recently provided the sender of such a message a cmpctblock for the block hash identified in this message MUST respond with an appropriate blocktxn message. Such a blocktxn message MUST contain exactly and only each transaction which is present in the appropriate block at the index specified in the getblocktxn indexes list, in the order requested. + +====blocktxn==== +# The blocktxn message is defined as as a message containing a serialized BlockTransactions message and pchCommand == "blocktxn". +# Upon receipt of a properly-formatted requested blocktxn message, nodes SHOULD attempt to reconstruct the full block by: +## Taking the prefilledtxn transactions from the original cmpctblock and placing them in the marked positions. +## For each short transaction ID from the original cmpctblock, in order, find the corresponding transaction either from the blocktxn message or from other sources and place it in the first available position in the block. +# Once the block has been reconstructed, it shall be processed as normal, keeping in mind that short transaction IDs are expected to occasionally collide, and that nodes MUST NOT be penalized for such collisions, wherever they appear. + +===Implementation Notes=== +# For nodes which have sufficient inbound bandwidth, sending a sendcmpct message with the first integer set to 1 to up to 3 peers is RECOMMENDED. If possible, it is RECOMMENDED that those peers be selected based on their past performance in providing blocks quickly (eg the three peers which provided the highest number of the recent N blocks the quickest), allowing nodes to receive blocks which come from those peers in only 0.5*RTT. + +# Nodes MUST NOT send such sendcmpct messages to more than three peers, as it encourages wasting outbound bandwidth across the network. + +# All nodes SHOULD send a sendcmpct message to all appropriate peers. This will reduce their outbound bandwidth usage by allowing their peers to request compact blocks instead of full blocks. + +# Nodes with limited inbound bandwidth SHOULD request blocks using MSG_CMPCT_BLOCK/getblocktxn requests, when possible. While this increases worst-case message round-trips, it is expected to reduce overall transfer latency as TCP is more likely to exhibit poor throughput on low-bandwidth nodes. + +# Nodes sending cmpctblock messages SHOULD limit prefilledtxn to 10KB of transactions. When in doubt, nodes SHOULD only include the coinbase transaction in prefilledtxn. + +# Nodes MAY pick one nonce per block they wish to send, and only build a cmpctblock message once for all peers which they wish to send a given block to. Nodes SHOULD NOT use the same nonce across multiple different blocks. + +# Nodes MAY impose additional requirements on when they announce new blocks by sending cmpctblock messages. For example, nodes with limited outbound bandwidth MAY choose to announce new blocks using inv/header messages (as per BIP130) to conserve outbound bandwidth. + +# Note that the MSG_CMPCT_BLOCK section does not require that nodes respond to MSG_CMPCT_BLOCK getdata requests for blocks which they did not recently announce. This allows nodes to calculate cmpctblock messages at announce-time instead of at request-time. Blocks which are requested with a MSG_CMPCT_BLOCK getdata, but which are not responded to with a cmpctblock message MUST be responded to with a block message, allowing nodes to request all blocks using MSG_CMPCT_BLOCK getdatas and rely on their peer to pick an appropriate response. + +# While the current version sends transactions with the same encodings as is used in tx messages and elsewhere in the protocol, the version field in sendcmpct is intended to allow this to change in the future. For this reason, it is recommended that the code used to decode PrefilledTransaction and BlockTransactions messages be prepared to take a different transaction encoding, if and when the version field in sendcmpct changes in a future BIP. + +# Any undefined behavior in this spec may cause failure to transfer block to, peer disconnection by, or self-destruction by the receiving node. A node receiving non-minimally-encoded CompactSize encodings should make a best-effort to eat the sender's cat. + +==Justification== + +====Protocol design==== +There have been many proposals to save wire bytes when relaying blocks. Many of them have a two-fold goal of reducing block relay time and thus rely on the use of significant processing power in order to avoid introducing additional worst-case RTTs. Because this work is not focused primarily on reducing block relay time, its design is much simpler (ie does not rely on set reconciliation protocols). Still, in testing at the time of writing, nodes are able to relay blocks without the extra getblocktxn/blocktxn RTT around 90% of the time. With a smart compact-block-announcement policy, it is thus expected that this work might allow blocks to be relayed between nodes in 0.5*RTT instead of 1.5*RTT at least 75% of the time. + +====Short transaction ID calculation==== + +There are several design goals for the Short ID calculation: +* '''Performance''' The sender needs to compute short IDs for all block transactions, and the receiver for all mempool transactions they are being compared to. As we're easily talking about several thousand transactions, sub-microsecond processing per-transactions is needed. +* '''Space''' cmpctblock messages are never optional in this protocol, and contain a short ID for each non-prefilled transaction in the block. Thus, the size of short IDs is directly proportional to the maximum bandwidth savings possible. +* '''Collision resistance''' It should be hard for network participants to create transactions that cause collisions. If an attacker were able to cause such collisions, filling mempools (and, thus, blocks) with them would cause poor network propagation of new (or non-attacker, in the case of a miner) blocks. + +SipHash is a secure, fast, and simple 64-bit MAC designed for network traffic authentication and collision-resistant hash tables. We truncate the output from SipHash-2-4 to 48 bits (see next section) in order to minimize space. The resulting 48-bit hash is certainly not large enough to avoid intentionally created individual collisons, but by using the block hash as a key to SipHash, an attacker cannot predict what keys will be used once their transactions are actually included in a relayed block. We mix in a per-connection 64-bit nonce to obtain independent short IDs on every connection, so that even block creators cannot control where collisions occur, and random collisions only ever affect a small number of connections at any given time. The mixing is done using SHA256(block_header || nonce), which is slow compared to SipHash, but only done once per block. It also adds the ability for nodes to choose the nonce in a better than random way to minimize collisions, though that is not necessary for correct behaviour. Conversely, nodes can also abuse this ability to increase their ability to introduce collisions in the blocks they relay themselves. However, they can already cause more problems by simply refusing to relay blocks. That is inevitable, and this design only seeks to prevent network-wide misbehavior. + +====Random collision probabilty==== + +Thanks to the block-header-based SipHash keys, we can assume that the only collisions on links between honest nodes are random ones. + +For each of the ''t'' block transactions, the receiver will compare its received short ID with that of a set of ''m'' mempool transactions. We assume that each of those ''t'' has a chance ''r'' to be included in that set of ''m''. If we use ''B'' bits short IDs, for each comparison between a received short ID and a mempool transaction, there is a chance of ''P = 1 - 1 / 2^B'' that a mismatch is detected as such. + +When comparing a given block transaction to the whole set of mempool transactions, there are 5 cases to distinguish: +# The receiver has exactly one match, which is the correct one. This has chance ''r * P^(m - 1)''. +# The receiver has no matches. This has chance ''(1 - r) * P^m''. +# The receiver has at least two matches, one of which is correct. This has chance ''r * (1 - P^(m - 1))''. +# The receiver has at least two matches, both of which are incorrect. This has chance ''(1 - r) * (1 - P^m - m * (1 - P) * P^(m - 1))''. +# The receiver has exactly one match, but an incorrect one. This has chance ''(1 - r) * m * (1 - P) * P^(m - 1)''. + +(note that these 5 numbers always add up to 100%) + +In case 1, we're good. In cases 2, 3, or 4, we request the full transaction because we know we're uncertain. Only in case 5, we fail to reconstruct. The chance that case 5 does not occur in any of the ''t'' transactions in a block is ''(1 - (1 - r) * m * (1 - P) * P^(m - 1))^t''. This expression is well approximated by ''1 - (1 - r) * m * (1 - P) * t'' = ''1 - (1 - r) * m * t / 2^B''. Thus, if we want only one in F block transmissions between honest nodes to fail under the conservative ''r = 0'' assumption, we need ''log2(F * m * t)'' bits hash functions. + +This means that ''B = 48'' bits short IDs suffice for blocks with up to ''t = 10000'' transactions, mempools up to ''m = 100000'' transactions, with failure to reconstruct at most one in ''F = 281474'' blocks. Since failure to reconstruct just means we fall back to normal inv/header based relay, it isn't necessary to avoid such failure completely. It just needs to be sufficiently rare they have a lower impact than random transmission failures (for example, network disconnection, node overloaded, ...). + +==Backward compatibility== + +Older clients remain fully compatible and interoperable after this change. + +==Implementation== + +https://github.com/bitcoin/bitcoin/pull/8068 + +==Acknowledgements== + +Thanks to Gregory Maxwell for the initial suggestion as well as a lot of back-and-forth design and significant testing. +Thanks to Nicolas Dorier for the protocol flow diagram. + +==Copyright== + +This document is placed in the public domain. diff --git a/bip-0152/protocol-flow.png b/bip-0152/protocol-flow.png Binary files differnew file mode 100644 index 0000000..0d2b07d --- /dev/null +++ b/bip-0152/protocol-flow.png |
