From e3ec9cf9db5cc8eb2af86c500f6fd77643502aba Mon Sep 17 00:00:00 2001 From: zizelevak Date: Mon, 23 Jan 2017 12:30:55 +0100 Subject: Create Czech.txt --- bip-0039/Czech.txt | 2048 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 2048 insertions(+) create mode 100644 bip-0039/Czech.txt diff --git a/bip-0039/Czech.txt b/bip-0039/Czech.txt new file mode 100644 index 0000000..24863ef --- /dev/null +++ b/bip-0039/Czech.txt @@ -0,0 +1,2048 @@ +abeceda +adresa +akce +aktovka +alej +alkohol +ananas +andulka +anekdota +anketa +antika +archa +asfalt +asistent +astma +astronom +atlas +atletika +atol +autobus +azyl +babka +bacil +baculka +badatel +bageta +bagr +bahno +bachor +bakterie +balada +baletka +balkon +balonek +balvan +balza +bambus +bankomat +barbar +baret +bariera +barman +baroko +barva +baterka +batoh +bavlna +bazalka +bazilika +bazuka +bedna +beran +beseda +bestie +beton +bezinka +bezmoc +beztak +bezva +bicykl +bidlo +biftek +bikiny +bilance +biograf +biolog +bitva +bizon +blaf +blahobyt +blatouch +bledule +blecha +blesk +blikat +blizna +blokovat +bloudit +blud +bobek +bobr +bodlina +bodnout +bohatost +bojkot +bojovat +bokorys +bolest +borec +borovice +borskev +bota +boubel +bouda +bouchat +boule +boxer +bradavka +brambora +branka +bratr +brepta +briketa +brko +brloh +bronz +broskev +brunetka +brusinka +brzda +brzy +bublina +bubnovat +buditel +budka +budova +bufet +buchta +bujarost +bukvice +buldok +bulva +bunda +bunkr +burza +butik +buvol +buzola +bydlet +bylina +bytovka +bzukot +capart +carevna +cedr +cedule +cejch +cela +celer +celkem +celnice +cenina +cennost +cenovka +centrum +cenzor +cestopis +cetka +cibule +cigareta +cihelna +cihla +cinkot +cirkus +cisterna +citace +citrus +cizinec +cizost +clona +cokoliv +couvnout +ctitel +ctnost +cudnost +cuketa +cukr +cupot +cvaknout +cval +cvik +cvrkot +cyklista +daleko +dareba +datel +datum +dcera +debata +decibel +deficit +dechovka +dekl +dekret +demokrat +deprese +derby +deska +detektiv +dikobraz +diktovat +diplom +disk +divadlo +divoch +dlaha +dlouho +dluhopis +dnes +dobro +dobytek +docent +dodnes +dohled +dohoda +dohra +dochutit +dojem +dojnice +doklad +dokola +doktor +dokument +dolar +doleva +dolina +doma +dominant +domluvit +domov +donutit +dopad +dopis +doplnit +doposud +doprovod +dopustit +dorazit +dorost +dort +dosah +doslov +dostatek +dosud +dosyta +dotaz +dotek +dotknout +doufat +doutnat +dovozce +dozadu +doznat +dozorce +drahota +drak +dramatik +dravec +draze +drdol +drobnost +drogerie +drozd +drsnost +drtivost +drzost +duben +dudek +duha +duhovka +duchovno +dusit +dusno +dutost +dvojice +dvorec +ekolog +ekonomie +elektron +elipsa +email +emise +emoce +empatie +epizoda +epocha +epos +esej +esemeska +esence +eskorta +eskymo +etiketa +euro +evoluce +exekuce +exkurze +expedice +exploze +export +extrakt +fabrika +facka +fajfka +fakulta +falzum +fanatik +fanda +fantazie +fara +farmacie +faul +favorit +fazole +federace +fejeton +fenka +fialka +fiflena +figurant +filozof +filtr +finance +finta +fixa +fjord +flanel +flirt +flotila +folklor +fond +fosfor +fotbal +fotka +foton +frakce +freska +fronta +funkce +fyzika +galeje +garant +genetika +geolog +gilotina +glejt +globus +golem +golfista +gondola +gotika +graf +gramofon +granule +grep +gril +grog +groteska +guma +hadice +hadr +hafan +hala +halenka +hanba +hanopis +harfa +harpuna +havran +hebkost +hejkal +hejno +hejtman +hektar +helma +herec +herna +heslo +hezky +hezoun +historik +hladovka +hlasivky +hlava +hledat +hlen +hlodavec +hloh +hloupost +hltan +hlubina +hmat +hmota +hmyz +hnis +hnojivo +hnout +hoblina +hoboj +hodiny +hodlat +hodnota +hodovat +hoch +hojnost +hokej +holinky +holka +holub +homole +honitba +honorace +horal +horizont +horko +horlivec +hormon +hornina +horoskop +horstvo +hospoda +hostina +hotovost +houba +houf +houpat +houska +hovor +hradba +hranice +hravost +hrazda +hrbolek +hrdina +hrdlo +hrdost +hrnek +hrobka +hromada +hrot +hrouda +hrozen +hrstka +hrubost +hryzat +hubenost +hubnout +hudba +hukot +humr +husita +hustota +hvozd +hybnost +hydrant +hygiena +hymna +hysterik +chalupa +chapadlo +charita +chata +cheb +chechtat +chemie +chichot +chile +chirurg +chlad +chleba +chlor +chlubit +chmel +chmura +chobot +chodba +chochol +cholera +chopit +choroba +chov +chrapot +chrlit +chrom +chrt +chrup +chtivost +chudina +chutnat +chvat +chvilka +chvost +chyba +chystat +chytit +idylka +ihned +ikona +iluze +imunita +infekce +inflace +inkaso +inovace +inspekce +internet +invalida +investor +inzerce +ironie +jablko +jahoda +jachta +jakmile +jakost +jalovec +jantar +jarmark +jaro +jasan +jasno +jatka +javor +jazyk +jedinec +jedle +jednatel +jehlan +jekot +jelen +jelito +jemnost +jenom +jepice +jeseter +jestli +jevit +jezdec +jezero +jinak +jindy +jinoch +jiskra +jistota +jitrnice +jizva +jmenovat +jogurt +jurta +kabaret +kabel +kabinet +kadet +kadidlo +kafe +kahan +kachna +kajak +kajuta +kakao +kaktus +kalamita +kalhoty +kalibr +kalnost +kamera +kamkoliv +kamna +kanibal +kanoe +kantor +kapalina +kapela +kapitola +kapka +kaple +kapota +kapr +kapusta +karamel +karfiol +karotka +karton +kasa +kasino +kastrol +katalog +katedra +kauce +kauza +kavalec +kazajka +kazeta +kazivost +kdekoliv +kdesi +kedluben +kemp +keramika +kino +klacek +kladivo +klam +klapot +klasika +klaun +klec +klenba +klepat +klesnout +klid +klisna +klobouk +klokan +klopa +kloub +klubovna +klusat +kluzkost +kmen +kmit +kmotr +kniha +knot +koberec +kobka +kobliha +kobyla +kocour +kohout +kojenec +kokos +koktejl +kolaps +koleda +kolo +komando +kometa +komik +komnata +komora +kompas +komunita +konat +koncept +kondice +konec +konfese +kongres +konina +konkurs +kontakt +konzerva +kopanec +kopie +kopnout +koprovka +korbel +korektor +kormidlo +koroptev +korpus +koruna +koryto +korzet +kosatec +kostka +kotel +kotleta +koukat +koupelna +kousek +kouzlo +kovboj +koza +kozoroh +krabice +krach +krajina +kralovat +krasopis +kravata +kredit +krejcar +kresba +kreveta +kriket +kritik +krize +krkavec +krmelec +krmivo +krocan +krok +kronika +kroupa +krovka +krtek +kruhadlo +krupice +krutost +krvinka +krychle +krypta +krystal +kryt +kudlanka +kufr +kujnost +kukla +kulajda +kulich +kulka +kulomet +kultura +kuna +kupodivu +kurt +kurzor +kutil +kvalita +kvasinka +kvestor +kynolog +kyselina +kytara +kytice +kytka +kytovec +kyvadlo +labrador +ladnost +lahev +lachtan +laik +lakomec +lampa +lanovka +lasice +laso +lastura +latinka +lavina +lebka +leckdy +leden +lednice +ledovka +ledvina +legenda +legie +legrace +lehce +lehkost +lehnout +lektvar +lenochod +lentilky +lepenka +lepidlo +lepra +letadlo +letec +letmo +letokruh +levhart +levitace +levnost +levobok +lhota +libra +lidojed +lidskost +lihovina +lichotka +lijavec +lilek +limetka +limitace +linie +linka +listopad +litina +litovat +lobista +logicky +logoped +lokalita +loket +lopata +lopuch +lord +losos +lotr +loudal +louh +louka +louskat +lovec +lstivost +lucerna +lucifer +lump +luneta +lusk +lustrace +lvice +lyra +lyrika +lysina +madam +madlo +magistr +machr +majetek +majitel +majorita +makak +makovice +makrela +malba +malina +malovat +malvice +maminka +mandle +marnost +marodka +masakr +maskot +masopust +matice +matrika +maturita +mazanec +mazivo +mazlit +mazurka +mdloba +meditace +medovina +mechanik +mejdan +meloun +mentolka +metla +metoda +metr +mezera +migrace +mihnout +mihule +mikina +mikrofon +milenec +milimetr +milost +mimika +mincovna +minibar +minomet +minulost +miska +mistr +mixer +mladost +mlha +mlhovina +mlok +mlsat +mluvit +mnich +mnohem +mobil +mocensky +mocnost +modelka +modlitba +mohyla +mokrost +molekula +momentka +monarcha +monokl +monstrum +montovat +monzun +mosaz +moskyt +most +motivace +motorka +motyka +moudrost +moucha +movitost +mozaika +mozek +mozol +mramor +mravenec +mrkev +mrtvola +mrzet +mrzutost +mstitel +mudrc +muflon +mulat +mumie +munice +muset +mutace +muzeum +muzika +myslivec +mzda +nabourat +nadace +nadbytek +nadhoz +nadlouho +nadobro +nadpis +nadrobno +naftalen +nahlas +nahnat +nahota +nahradit +nachytat +naivita +najednou +najisto +najmout +naklonit +nakonec +nakrmit +nalevo +namazat +namluvit +naoko +naopak +naostro +napadat +napevno +naplnit +napnout +naposled +naprosto +narodit +naruby +narychlo +nasadit +nasekat +naslepo +nastat +natolik +natrvalo +natvrdo +navenek +navrch +navzdory +nazvat +nebe +necky +nedaleko +nedbat +neduh +nefrit +negace +nehet +nehoda +nechat +nejen +nejprve +neklid +nelibost +nemilost +nemoc +neochota +neonka +nepokoj +nerost +nerv +nesmysl +nesoulad +nestor +netvor +neuron +nevina +nezvykle +nicota +nijak +nikam +nikdy +nikl +nikterak +nimrod +nitro +nocleh +nohavice +nominace +norek +normativ +nositel +nosnost +nouze +noviny +novotvar +nozdra +nuda +nudle +nuget +nutit +nutnost +nynfa +obal +obarvit +obava +obdiv +obec +obehnat +obejmout +obezita +obhajoba +obilnice +objasnit +objekt +obklopit +oblast +oblek +obliba +obloha +obluda +obnos +obohatit +obojek +obout +obrazec +obrna +obruba +obrys +obsah +obsluha +obstarat +obuv +obvaz +obvinit +obvod +obvykle +obyvatel +obzor +ocas +ocel +ocenit +ocitnout +odboj +odbyt +odcizit +odebrat +odeslat +odevzdat +odezva +odhadce +odhodit +odchod +odjet +odjinud +odkaz +odkoupit +odliv +odluka +odmlka +odolnost +odpad +odpis +odplout +odpor +odpustit +odpykat +odrazka +odsoudit +odstup +odsun +odtok +odtud +odvaha +odveta +odvolat +odvracet +odznak +ofina +ofsajd +ohlas +ohnisko +ohrada +ohrozit +ohryzek +ochladit +ochota +ochrana +okap +okenice +oklika +okno +okouzlit +okovy +okrasa +okres +okrsek +okruh +oktet +okupant +okurka +okusit +olejnina +olizovat +omak +omeleta +omezit +omladina +omlouvat +omluva +omyl +onehdy +onkolog +opakovat +opasek +operace +opice +opilost +opisovat +opora +opozice +opravdu +oproti +opuka +orbital +orgie +orchest +orlice +orloj +ortel +osada +oschnout +osika +osivo +oslava +oslepit +oslnit +oslovit +osnova +osoba +osolit +ospalec +osten +ostraha +ostuda +ostych +osvojit +oteplit +otisk +otop +otrhanec +otrlost +otrok +otruby +otvor +ovanout +ovar +oves +ovlivnit +ovoce +oxid +ozdoba +ozon +pacient +padouch +pagoda +pahorek +pachatel +pakt +palanda +palec +palivo +paluba +pamflet +pamlsek +panenka +panika +panna +panovat +panstvo +pantofle +paprika +parketa +parodie +parta +paruka +paryba +paseka +pasivum +pastelka +patent +patrona +pavouk +pazneht +pazourek +pecka +pedagog +pejsek +peklo +peleton +penalta +pendrek +penze +pero +pestrost +petarda +petice +petrolej +pevnina +pexeso +pianista +piha +pijavice +pikle +piknik +piliny +pilnost +pilulka +pinzeta +pipeta +pisatel +pistole +pitevna +pivnice +pivovar +placenta +plakat +plamen +planeta +plastika +platit +plavidlo +plaz +plech +plemeno +plenta +ples +pletivo +plevel +plivat +plnit +plno +plodina +plocha +plomba +plout +pluk +plyn +pobavit +pobyt +pocit +poctivec +podat +podcenit +podepsat +podhled +podivit +podklad +podle +podmanit +podnik +podoba +podpora +podraz +podstata +podvod +podzim +poezie +pohanka +pohnutka +pohovor +pohroma +pohyb +pochod +pointa +pojistka +pojmout +pokazit +pokles +pokoj +pokrok +pokuta +pokyn +poledne +polibek +polknout +poloha +polynom +pomalu +pominout +pomlka +pomoc +pomsta +pomyslet +ponechat +ponorka +ponton +ponurost +popadat +popel +popisek +poplach +poprosit +popsat +popud +poradce +porce +porod +porucha +poryv +posadit +posed +posila +poskok +poslanec +posoudit +pospolu +postava +posudek +posyp +potah +potkan +potlesk +potomek +potrava +potupa +potvora +poukaz +pouto +pouzdro +povaha +povidla +povlak +povoz +povrch +povstat +povyk +povzdech +pozdrav +pozemek +poznatek +pozor +pozvat +pracovat +praktika +prales +praotec +praporek +pravda +princip +prkno +probudit +procento +prodej +profese +prohra +projekt +prolomit +promile +pronikat +propad +prorok +prosba +proton +proutek +provaz +prskavka +prsten +prudkost +prvek +prvohory +psanec +psovod +pstruh +ptactvo +puberta +pudl +puch +pukavec +puklina +pukrle +pult +pumpa +punc +pupen +pusa +pusinka +pustina +putovat +putyka +pyramida +pysk +pytel +racek +radiace +radnice +radon +raft +ragby +rachot +raketa +rameno +rampouch +rande +rarach +rasovna +rastr +ratolest +razance +razidlo +reagovat +reakce +recept +redaktor +rejnok +reklama +rekord +rekrut +rektor +revize +revma +revolver +rezerva +riskovat +riziko +robotika +rodokmen +rohovka +rokle +rokoko +romaneto +ropovod +ropucha +rorejs +rosol +rostlina +rotmistr +rotunda +roubenka +roucho +roup +roura +rovina +rovnice +rozbor +rozdat +rozeznat +rozhodce +rozchod +rozinka +rozjezd +rozkaz +rozloha +rozmar +rozpad +rozruch +rozsah +roztok +rozum +rozvod +ruchadlo +rukavice +rukopis +ryba +rybolov +rychlost +rypadlo +rytec +rytina +ryzost +sadista +sahat +sahel +sako +samec +samizdat +samota +sanitka +sardinka +sasanka +satelit +sazba +sazenice +sbor +sebranka +secese +sedadlo +sedlo +sehnat +sejmout +sekera +sekta +sekunda +sekvoje +semeno +seno +servis +sesadit +seshora +seskok +seslat +sestra +sesuv +sesypat +setba +setina +setkat +setnout +setrvat +sever +seznam +shoda +shrnout +schovat +sifon +silnice +sirka +sirotek +sirup +situace +skalisko +skanzen +skaut +skeptik +skica +skladba +sklenice +sklivec +sklo +skluz +skoba +skokan +skoro +skripta +skrz +skupina +skvost +skvrna +slabika +sladidlo +slanina +slast +slavnost +sledovat +slepec +sleva +slezina +slib +slina +sliznice +slon +sloupek +slovo +sluha +sluch +slunce +slupka +slza +smetana +smilstvo +smlouva +smog +smola +smrad +smrk +smrtka +smutek +smysl +snad +snaha +snob +sobota +sodovka +socha +sokol +sopka +sotva +souboj +soucit +soudce +souhlas +soulad +soumrak +souprava +soused +soutok +souviset +spalovna +spasitel +spis +splav +spodek +spojenec +spolu +spornost +spousta +spratek +sprcha +spustit +sranda +sraz +srdce +srna +srnec +srovnat +srpen +srst +srub +stanice +starosta +statika +stavba +stehno +stezka +stodola +stolek +stopa +storno +stoupat +strach +stres +strhnout +strom +struna +studna +stupnice +stvol +styk +subjekt +subtropy +sudost +suchar +suknice +sundat +sunout +surovina +sutana +svah +svalstvo +svatba +svazek +svetr +svisle +svitek +svoboda +svorka +svrab +sykavka +sykot +synek +synovec +sypat +sypkost +syrovost +sysel +sytost +tabletka +tabule +tahoun +tajemno +tajfun +tajga +tajit +tajnost +taktika +tamhle +tampon +tancovat +tanec +tankista +tapeta +tatarka +tavenina +tazatel +tehdy +technika +tekutina +telefon +temnota +tendence +tenista +tenor +teplota +tepna +teprve +terapie +textil +ticho +titulek +tkadlec +tkanina +tlapka +tleskat +tlukot +tlupa +tmel +toaleta +topinka +topol +torzo +touha +toulec +tradice +traktor +tramp +trasa +trefit +trest +trezor +trhavina +trhlina +trochu +trojice +troska +trouba +trpce +trpitel +trpkost +trubec +truhlice +truchlit +trus +trvat +tudy +tuhnout +tuhost +tundra +tunika +turista +turnaj +tuzemsko +tvaroh +tvorba +tvrdost +tvrz +tygr +tykev +ubohost +uboze +ubrat +ubrousek +ubrus +ubytovna +uctivost +udivit +uhradit +ucho +ujednat +ujistit +ujmout +ukazatel +uklidnit +uklonit +ukotvit +ukrojit +ulice +ulita +ulovit +umyvadlo +uniforma +uniknout +upadnout +uplatnit +uplynout +upoutat +upravit +uran +urazit +usednout +usilovat +usmrtit +usnadnit +usnout +usoudit +ustlat +ustrnout +utahovat +utkat +utlumit +utonout +utopenec +utrousit +uvalit +uvolnit +uvozovka +uzdravit +uzel +uzenina +uzlina +uznat +vagon +valcha +valoun +vana +vandal +vanilka +varan +varhany +varovat +vatra +vcelku +vdova +vedro +vegetace +vejce +velbloud +veletrh +velitel +velmoc +velryba +venkov +verze +veselka +veskrze +vesnice +vespod +vesta +veterina +veverka +vchod +vibrace +videohra +vidina +vidle +vichr +vila +vinice +viset +vitamin +vize +vizitka +vjezd +vklad +vkus +vlajka +vlak +vlasec +vlevo +vlhkost +vliv +vlnovka +vloni +vloupat +vnucovat +vnuk +voda +vodoznak +vodstvo +vojensky +vojna +vojsko +volant +volba +volit +volno +voskovka +vozidlo +vozovna +vpravo +vrabec +vracet +vrah +vrata +vrba +vrhat +vrcholek +vrstva +vrtule +vsadit +vstoupit +vstup +vtip +vybavit +vybrat +vydat +vydra +vyfotit +vyhledat +vyhnout +vyhodit +vyhradit +vyhubit +vychovat +vyjasnit +vyjet +vyjmout +vyklopit +vykonat +vylekat +vymazat +vymezit +vymizet +vymyslet +vynechat +vynikat +vynutit +vypadat +vyplatit +vypravit +vypustit +vyrazit +vyrovnat +vyslovit +vysoko +vystavit +vysunout +vysypat +vytasit +vytesat +vytratit +vyvinout +vyvolat +vyvrhel +vyzdobit +vyznat +vzadu +vzbudit +vzdor +vzduch +vzdychat +vzestup +vzhledem +vzchopit +vzkaz +vzlykat +vznik +vzorek +vzpoura +vztah +vztek +zabrat +zabydlet +zadarmo +zadusit +zafoukat +zahltit +zahodit +zahrada +zahynout +zachovat +zajatec +zajet +zajistit +zaklepat +zakoupit +zalepit +zamezit +zamotat +zamyslet +zanechat +zanikat +zaplatit +zapojit +zapsat +zarazit +zastavit +zasunout +zatajit +zatemnit +zatknout +zaujmout +zavalit +zavelet +zavinit +zavolat +zavrtat +zazvonit +zbavit +zbrusu +zbudovat +zbytek +zdaleka +zdarma +zdatnost +zdivo +zdobit +zdroj +zdrtit +zdvih +zdymadlo +zelenina +zeman +zemina +zeptat +zezadu +zezdola +zhltnout +zhluboka +zhotovit +zhruba +zima +zimnice +zjemnit +zklamat +zkoumat +zkratka +zkumavka +zlato +zlehka +zloba +zlom +zlost +zlozvyk +zmapovat +zmar +zmatek +zmije +zmizet +zmocnit +zmodrat +zmrzlina +znak +znalost +znamenat +znovu +zobrazit +zotavit +zoubek +zoufale +zplodit +zpomalit +zprava +zprostit +zprudka +zprvu +zrada +zranit +zrcadlo +zrnitost +zrno +zrovna +zrychlit +zrzek +zticha +ztratit +zubovina +zubr +zvednout +zvenku +zvesela +zvon +zvrat +zvukovod +zvyk -- cgit v1.2.3 From 5be84b01f1641ddbaa922b3d44e2728bd7849608 Mon Sep 17 00:00:00 2001 From: zizelevak Date: Mon, 23 Jan 2017 12:32:52 +0100 Subject: Rename Czech.txt to czech.txt --- bip-0039/Czech.txt | 2048 ---------------------------------------------------- bip-0039/czech.txt | 2048 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 2048 insertions(+), 2048 deletions(-) delete mode 100644 bip-0039/Czech.txt create mode 100644 bip-0039/czech.txt diff --git a/bip-0039/Czech.txt b/bip-0039/Czech.txt deleted file mode 100644 index 24863ef..0000000 --- a/bip-0039/Czech.txt +++ /dev/null @@ -1,2048 +0,0 @@ -abeceda -adresa -akce -aktovka -alej -alkohol -ananas -andulka -anekdota -anketa -antika -archa -asfalt -asistent -astma -astronom -atlas -atletika -atol -autobus -azyl -babka -bacil -baculka -badatel -bageta -bagr -bahno -bachor -bakterie -balada -baletka -balkon -balonek -balvan -balza -bambus -bankomat -barbar -baret -bariera -barman -baroko -barva -baterka -batoh -bavlna -bazalka -bazilika -bazuka -bedna -beran -beseda -bestie -beton -bezinka -bezmoc -beztak -bezva -bicykl -bidlo -biftek -bikiny -bilance -biograf -biolog -bitva -bizon -blaf -blahobyt -blatouch -bledule -blecha -blesk -blikat -blizna -blokovat -bloudit -blud -bobek -bobr -bodlina -bodnout -bohatost -bojkot -bojovat -bokorys -bolest -borec -borovice -borskev -bota -boubel -bouda -bouchat -boule -boxer -bradavka -brambora -branka -bratr -brepta -briketa -brko -brloh -bronz -broskev -brunetka -brusinka -brzda -brzy -bublina -bubnovat -buditel -budka -budova -bufet -buchta -bujarost -bukvice -buldok -bulva -bunda -bunkr -burza -butik -buvol -buzola -bydlet -bylina -bytovka -bzukot -capart -carevna -cedr -cedule -cejch -cela -celer -celkem -celnice -cenina -cennost -cenovka -centrum -cenzor -cestopis -cetka -cibule -cigareta -cihelna -cihla -cinkot -cirkus -cisterna -citace -citrus -cizinec -cizost -clona -cokoliv -couvnout -ctitel -ctnost -cudnost -cuketa -cukr -cupot -cvaknout -cval -cvik -cvrkot -cyklista -daleko -dareba -datel -datum -dcera -debata -decibel -deficit -dechovka -dekl -dekret -demokrat -deprese -derby -deska -detektiv -dikobraz -diktovat -diplom -disk -divadlo -divoch -dlaha -dlouho -dluhopis -dnes -dobro -dobytek -docent -dodnes -dohled -dohoda -dohra -dochutit -dojem -dojnice -doklad -dokola -doktor -dokument -dolar -doleva -dolina -doma -dominant -domluvit -domov -donutit -dopad -dopis -doplnit -doposud -doprovod -dopustit -dorazit -dorost -dort -dosah -doslov -dostatek -dosud -dosyta -dotaz -dotek -dotknout -doufat -doutnat -dovozce -dozadu -doznat -dozorce -drahota -drak -dramatik -dravec -draze -drdol -drobnost -drogerie -drozd -drsnost -drtivost -drzost -duben -dudek -duha -duhovka -duchovno -dusit -dusno -dutost -dvojice -dvorec -ekolog -ekonomie -elektron -elipsa -email -emise -emoce -empatie -epizoda -epocha -epos -esej -esemeska -esence -eskorta -eskymo -etiketa -euro -evoluce -exekuce -exkurze -expedice -exploze -export -extrakt -fabrika -facka -fajfka -fakulta -falzum -fanatik -fanda -fantazie -fara -farmacie -faul -favorit -fazole -federace -fejeton -fenka -fialka -fiflena -figurant -filozof -filtr -finance -finta -fixa -fjord -flanel -flirt -flotila -folklor -fond -fosfor -fotbal -fotka -foton -frakce -freska -fronta -funkce -fyzika -galeje -garant -genetika -geolog -gilotina -glejt -globus -golem -golfista -gondola -gotika -graf -gramofon -granule -grep -gril -grog -groteska -guma -hadice -hadr -hafan -hala -halenka -hanba -hanopis -harfa -harpuna -havran -hebkost -hejkal -hejno -hejtman -hektar -helma -herec -herna -heslo -hezky -hezoun -historik -hladovka -hlasivky -hlava -hledat -hlen -hlodavec -hloh -hloupost -hltan -hlubina -hmat -hmota -hmyz -hnis -hnojivo -hnout -hoblina -hoboj -hodiny -hodlat -hodnota -hodovat -hoch -hojnost -hokej -holinky -holka -holub -homole -honitba -honorace -horal -horizont -horko -horlivec -hormon -hornina -horoskop -horstvo -hospoda -hostina -hotovost -houba -houf -houpat -houska -hovor -hradba -hranice -hravost -hrazda -hrbolek -hrdina -hrdlo -hrdost -hrnek -hrobka -hromada -hrot -hrouda -hrozen -hrstka -hrubost -hryzat -hubenost -hubnout -hudba -hukot -humr -husita -hustota -hvozd -hybnost -hydrant -hygiena -hymna -hysterik -chalupa -chapadlo -charita -chata -cheb -chechtat -chemie -chichot -chile -chirurg -chlad -chleba -chlor -chlubit -chmel -chmura -chobot -chodba -chochol -cholera -chopit -choroba -chov -chrapot -chrlit -chrom -chrt -chrup -chtivost -chudina -chutnat -chvat -chvilka -chvost -chyba -chystat -chytit -idylka -ihned -ikona -iluze -imunita -infekce -inflace -inkaso -inovace -inspekce -internet -invalida -investor -inzerce -ironie -jablko -jahoda -jachta -jakmile -jakost -jalovec -jantar -jarmark -jaro -jasan -jasno -jatka -javor -jazyk -jedinec -jedle -jednatel -jehlan -jekot -jelen -jelito -jemnost -jenom -jepice -jeseter -jestli -jevit -jezdec -jezero -jinak -jindy -jinoch -jiskra -jistota -jitrnice -jizva -jmenovat -jogurt -jurta -kabaret -kabel -kabinet -kadet -kadidlo -kafe -kahan -kachna -kajak -kajuta -kakao -kaktus -kalamita -kalhoty -kalibr -kalnost -kamera -kamkoliv -kamna -kanibal -kanoe -kantor -kapalina -kapela -kapitola -kapka -kaple -kapota -kapr -kapusta -karamel -karfiol -karotka -karton -kasa -kasino -kastrol -katalog -katedra -kauce -kauza -kavalec -kazajka -kazeta -kazivost -kdekoliv -kdesi -kedluben -kemp -keramika -kino -klacek -kladivo -klam -klapot -klasika -klaun -klec -klenba -klepat -klesnout -klid -klisna -klobouk -klokan -klopa -kloub -klubovna -klusat -kluzkost -kmen -kmit -kmotr -kniha -knot -koberec -kobka -kobliha -kobyla -kocour -kohout -kojenec -kokos -koktejl -kolaps -koleda -kolo -komando -kometa -komik -komnata -komora -kompas -komunita -konat -koncept -kondice -konec -konfese -kongres -konina -konkurs -kontakt -konzerva -kopanec -kopie -kopnout -koprovka -korbel -korektor -kormidlo -koroptev -korpus -koruna -koryto -korzet -kosatec -kostka -kotel -kotleta -koukat -koupelna -kousek -kouzlo -kovboj -koza -kozoroh -krabice -krach -krajina -kralovat -krasopis -kravata -kredit -krejcar -kresba -kreveta -kriket -kritik -krize -krkavec -krmelec -krmivo -krocan -krok -kronika -kroupa -krovka -krtek -kruhadlo -krupice -krutost -krvinka -krychle -krypta -krystal -kryt -kudlanka -kufr -kujnost -kukla -kulajda -kulich -kulka -kulomet -kultura -kuna -kupodivu -kurt -kurzor -kutil -kvalita -kvasinka -kvestor -kynolog -kyselina -kytara -kytice -kytka -kytovec -kyvadlo -labrador -ladnost -lahev -lachtan -laik -lakomec -lampa -lanovka -lasice -laso -lastura -latinka -lavina -lebka -leckdy -leden -lednice -ledovka -ledvina -legenda -legie -legrace -lehce -lehkost -lehnout -lektvar -lenochod -lentilky -lepenka -lepidlo -lepra -letadlo -letec -letmo -letokruh -levhart -levitace -levnost -levobok -lhota -libra -lidojed -lidskost -lihovina -lichotka -lijavec -lilek -limetka -limitace -linie -linka -listopad -litina -litovat -lobista -logicky -logoped -lokalita -loket -lopata -lopuch -lord -losos -lotr -loudal -louh -louka -louskat -lovec -lstivost -lucerna -lucifer -lump -luneta -lusk -lustrace -lvice -lyra -lyrika -lysina -madam -madlo -magistr -machr -majetek -majitel -majorita -makak -makovice -makrela -malba -malina -malovat -malvice -maminka -mandle -marnost -marodka -masakr -maskot -masopust -matice -matrika -maturita -mazanec -mazivo -mazlit -mazurka -mdloba -meditace -medovina -mechanik -mejdan -meloun -mentolka -metla -metoda -metr -mezera -migrace -mihnout -mihule -mikina -mikrofon -milenec -milimetr -milost -mimika -mincovna -minibar -minomet -minulost -miska -mistr -mixer -mladost -mlha -mlhovina -mlok -mlsat -mluvit -mnich -mnohem -mobil -mocensky -mocnost -modelka -modlitba -mohyla -mokrost -molekula -momentka -monarcha -monokl -monstrum -montovat -monzun -mosaz -moskyt -most -motivace -motorka -motyka -moudrost -moucha -movitost -mozaika -mozek -mozol -mramor -mravenec -mrkev -mrtvola -mrzet -mrzutost -mstitel -mudrc -muflon -mulat -mumie -munice -muset -mutace -muzeum -muzika -myslivec -mzda -nabourat -nadace -nadbytek -nadhoz -nadlouho -nadobro -nadpis -nadrobno -naftalen -nahlas -nahnat -nahota -nahradit -nachytat -naivita -najednou -najisto -najmout -naklonit -nakonec -nakrmit -nalevo -namazat -namluvit -naoko -naopak -naostro -napadat -napevno -naplnit -napnout -naposled -naprosto -narodit -naruby -narychlo -nasadit -nasekat -naslepo -nastat -natolik -natrvalo -natvrdo -navenek -navrch -navzdory -nazvat -nebe -necky -nedaleko -nedbat -neduh -nefrit -negace -nehet -nehoda -nechat -nejen -nejprve -neklid -nelibost -nemilost -nemoc -neochota -neonka -nepokoj -nerost -nerv -nesmysl -nesoulad -nestor -netvor -neuron -nevina -nezvykle -nicota -nijak -nikam -nikdy -nikl -nikterak -nimrod -nitro -nocleh -nohavice -nominace -norek -normativ -nositel -nosnost -nouze -noviny -novotvar -nozdra -nuda -nudle -nuget -nutit -nutnost -nynfa -obal -obarvit -obava -obdiv -obec -obehnat -obejmout -obezita -obhajoba -obilnice -objasnit -objekt -obklopit -oblast -oblek -obliba -obloha -obluda -obnos -obohatit -obojek -obout -obrazec -obrna -obruba -obrys -obsah -obsluha -obstarat -obuv -obvaz -obvinit -obvod -obvykle -obyvatel -obzor -ocas -ocel -ocenit -ocitnout -odboj -odbyt -odcizit -odebrat -odeslat -odevzdat -odezva -odhadce -odhodit -odchod -odjet -odjinud -odkaz -odkoupit -odliv -odluka -odmlka -odolnost -odpad -odpis -odplout -odpor -odpustit -odpykat -odrazka -odsoudit -odstup -odsun -odtok -odtud -odvaha -odveta -odvolat -odvracet -odznak -ofina -ofsajd -ohlas -ohnisko -ohrada -ohrozit -ohryzek -ochladit -ochota -ochrana -okap -okenice -oklika -okno -okouzlit -okovy -okrasa -okres -okrsek -okruh -oktet -okupant -okurka -okusit -olejnina -olizovat -omak -omeleta -omezit -omladina -omlouvat -omluva -omyl -onehdy -onkolog -opakovat -opasek -operace -opice -opilost -opisovat -opora -opozice -opravdu -oproti -opuka -orbital -orgie -orchest -orlice -orloj -ortel -osada -oschnout -osika -osivo -oslava -oslepit -oslnit -oslovit -osnova -osoba -osolit -ospalec -osten -ostraha -ostuda -ostych -osvojit -oteplit -otisk -otop -otrhanec -otrlost -otrok -otruby -otvor -ovanout -ovar -oves -ovlivnit -ovoce -oxid -ozdoba -ozon -pacient -padouch -pagoda -pahorek -pachatel -pakt -palanda -palec -palivo -paluba -pamflet -pamlsek -panenka -panika -panna -panovat -panstvo -pantofle -paprika -parketa -parodie -parta -paruka -paryba -paseka -pasivum -pastelka -patent -patrona -pavouk -pazneht -pazourek -pecka -pedagog -pejsek -peklo -peleton -penalta -pendrek -penze -pero -pestrost -petarda -petice -petrolej -pevnina -pexeso -pianista -piha -pijavice -pikle -piknik -piliny -pilnost -pilulka -pinzeta -pipeta -pisatel -pistole -pitevna -pivnice -pivovar -placenta -plakat -plamen -planeta -plastika -platit -plavidlo -plaz -plech -plemeno -plenta -ples -pletivo -plevel -plivat -plnit -plno -plodina -plocha -plomba -plout -pluk -plyn -pobavit -pobyt -pocit -poctivec -podat -podcenit -podepsat -podhled -podivit -podklad -podle -podmanit -podnik -podoba -podpora -podraz -podstata -podvod -podzim -poezie -pohanka -pohnutka -pohovor -pohroma -pohyb -pochod -pointa -pojistka -pojmout -pokazit -pokles -pokoj -pokrok -pokuta -pokyn -poledne -polibek -polknout -poloha -polynom -pomalu -pominout -pomlka -pomoc -pomsta -pomyslet -ponechat -ponorka -ponton -ponurost -popadat -popel -popisek -poplach -poprosit -popsat -popud -poradce -porce -porod -porucha -poryv -posadit -posed -posila -poskok -poslanec -posoudit -pospolu -postava -posudek -posyp -potah -potkan -potlesk -potomek -potrava -potupa -potvora -poukaz -pouto -pouzdro -povaha -povidla -povlak -povoz -povrch -povstat -povyk -povzdech -pozdrav -pozemek -poznatek -pozor -pozvat -pracovat -praktika -prales -praotec -praporek -pravda -princip -prkno -probudit -procento -prodej -profese -prohra -projekt -prolomit -promile -pronikat -propad -prorok -prosba -proton -proutek -provaz -prskavka -prsten -prudkost -prvek -prvohory -psanec -psovod -pstruh -ptactvo -puberta -pudl -puch -pukavec -puklina -pukrle -pult -pumpa -punc -pupen -pusa -pusinka -pustina -putovat -putyka -pyramida -pysk -pytel -racek -radiace -radnice -radon -raft -ragby -rachot -raketa -rameno -rampouch -rande -rarach -rasovna -rastr -ratolest -razance -razidlo -reagovat -reakce -recept -redaktor -rejnok -reklama -rekord -rekrut -rektor -revize -revma -revolver -rezerva -riskovat -riziko -robotika -rodokmen -rohovka -rokle -rokoko -romaneto -ropovod -ropucha -rorejs -rosol -rostlina -rotmistr -rotunda -roubenka -roucho -roup -roura -rovina -rovnice -rozbor -rozdat -rozeznat -rozhodce -rozchod -rozinka -rozjezd -rozkaz -rozloha -rozmar -rozpad -rozruch -rozsah -roztok -rozum -rozvod -ruchadlo -rukavice -rukopis -ryba -rybolov -rychlost -rypadlo -rytec -rytina -ryzost -sadista -sahat -sahel -sako -samec -samizdat -samota -sanitka -sardinka -sasanka -satelit -sazba -sazenice -sbor -sebranka -secese -sedadlo -sedlo -sehnat -sejmout -sekera -sekta -sekunda -sekvoje -semeno -seno -servis -sesadit -seshora -seskok -seslat -sestra -sesuv -sesypat -setba -setina -setkat -setnout -setrvat -sever -seznam -shoda -shrnout -schovat -sifon -silnice -sirka -sirotek -sirup -situace -skalisko -skanzen -skaut -skeptik -skica -skladba -sklenice -sklivec -sklo -skluz -skoba -skokan -skoro -skripta -skrz -skupina -skvost -skvrna -slabika -sladidlo -slanina -slast -slavnost -sledovat -slepec -sleva -slezina -slib -slina -sliznice -slon -sloupek -slovo -sluha -sluch -slunce -slupka -slza -smetana -smilstvo -smlouva -smog -smola -smrad -smrk -smrtka -smutek -smysl -snad -snaha -snob -sobota -sodovka -socha -sokol -sopka -sotva -souboj -soucit -soudce -souhlas -soulad -soumrak -souprava -soused -soutok -souviset -spalovna -spasitel -spis -splav -spodek -spojenec -spolu -spornost -spousta -spratek -sprcha -spustit -sranda -sraz -srdce -srna -srnec -srovnat -srpen -srst -srub -stanice -starosta -statika -stavba -stehno -stezka -stodola -stolek -stopa -storno -stoupat -strach -stres -strhnout -strom -struna -studna -stupnice -stvol -styk -subjekt -subtropy -sudost -suchar -suknice -sundat -sunout -surovina -sutana -svah -svalstvo -svatba -svazek -svetr -svisle -svitek -svoboda -svorka -svrab -sykavka -sykot -synek -synovec -sypat -sypkost -syrovost -sysel -sytost -tabletka -tabule -tahoun -tajemno -tajfun -tajga -tajit -tajnost -taktika -tamhle -tampon -tancovat -tanec -tankista -tapeta -tatarka -tavenina -tazatel -tehdy -technika -tekutina -telefon -temnota -tendence -tenista -tenor -teplota -tepna -teprve -terapie -textil -ticho -titulek -tkadlec -tkanina -tlapka -tleskat -tlukot -tlupa -tmel -toaleta -topinka -topol -torzo -touha -toulec -tradice -traktor -tramp -trasa -trefit -trest -trezor -trhavina -trhlina -trochu -trojice -troska -trouba -trpce -trpitel -trpkost -trubec -truhlice -truchlit -trus -trvat -tudy -tuhnout -tuhost -tundra -tunika -turista -turnaj -tuzemsko -tvaroh -tvorba -tvrdost -tvrz -tygr -tykev -ubohost -uboze -ubrat -ubrousek -ubrus -ubytovna -uctivost -udivit -uhradit -ucho -ujednat -ujistit -ujmout -ukazatel -uklidnit -uklonit -ukotvit -ukrojit -ulice -ulita -ulovit -umyvadlo -uniforma -uniknout -upadnout -uplatnit -uplynout -upoutat -upravit -uran -urazit -usednout -usilovat -usmrtit -usnadnit -usnout -usoudit -ustlat -ustrnout -utahovat -utkat -utlumit -utonout -utopenec -utrousit -uvalit -uvolnit -uvozovka -uzdravit -uzel -uzenina -uzlina -uznat -vagon -valcha -valoun -vana -vandal -vanilka -varan -varhany -varovat -vatra -vcelku -vdova -vedro -vegetace -vejce -velbloud -veletrh -velitel -velmoc -velryba -venkov -verze -veselka -veskrze -vesnice -vespod -vesta -veterina -veverka -vchod -vibrace -videohra -vidina -vidle -vichr -vila -vinice -viset -vitamin -vize -vizitka -vjezd -vklad -vkus -vlajka -vlak -vlasec -vlevo -vlhkost -vliv -vlnovka -vloni -vloupat -vnucovat -vnuk -voda -vodoznak -vodstvo -vojensky -vojna -vojsko -volant -volba -volit -volno -voskovka -vozidlo -vozovna -vpravo -vrabec -vracet -vrah -vrata -vrba -vrhat -vrcholek -vrstva -vrtule -vsadit -vstoupit -vstup -vtip -vybavit -vybrat -vydat -vydra -vyfotit -vyhledat -vyhnout -vyhodit -vyhradit -vyhubit -vychovat -vyjasnit -vyjet -vyjmout -vyklopit -vykonat -vylekat -vymazat -vymezit -vymizet -vymyslet -vynechat -vynikat -vynutit -vypadat -vyplatit -vypravit -vypustit -vyrazit -vyrovnat -vyslovit -vysoko -vystavit -vysunout -vysypat -vytasit -vytesat -vytratit -vyvinout -vyvolat -vyvrhel -vyzdobit -vyznat -vzadu -vzbudit -vzdor -vzduch -vzdychat -vzestup -vzhledem -vzchopit -vzkaz -vzlykat -vznik -vzorek -vzpoura -vztah -vztek -zabrat -zabydlet -zadarmo -zadusit -zafoukat -zahltit -zahodit -zahrada -zahynout -zachovat -zajatec -zajet -zajistit -zaklepat -zakoupit -zalepit -zamezit -zamotat -zamyslet -zanechat -zanikat -zaplatit -zapojit -zapsat -zarazit -zastavit -zasunout -zatajit -zatemnit -zatknout -zaujmout -zavalit -zavelet -zavinit -zavolat -zavrtat -zazvonit -zbavit -zbrusu -zbudovat -zbytek -zdaleka -zdarma -zdatnost -zdivo -zdobit -zdroj -zdrtit -zdvih -zdymadlo -zelenina -zeman -zemina -zeptat -zezadu -zezdola -zhltnout -zhluboka -zhotovit -zhruba -zima -zimnice -zjemnit -zklamat -zkoumat -zkratka -zkumavka -zlato -zlehka -zloba -zlom -zlost -zlozvyk -zmapovat -zmar -zmatek -zmije -zmizet -zmocnit -zmodrat -zmrzlina -znak -znalost -znamenat -znovu -zobrazit -zotavit -zoubek -zoufale -zplodit -zpomalit -zprava -zprostit -zprudka -zprvu -zrada -zranit -zrcadlo -zrnitost -zrno -zrovna -zrychlit -zrzek -zticha -ztratit -zubovina -zubr -zvednout -zvenku -zvesela -zvon -zvrat -zvukovod -zvyk diff --git a/bip-0039/czech.txt b/bip-0039/czech.txt new file mode 100644 index 0000000..24863ef --- /dev/null +++ b/bip-0039/czech.txt @@ -0,0 +1,2048 @@ +abeceda +adresa +akce +aktovka +alej +alkohol +ananas +andulka +anekdota +anketa +antika +archa +asfalt +asistent +astma +astronom +atlas +atletika +atol +autobus +azyl +babka +bacil +baculka +badatel +bageta +bagr +bahno +bachor +bakterie +balada +baletka +balkon +balonek +balvan +balza +bambus +bankomat +barbar +baret +bariera +barman +baroko +barva +baterka +batoh +bavlna +bazalka +bazilika +bazuka +bedna +beran +beseda +bestie +beton +bezinka +bezmoc +beztak +bezva +bicykl +bidlo +biftek +bikiny +bilance +biograf +biolog +bitva +bizon +blaf +blahobyt +blatouch +bledule +blecha +blesk +blikat +blizna +blokovat +bloudit +blud +bobek +bobr +bodlina +bodnout +bohatost +bojkot +bojovat +bokorys +bolest +borec +borovice +borskev +bota +boubel +bouda +bouchat +boule +boxer +bradavka +brambora +branka +bratr +brepta +briketa +brko +brloh +bronz +broskev +brunetka +brusinka +brzda +brzy +bublina +bubnovat +buditel +budka +budova +bufet +buchta +bujarost +bukvice +buldok +bulva +bunda +bunkr +burza +butik +buvol +buzola +bydlet +bylina +bytovka +bzukot +capart +carevna +cedr +cedule +cejch +cela +celer +celkem +celnice +cenina +cennost +cenovka +centrum +cenzor +cestopis +cetka +cibule +cigareta +cihelna +cihla +cinkot +cirkus +cisterna +citace +citrus +cizinec +cizost +clona +cokoliv +couvnout +ctitel +ctnost +cudnost +cuketa +cukr +cupot +cvaknout +cval +cvik +cvrkot +cyklista +daleko +dareba +datel +datum +dcera +debata +decibel +deficit +dechovka +dekl +dekret +demokrat +deprese +derby +deska +detektiv +dikobraz +diktovat +diplom +disk +divadlo +divoch +dlaha +dlouho +dluhopis +dnes +dobro +dobytek +docent +dodnes +dohled +dohoda +dohra +dochutit +dojem +dojnice +doklad +dokola +doktor +dokument +dolar +doleva +dolina +doma +dominant +domluvit +domov +donutit +dopad +dopis +doplnit +doposud +doprovod +dopustit +dorazit +dorost +dort +dosah +doslov +dostatek +dosud +dosyta +dotaz +dotek +dotknout +doufat +doutnat +dovozce +dozadu +doznat +dozorce +drahota +drak +dramatik +dravec +draze +drdol +drobnost +drogerie +drozd +drsnost +drtivost +drzost +duben +dudek +duha +duhovka +duchovno +dusit +dusno +dutost +dvojice +dvorec +ekolog +ekonomie +elektron +elipsa +email +emise +emoce +empatie +epizoda +epocha +epos +esej +esemeska +esence +eskorta +eskymo +etiketa +euro +evoluce +exekuce +exkurze +expedice +exploze +export +extrakt +fabrika +facka +fajfka +fakulta +falzum +fanatik +fanda +fantazie +fara +farmacie +faul +favorit +fazole +federace +fejeton +fenka +fialka +fiflena +figurant +filozof +filtr +finance +finta +fixa +fjord +flanel +flirt +flotila +folklor +fond +fosfor +fotbal +fotka +foton +frakce +freska +fronta +funkce +fyzika +galeje +garant +genetika +geolog +gilotina +glejt +globus +golem +golfista +gondola +gotika +graf +gramofon +granule +grep +gril +grog +groteska +guma +hadice +hadr +hafan +hala +halenka +hanba +hanopis +harfa +harpuna +havran +hebkost +hejkal +hejno +hejtman +hektar +helma +herec +herna +heslo +hezky +hezoun +historik +hladovka +hlasivky +hlava +hledat +hlen +hlodavec +hloh +hloupost +hltan +hlubina +hmat +hmota +hmyz +hnis +hnojivo +hnout +hoblina +hoboj +hodiny +hodlat +hodnota +hodovat +hoch +hojnost +hokej +holinky +holka +holub +homole +honitba +honorace +horal +horizont +horko +horlivec +hormon +hornina +horoskop +horstvo +hospoda +hostina +hotovost +houba +houf +houpat +houska +hovor +hradba +hranice +hravost +hrazda +hrbolek +hrdina +hrdlo +hrdost +hrnek +hrobka +hromada +hrot +hrouda +hrozen +hrstka +hrubost +hryzat +hubenost +hubnout +hudba +hukot +humr +husita +hustota +hvozd +hybnost +hydrant +hygiena +hymna +hysterik +chalupa +chapadlo +charita +chata +cheb +chechtat +chemie +chichot +chile +chirurg +chlad +chleba +chlor +chlubit +chmel +chmura +chobot +chodba +chochol +cholera +chopit +choroba +chov +chrapot +chrlit +chrom +chrt +chrup +chtivost +chudina +chutnat +chvat +chvilka +chvost +chyba +chystat +chytit +idylka +ihned +ikona +iluze +imunita +infekce +inflace +inkaso +inovace +inspekce +internet +invalida +investor +inzerce +ironie +jablko +jahoda +jachta +jakmile +jakost +jalovec +jantar +jarmark +jaro +jasan +jasno +jatka +javor +jazyk +jedinec +jedle +jednatel +jehlan +jekot +jelen +jelito +jemnost +jenom +jepice +jeseter +jestli +jevit +jezdec +jezero +jinak +jindy +jinoch +jiskra +jistota +jitrnice +jizva +jmenovat +jogurt +jurta +kabaret +kabel +kabinet +kadet +kadidlo +kafe +kahan +kachna +kajak +kajuta +kakao +kaktus +kalamita +kalhoty +kalibr +kalnost +kamera +kamkoliv +kamna +kanibal +kanoe +kantor +kapalina +kapela +kapitola +kapka +kaple +kapota +kapr +kapusta +karamel +karfiol +karotka +karton +kasa +kasino +kastrol +katalog +katedra +kauce +kauza +kavalec +kazajka +kazeta +kazivost +kdekoliv +kdesi +kedluben +kemp +keramika +kino +klacek +kladivo +klam +klapot +klasika +klaun +klec +klenba +klepat +klesnout +klid +klisna +klobouk +klokan +klopa +kloub +klubovna +klusat +kluzkost +kmen +kmit +kmotr +kniha +knot +koberec +kobka +kobliha +kobyla +kocour +kohout +kojenec +kokos +koktejl +kolaps +koleda +kolo +komando +kometa +komik +komnata +komora +kompas +komunita +konat +koncept +kondice +konec +konfese +kongres +konina +konkurs +kontakt +konzerva +kopanec +kopie +kopnout +koprovka +korbel +korektor +kormidlo +koroptev +korpus +koruna +koryto +korzet +kosatec +kostka +kotel +kotleta +koukat +koupelna +kousek +kouzlo +kovboj +koza +kozoroh +krabice +krach +krajina +kralovat +krasopis +kravata +kredit +krejcar +kresba +kreveta +kriket +kritik +krize +krkavec +krmelec +krmivo +krocan +krok +kronika +kroupa +krovka +krtek +kruhadlo +krupice +krutost +krvinka +krychle +krypta +krystal +kryt +kudlanka +kufr +kujnost +kukla +kulajda +kulich +kulka +kulomet +kultura +kuna +kupodivu +kurt +kurzor +kutil +kvalita +kvasinka +kvestor +kynolog +kyselina +kytara +kytice +kytka +kytovec +kyvadlo +labrador +ladnost +lahev +lachtan +laik +lakomec +lampa +lanovka +lasice +laso +lastura +latinka +lavina +lebka +leckdy +leden +lednice +ledovka +ledvina +legenda +legie +legrace +lehce +lehkost +lehnout +lektvar +lenochod +lentilky +lepenka +lepidlo +lepra +letadlo +letec +letmo +letokruh +levhart +levitace +levnost +levobok +lhota +libra +lidojed +lidskost +lihovina +lichotka +lijavec +lilek +limetka +limitace +linie +linka +listopad +litina +litovat +lobista +logicky +logoped +lokalita +loket +lopata +lopuch +lord +losos +lotr +loudal +louh +louka +louskat +lovec +lstivost +lucerna +lucifer +lump +luneta +lusk +lustrace +lvice +lyra +lyrika +lysina +madam +madlo +magistr +machr +majetek +majitel +majorita +makak +makovice +makrela +malba +malina +malovat +malvice +maminka +mandle +marnost +marodka +masakr +maskot +masopust +matice +matrika +maturita +mazanec +mazivo +mazlit +mazurka +mdloba +meditace +medovina +mechanik +mejdan +meloun +mentolka +metla +metoda +metr +mezera +migrace +mihnout +mihule +mikina +mikrofon +milenec +milimetr +milost +mimika +mincovna +minibar +minomet +minulost +miska +mistr +mixer +mladost +mlha +mlhovina +mlok +mlsat +mluvit +mnich +mnohem +mobil +mocensky +mocnost +modelka +modlitba +mohyla +mokrost +molekula +momentka +monarcha +monokl +monstrum +montovat +monzun +mosaz +moskyt +most +motivace +motorka +motyka +moudrost +moucha +movitost +mozaika +mozek +mozol +mramor +mravenec +mrkev +mrtvola +mrzet +mrzutost +mstitel +mudrc +muflon +mulat +mumie +munice +muset +mutace +muzeum +muzika +myslivec +mzda +nabourat +nadace +nadbytek +nadhoz +nadlouho +nadobro +nadpis +nadrobno +naftalen +nahlas +nahnat +nahota +nahradit +nachytat +naivita +najednou +najisto +najmout +naklonit +nakonec +nakrmit +nalevo +namazat +namluvit +naoko +naopak +naostro +napadat +napevno +naplnit +napnout +naposled +naprosto +narodit +naruby +narychlo +nasadit +nasekat +naslepo +nastat +natolik +natrvalo +natvrdo +navenek +navrch +navzdory +nazvat +nebe +necky +nedaleko +nedbat +neduh +nefrit +negace +nehet +nehoda +nechat +nejen +nejprve +neklid +nelibost +nemilost +nemoc +neochota +neonka +nepokoj +nerost +nerv +nesmysl +nesoulad +nestor +netvor +neuron +nevina +nezvykle +nicota +nijak +nikam +nikdy +nikl +nikterak +nimrod +nitro +nocleh +nohavice +nominace +norek +normativ +nositel +nosnost +nouze +noviny +novotvar +nozdra +nuda +nudle +nuget +nutit +nutnost +nynfa +obal +obarvit +obava +obdiv +obec +obehnat +obejmout +obezita +obhajoba +obilnice +objasnit +objekt +obklopit +oblast +oblek +obliba +obloha +obluda +obnos +obohatit +obojek +obout +obrazec +obrna +obruba +obrys +obsah +obsluha +obstarat +obuv +obvaz +obvinit +obvod +obvykle +obyvatel +obzor +ocas +ocel +ocenit +ocitnout +odboj +odbyt +odcizit +odebrat +odeslat +odevzdat +odezva +odhadce +odhodit +odchod +odjet +odjinud +odkaz +odkoupit +odliv +odluka +odmlka +odolnost +odpad +odpis +odplout +odpor +odpustit +odpykat +odrazka +odsoudit +odstup +odsun +odtok +odtud +odvaha +odveta +odvolat +odvracet +odznak +ofina +ofsajd +ohlas +ohnisko +ohrada +ohrozit +ohryzek +ochladit +ochota +ochrana +okap +okenice +oklika +okno +okouzlit +okovy +okrasa +okres +okrsek +okruh +oktet +okupant +okurka +okusit +olejnina +olizovat +omak +omeleta +omezit +omladina +omlouvat +omluva +omyl +onehdy +onkolog +opakovat +opasek +operace +opice +opilost +opisovat +opora +opozice +opravdu +oproti +opuka +orbital +orgie +orchest +orlice +orloj +ortel +osada +oschnout +osika +osivo +oslava +oslepit +oslnit +oslovit +osnova +osoba +osolit +ospalec +osten +ostraha +ostuda +ostych +osvojit +oteplit +otisk +otop +otrhanec +otrlost +otrok +otruby +otvor +ovanout +ovar +oves +ovlivnit +ovoce +oxid +ozdoba +ozon +pacient +padouch +pagoda +pahorek +pachatel +pakt +palanda +palec +palivo +paluba +pamflet +pamlsek +panenka +panika +panna +panovat +panstvo +pantofle +paprika +parketa +parodie +parta +paruka +paryba +paseka +pasivum +pastelka +patent +patrona +pavouk +pazneht +pazourek +pecka +pedagog +pejsek +peklo +peleton +penalta +pendrek +penze +pero +pestrost +petarda +petice +petrolej +pevnina +pexeso +pianista +piha +pijavice +pikle +piknik +piliny +pilnost +pilulka +pinzeta +pipeta +pisatel +pistole +pitevna +pivnice +pivovar +placenta +plakat +plamen +planeta +plastika +platit +plavidlo +plaz +plech +plemeno +plenta +ples +pletivo +plevel +plivat +plnit +plno +plodina +plocha +plomba +plout +pluk +plyn +pobavit +pobyt +pocit +poctivec +podat +podcenit +podepsat +podhled +podivit +podklad +podle +podmanit +podnik +podoba +podpora +podraz +podstata +podvod +podzim +poezie +pohanka +pohnutka +pohovor +pohroma +pohyb +pochod +pointa +pojistka +pojmout +pokazit +pokles +pokoj +pokrok +pokuta +pokyn +poledne +polibek +polknout +poloha +polynom +pomalu +pominout +pomlka +pomoc +pomsta +pomyslet +ponechat +ponorka +ponton +ponurost +popadat +popel +popisek +poplach +poprosit +popsat +popud +poradce +porce +porod +porucha +poryv +posadit +posed +posila +poskok +poslanec +posoudit +pospolu +postava +posudek +posyp +potah +potkan +potlesk +potomek +potrava +potupa +potvora +poukaz +pouto +pouzdro +povaha +povidla +povlak +povoz +povrch +povstat +povyk +povzdech +pozdrav +pozemek +poznatek +pozor +pozvat +pracovat +praktika +prales +praotec +praporek +pravda +princip +prkno +probudit +procento +prodej +profese +prohra +projekt +prolomit +promile +pronikat +propad +prorok +prosba +proton +proutek +provaz +prskavka +prsten +prudkost +prvek +prvohory +psanec +psovod +pstruh +ptactvo +puberta +pudl +puch +pukavec +puklina +pukrle +pult +pumpa +punc +pupen +pusa +pusinka +pustina +putovat +putyka +pyramida +pysk +pytel +racek +radiace +radnice +radon +raft +ragby +rachot +raketa +rameno +rampouch +rande +rarach +rasovna +rastr +ratolest +razance +razidlo +reagovat +reakce +recept +redaktor +rejnok +reklama +rekord +rekrut +rektor +revize +revma +revolver +rezerva +riskovat +riziko +robotika +rodokmen +rohovka +rokle +rokoko +romaneto +ropovod +ropucha +rorejs +rosol +rostlina +rotmistr +rotunda +roubenka +roucho +roup +roura +rovina +rovnice +rozbor +rozdat +rozeznat +rozhodce +rozchod +rozinka +rozjezd +rozkaz +rozloha +rozmar +rozpad +rozruch +rozsah +roztok +rozum +rozvod +ruchadlo +rukavice +rukopis +ryba +rybolov +rychlost +rypadlo +rytec +rytina +ryzost +sadista +sahat +sahel +sako +samec +samizdat +samota +sanitka +sardinka +sasanka +satelit +sazba +sazenice +sbor +sebranka +secese +sedadlo +sedlo +sehnat +sejmout +sekera +sekta +sekunda +sekvoje +semeno +seno +servis +sesadit +seshora +seskok +seslat +sestra +sesuv +sesypat +setba +setina +setkat +setnout +setrvat +sever +seznam +shoda +shrnout +schovat +sifon +silnice +sirka +sirotek +sirup +situace +skalisko +skanzen +skaut +skeptik +skica +skladba +sklenice +sklivec +sklo +skluz +skoba +skokan +skoro +skripta +skrz +skupina +skvost +skvrna +slabika +sladidlo +slanina +slast +slavnost +sledovat +slepec +sleva +slezina +slib +slina +sliznice +slon +sloupek +slovo +sluha +sluch +slunce +slupka +slza +smetana +smilstvo +smlouva +smog +smola +smrad +smrk +smrtka +smutek +smysl +snad +snaha +snob +sobota +sodovka +socha +sokol +sopka +sotva +souboj +soucit +soudce +souhlas +soulad +soumrak +souprava +soused +soutok +souviset +spalovna +spasitel +spis +splav +spodek +spojenec +spolu +spornost +spousta +spratek +sprcha +spustit +sranda +sraz +srdce +srna +srnec +srovnat +srpen +srst +srub +stanice +starosta +statika +stavba +stehno +stezka +stodola +stolek +stopa +storno +stoupat +strach +stres +strhnout +strom +struna +studna +stupnice +stvol +styk +subjekt +subtropy +sudost +suchar +suknice +sundat +sunout +surovina +sutana +svah +svalstvo +svatba +svazek +svetr +svisle +svitek +svoboda +svorka +svrab +sykavka +sykot +synek +synovec +sypat +sypkost +syrovost +sysel +sytost +tabletka +tabule +tahoun +tajemno +tajfun +tajga +tajit +tajnost +taktika +tamhle +tampon +tancovat +tanec +tankista +tapeta +tatarka +tavenina +tazatel +tehdy +technika +tekutina +telefon +temnota +tendence +tenista +tenor +teplota +tepna +teprve +terapie +textil +ticho +titulek +tkadlec +tkanina +tlapka +tleskat +tlukot +tlupa +tmel +toaleta +topinka +topol +torzo +touha +toulec +tradice +traktor +tramp +trasa +trefit +trest +trezor +trhavina +trhlina +trochu +trojice +troska +trouba +trpce +trpitel +trpkost +trubec +truhlice +truchlit +trus +trvat +tudy +tuhnout +tuhost +tundra +tunika +turista +turnaj +tuzemsko +tvaroh +tvorba +tvrdost +tvrz +tygr +tykev +ubohost +uboze +ubrat +ubrousek +ubrus +ubytovna +uctivost +udivit +uhradit +ucho +ujednat +ujistit +ujmout +ukazatel +uklidnit +uklonit +ukotvit +ukrojit +ulice +ulita +ulovit +umyvadlo +uniforma +uniknout +upadnout +uplatnit +uplynout +upoutat +upravit +uran +urazit +usednout +usilovat +usmrtit +usnadnit +usnout +usoudit +ustlat +ustrnout +utahovat +utkat +utlumit +utonout +utopenec +utrousit +uvalit +uvolnit +uvozovka +uzdravit +uzel +uzenina +uzlina +uznat +vagon +valcha +valoun +vana +vandal +vanilka +varan +varhany +varovat +vatra +vcelku +vdova +vedro +vegetace +vejce +velbloud +veletrh +velitel +velmoc +velryba +venkov +verze +veselka +veskrze +vesnice +vespod +vesta +veterina +veverka +vchod +vibrace +videohra +vidina +vidle +vichr +vila +vinice +viset +vitamin +vize +vizitka +vjezd +vklad +vkus +vlajka +vlak +vlasec +vlevo +vlhkost +vliv +vlnovka +vloni +vloupat +vnucovat +vnuk +voda +vodoznak +vodstvo +vojensky +vojna +vojsko +volant +volba +volit +volno +voskovka +vozidlo +vozovna +vpravo +vrabec +vracet +vrah +vrata +vrba +vrhat +vrcholek +vrstva +vrtule +vsadit +vstoupit +vstup +vtip +vybavit +vybrat +vydat +vydra +vyfotit +vyhledat +vyhnout +vyhodit +vyhradit +vyhubit +vychovat +vyjasnit +vyjet +vyjmout +vyklopit +vykonat +vylekat +vymazat +vymezit +vymizet +vymyslet +vynechat +vynikat +vynutit +vypadat +vyplatit +vypravit +vypustit +vyrazit +vyrovnat +vyslovit +vysoko +vystavit +vysunout +vysypat +vytasit +vytesat +vytratit +vyvinout +vyvolat +vyvrhel +vyzdobit +vyznat +vzadu +vzbudit +vzdor +vzduch +vzdychat +vzestup +vzhledem +vzchopit +vzkaz +vzlykat +vznik +vzorek +vzpoura +vztah +vztek +zabrat +zabydlet +zadarmo +zadusit +zafoukat +zahltit +zahodit +zahrada +zahynout +zachovat +zajatec +zajet +zajistit +zaklepat +zakoupit +zalepit +zamezit +zamotat +zamyslet +zanechat +zanikat +zaplatit +zapojit +zapsat +zarazit +zastavit +zasunout +zatajit +zatemnit +zatknout +zaujmout +zavalit +zavelet +zavinit +zavolat +zavrtat +zazvonit +zbavit +zbrusu +zbudovat +zbytek +zdaleka +zdarma +zdatnost +zdivo +zdobit +zdroj +zdrtit +zdvih +zdymadlo +zelenina +zeman +zemina +zeptat +zezadu +zezdola +zhltnout +zhluboka +zhotovit +zhruba +zima +zimnice +zjemnit +zklamat +zkoumat +zkratka +zkumavka +zlato +zlehka +zloba +zlom +zlost +zlozvyk +zmapovat +zmar +zmatek +zmije +zmizet +zmocnit +zmodrat +zmrzlina +znak +znalost +znamenat +znovu +zobrazit +zotavit +zoubek +zoufale +zplodit +zpomalit +zprava +zprostit +zprudka +zprvu +zrada +zranit +zrcadlo +zrnitost +zrno +zrovna +zrychlit +zrzek +zticha +ztratit +zubovina +zubr +zvednout +zvenku +zvesela +zvon +zvrat +zvukovod +zvyk -- cgit v1.2.3 From 469a12b91845ba6c7da77b48e34e3abd4aeeb66b Mon Sep 17 00:00:00 2001 From: zizelevak Date: Mon, 23 Jan 2017 13:46:36 +0100 Subject: Update bip-0039-wordlists.md --- bip-0039/bip-0039-wordlists.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/bip-0039/bip-0039-wordlists.md b/bip-0039/bip-0039-wordlists.md index aef1a23..f31a7f2 100644 --- a/bip-0039/bip-0039-wordlists.md +++ b/bip-0039/bip-0039-wordlists.md @@ -7,6 +7,7 @@ * [Chinese (Traditional)](chinese_traditional.txt) * [French](french.txt) * [Italian](italian.txt) +* [Czech](czech.txt) ##Wordlists (Special Considerations) @@ -81,3 +82,17 @@ Words chosen using the following rules: Rules 11 and 12 prevent the selection words that are not different enough. This makes each word more recognizable among others and less error prone. For example: the wordlist contains "atono", then "atomo" is rejected, but "atomico" is good. All the words have been manually selected and automatically checked against the rules. + +### Czech + +Credits: @zizelevak + +Words chosen using the following rules: + +1. Words are 4-8 letters long. +2. Words can be uniquely determined typing the first 4 letters. +3. Only words containing all letters without diacritical marks. (It was the hardest task, because in one third of all Czech letters has diacritical marks.) +4. Only nouns, werbs and adverbs, no other word types. All words are in basic form. +5. No personal names or geografical names. +6. No very similar words with 1 letter of difference. +7. No words already used in other language mnemonic sets (english, italian, french, spanish). Letters with diacritical marks from these sets are counted as analogous letters without diacritical marks. -- cgit v1.2.3 From 9eefdc9151b8765c571261ba6ea9fc5dd9195363 Mon Sep 17 00:00:00 2001 From: zizelevak Date: Mon, 23 Jan 2017 13:47:51 +0100 Subject: Update bip-0039-wordlists.md --- bip-0039/bip-0039-wordlists.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bip-0039/bip-0039-wordlists.md b/bip-0039/bip-0039-wordlists.md index f31a7f2..37ba588 100644 --- a/bip-0039/bip-0039-wordlists.md +++ b/bip-0039/bip-0039-wordlists.md @@ -85,7 +85,7 @@ All the words have been manually selected and automatically checked against the ### Czech -Credits: @zizelevak +Credits: @zizelevak (Jan Lansky zizelevak@gmail.com) Words chosen using the following rules: -- cgit v1.2.3 From af07fb993e8b9c322eb69f7f6282da41a2e6eca4 Mon Sep 17 00:00:00 2001 From: zizelevak Date: Fri, 27 Jan 2017 16:49:45 +0100 Subject: Update czech.txt MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Words are sorting according English alphabet (Czech sorting has difference in β€œch”) --- bip-0039/czech.txt | 156 ++++++++++++++++++++++++++--------------------------- 1 file changed, 78 insertions(+), 78 deletions(-) diff --git a/bip-0039/czech.txt b/bip-0039/czech.txt index 24863ef..db69cfa 100644 --- a/bip-0039/czech.txt +++ b/bip-0039/czech.txt @@ -20,13 +20,13 @@ atol autobus azyl babka +bachor bacil baculka badatel bageta bagr bahno -bachor bakterie balada baletka @@ -69,8 +69,8 @@ bizon blaf blahobyt blatouch -bledule blecha +bledule blesk blikat blizna @@ -91,8 +91,8 @@ borovice borskev bota boubel -bouda bouchat +bouda boule boxer bradavka @@ -111,11 +111,11 @@ brzda brzy bublina bubnovat +buchta buditel budka budova bufet -buchta bujarost bukvice buldok @@ -146,6 +146,43 @@ centrum cenzor cestopis cetka +chalupa +chapadlo +charita +chata +cheb +chechtat +chemie +chichot +chile +chirurg +chlad +chleba +chlor +chlubit +chmel +chmura +chobot +chochol +chodba +cholera +chopit +choroba +chov +chrapot +chrlit +chrom +chrt +chrup +chtivost +chudina +chutnat +chvat +chvilka +chvost +chyba +chystat +chytit cibule cigareta cihelna @@ -177,9 +214,9 @@ datel datum dcera debata +dechovka decibel deficit -dechovka dekl dekret demokrat @@ -200,11 +237,11 @@ dnes dobro dobytek docent +dochutit dodnes dohled dohoda dohra -dochutit dojem dojnice doklad @@ -255,10 +292,10 @@ drsnost drtivost drzost duben +duchovno dudek duha duhovka -duchovno dusit dusno dutost @@ -387,11 +424,11 @@ hnojivo hnout hoblina hoboj +hoch hodiny hodlat hodnota hodovat -hoch hojnost hokej holinky @@ -446,43 +483,6 @@ hydrant hygiena hymna hysterik -chalupa -chapadlo -charita -chata -cheb -chechtat -chemie -chichot -chile -chirurg -chlad -chleba -chlor -chlubit -chmel -chmura -chobot -chodba -chochol -cholera -chopit -choroba -chov -chrapot -chrlit -chrom -chrt -chrup -chtivost -chudina -chutnat -chvat -chvilka -chvost -chyba -chystat -chytit idylka ihned ikona @@ -499,8 +499,8 @@ investor inzerce ironie jablko -jahoda jachta +jahoda jakmile jakost jalovec @@ -540,11 +540,11 @@ jurta kabaret kabel kabinet +kachna kadet kadidlo kafe kahan -kachna kajak kajuta kakao @@ -719,9 +719,9 @@ kytka kytovec kyvadlo labrador +lachtan ladnost lahev -lachtan laik lakomec lampa @@ -759,10 +759,10 @@ levnost levobok lhota libra +lichotka lidojed lidskost lihovina -lichotka lijavec lilek limetka @@ -798,10 +798,10 @@ lvice lyra lyrika lysina +machr madam madlo magistr -machr majetek majitel majorita @@ -827,9 +827,9 @@ mazivo mazlit mazurka mdloba +mechanik meditace medovina -mechanik mejdan meloun mentolka @@ -881,8 +881,8 @@ most motivace motorka motyka -moudrost moucha +moudrost movitost mozaika mozek @@ -906,6 +906,7 @@ muzika myslivec mzda nabourat +nachytat nadace nadbytek nadhoz @@ -918,7 +919,6 @@ nahlas nahnat nahota nahradit -nachytat naivita najednou najisto @@ -953,6 +953,7 @@ navrch navzdory nazvat nebe +nechat necky nedaleko nedbat @@ -961,7 +962,6 @@ nefrit negace nehet nehoda -nechat nejen nejprve neklid @@ -1044,9 +1044,13 @@ obzor ocas ocel ocenit +ochladit +ochota +ochrana ocitnout odboj odbyt +odchod odcizit odebrat odeslat @@ -1054,7 +1058,6 @@ odevzdat odezva odhadce odhodit -odchod odjet odjinud odkaz @@ -1087,9 +1090,6 @@ ohnisko ohrada ohrozit ohryzek -ochladit -ochota -ochrana okap okenice oklika @@ -1127,8 +1127,8 @@ opravdu oproti opuka orbital -orgie orchest +orgie orlice orloj ortel @@ -1165,11 +1165,11 @@ ovoce oxid ozdoba ozon +pachatel pacient padouch pagoda pahorek -pachatel pakt palanda palec @@ -1244,14 +1244,15 @@ plevel plivat plnit plno -plodina plocha +plodina plomba plout pluk plyn pobavit pobyt +pochod pocit poctivec podat @@ -1275,7 +1276,6 @@ pohnutka pohovor pohroma pohyb -pochod pointa pojistka pojmout @@ -1378,8 +1378,8 @@ psovod pstruh ptactvo puberta -pudl puch +pudl pukavec puklina pukrle @@ -1396,12 +1396,12 @@ pyramida pysk pytel racek +rachot radiace radnice radon raft ragby -rachot raketa rameno rampouch @@ -1447,10 +1447,10 @@ roura rovina rovnice rozbor +rozchod rozdat rozeznat rozhodce -rozchod rozinka rozjezd rozkaz @@ -1486,6 +1486,7 @@ satelit sazba sazenice sbor +schovat sebranka secese sedadlo @@ -1515,7 +1516,6 @@ sever seznam shoda shrnout -schovat sifon silnice sirka @@ -1555,8 +1555,8 @@ sliznice slon sloupek slovo -sluha sluch +sluha slunce slupka slza @@ -1574,8 +1574,8 @@ snad snaha snob sobota -sodovka socha +sodovka sokol sopka sotva @@ -1632,8 +1632,8 @@ stvol styk subjekt subtropy -sudost suchar +sudost suknice sundat sunout @@ -1676,8 +1676,8 @@ tapeta tatarka tavenina tazatel -tehdy technika +tehdy tekutina telefon temnota @@ -1721,8 +1721,8 @@ trpce trpitel trpkost trubec -truhlice truchlit +truhlice trus trvat tudy @@ -1745,10 +1745,10 @@ ubrat ubrousek ubrus ubytovna +ucho uctivost udivit uhradit -ucho ujednat ujistit ujmout @@ -1803,6 +1803,7 @@ varhany varovat vatra vcelku +vchod vdova vedro vegetace @@ -1821,12 +1822,11 @@ vespod vesta veterina veverka -vchod vibrace +vichr videohra vidina vidle -vichr vila vinice viset @@ -1866,8 +1866,8 @@ vracet vrah vrata vrba -vrhat vrcholek +vrhat vrstva vrtule vsadit @@ -1876,6 +1876,7 @@ vstup vtip vybavit vybrat +vychovat vydat vydra vyfotit @@ -1884,7 +1885,6 @@ vyhnout vyhodit vyhradit vyhubit -vychovat vyjasnit vyjet vyjmout @@ -1919,12 +1919,12 @@ vyzdobit vyznat vzadu vzbudit +vzchopit vzdor vzduch vzdychat vzestup vzhledem -vzchopit vzkaz vzlykat vznik @@ -1934,6 +1934,7 @@ vztah vztek zabrat zabydlet +zachovat zadarmo zadusit zafoukat @@ -1941,7 +1942,6 @@ zahltit zahodit zahrada zahynout -zachovat zajatec zajet zajistit -- cgit v1.2.3 From 875f35a07a3c8690ab5ebe980258172e0ccabf90 Mon Sep 17 00:00:00 2001 From: zizelevak Date: Fri, 27 Jan 2017 16:51:06 +0100 Subject: Update bip-0039-wordlists.md --- bip-0039/bip-0039-wordlists.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/bip-0039/bip-0039-wordlists.md b/bip-0039/bip-0039-wordlists.md index 37ba588..5b4a1de 100644 --- a/bip-0039/bip-0039-wordlists.md +++ b/bip-0039/bip-0039-wordlists.md @@ -95,4 +95,5 @@ Words chosen using the following rules: 4. Only nouns, werbs and adverbs, no other word types. All words are in basic form. 5. No personal names or geografical names. 6. No very similar words with 1 letter of difference. -7. No words already used in other language mnemonic sets (english, italian, french, spanish). Letters with diacritical marks from these sets are counted as analogous letters without diacritical marks. +7. Words are sorting according English alphabet (Czech sorting has difference in "ch"). +8. No words already used in other language mnemonic sets (english, italian, french, spanish). Letters with diacritical marks from these sets are counted as analogous letters without diacritical marks. -- cgit v1.2.3 From 1a3ec3a459febc0430e13928ca03df278e4f3a44 Mon Sep 17 00:00:00 2001 From: zizelevak Date: Fri, 27 Jan 2017 17:34:25 +0100 Subject: Update czech.txt --- bip-0039/czech.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/bip-0039/czech.txt b/bip-0039/czech.txt index db69cfa..48c7730 100644 --- a/bip-0039/czech.txt +++ b/bip-0039/czech.txt @@ -88,12 +88,12 @@ bokorys bolest borec borovice -borskev bota boubel bouchat bouda boule +bourat boxer bradavka brambora @@ -135,6 +135,7 @@ carevna cedr cedule cejch +cejn cela celer celkem @@ -150,11 +151,9 @@ chalupa chapadlo charita chata -cheb chechtat chemie chichot -chile chirurg chlad chleba @@ -166,6 +165,7 @@ chobot chochol chodba cholera +chomout chopit choroba chov -- cgit v1.2.3 From 7ae1049232ffcc7c89dfb947fddbb785233a3730 Mon Sep 17 00:00:00 2001 From: zizelevak Date: Fri, 27 Jan 2017 18:23:25 +0100 Subject: Update czech.txt --- bip-0039/czech.txt | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/bip-0039/czech.txt b/bip-0039/czech.txt index 48c7730..e0fdc67 100644 --- a/bip-0039/czech.txt +++ b/bip-0039/czech.txt @@ -66,7 +66,6 @@ biograf biolog bitva bizon -blaf blahobyt blatouch blecha @@ -336,7 +335,6 @@ fanda fantazie fara farmacie -faul favorit fazole federace @@ -623,6 +621,7 @@ kokos koktejl kolaps koleda +kolize kolo komando kometa @@ -1350,6 +1349,7 @@ praktika prales praotec praporek +prase pravda princip prkno -- cgit v1.2.3 From 92a07ee121bc85def9bf7bec2165ac25172498f6 Mon Sep 17 00:00:00 2001 From: zizelevak Date: Fri, 27 Jan 2017 19:54:28 +0100 Subject: Update czech.txt --- bip-0039/czech.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/bip-0039/czech.txt b/bip-0039/czech.txt index e0fdc67..6903d25 100644 --- a/bip-0039/czech.txt +++ b/bip-0039/czech.txt @@ -1,5 +1,6 @@ abeceda adresa +agrese akce aktovka alej @@ -56,7 +57,6 @@ beton bezinka bezmoc beztak -bezva bicykl bidlo biftek @@ -312,11 +312,11 @@ epizoda epocha epos esej -esemeska esence eskorta eskymo etiketa +euforie euro evoluce exekuce @@ -521,7 +521,6 @@ jemnost jenom jepice jeseter -jestli jevit jezdec jezero @@ -1932,6 +1931,7 @@ vzorek vzpoura vztah vztek +xylofon zabrat zabydlet zachovat -- cgit v1.2.3 From dab9d97b44f5dcb60e0bcb33900aed97c58b904b Mon Sep 17 00:00:00 2001 From: zizelevak Date: Tue, 31 Jan 2017 08:08:42 +0100 Subject: Update czech.txt --- bip-0039/czech.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bip-0039/czech.txt b/bip-0039/czech.txt index 6903d25..baf835c 100644 --- a/bip-0039/czech.txt +++ b/bip-0039/czech.txt @@ -1002,7 +1002,7 @@ nudle nuget nutit nutnost -nynfa +nymfa obal obarvit obava -- cgit v1.2.3 From f4691796c60d4ceee51f06b7dacd1cc760890f35 Mon Sep 17 00:00:00 2001 From: zizelevak Date: Tue, 31 Jan 2017 08:15:12 +0100 Subject: Update czech.txt --- bip-0039/czech.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bip-0039/czech.txt b/bip-0039/czech.txt index baf835c..abaabbe 100644 --- a/bip-0039/czech.txt +++ b/bip-0039/czech.txt @@ -1125,7 +1125,7 @@ opravdu oproti opuka orbital -orchest +orchestr orgie orlice orloj -- cgit v1.2.3 From b9f09aae63ac1374fe9284afbeeda2625616871a Mon Sep 17 00:00:00 2001 From: zizelevak Date: Tue, 31 Jan 2017 08:49:29 +0100 Subject: Update czech.txt --- bip-0039/czech.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bip-0039/czech.txt b/bip-0039/czech.txt index abaabbe..a3643a8 100644 --- a/bip-0039/czech.txt +++ b/bip-0039/czech.txt @@ -1199,7 +1199,6 @@ pecka pedagog pejsek peklo -peleton penalta pendrek penze @@ -1215,6 +1214,7 @@ piha pijavice pikle piknik +piksla piliny pilnost pilulka -- cgit v1.2.3 -- cgit v1.2.3 From 8a2a06e94cf6a7ff9cb6739bf05a1aef6c8ed3be Mon Sep 17 00:00:00 2001 From: zizelevak Date: Fri, 10 Feb 2017 23:56:16 +0100 Subject: Update czech.txt --- bip-0039/czech.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/bip-0039/czech.txt b/bip-0039/czech.txt index a3643a8..cbf4651 100644 --- a/bip-0039/czech.txt +++ b/bip-0039/czech.txt @@ -39,7 +39,6 @@ bambus bankomat barbar baret -bariera barman baroko barva @@ -995,7 +994,7 @@ nositel nosnost nouze noviny -novotvar +novota nozdra nuda nudle @@ -1150,7 +1149,7 @@ osvojit oteplit otisk otop -otrhanec +otrhat otrlost otrok otruby @@ -1559,6 +1558,7 @@ sluha slunce slupka slza +smaragd smetana smilstvo smlouva -- cgit v1.2.3 From b7f682f702bfb5679b39ef19c994b191693f59b4 Mon Sep 17 00:00:00 2001 From: zizelevak Date: Sun, 12 Feb 2017 03:40:30 +0100 Subject: Update czech.txt --- bip-0039/czech.txt | 152 ++++++++++++++++++++++++++--------------------------- 1 file changed, 76 insertions(+), 76 deletions(-) diff --git a/bip-0039/czech.txt b/bip-0039/czech.txt index cbf4651..fdab4a2 100644 --- a/bip-0039/czech.txt +++ b/bip-0039/czech.txt @@ -1,3 +1,4 @@ +abdikace abeceda adresa agrese @@ -5,14 +6,18 @@ akce aktovka alej alkohol +amputace ananas andulka anekdota anketa antika +anulovat archa +arogance asfalt asistent +aspirace astma astronom atlas @@ -155,7 +160,6 @@ chichot chirurg chlad chleba -chlor chlubit chmel chmura @@ -169,7 +173,6 @@ choroba chov chrapot chrlit -chrom chrt chrup chtivost @@ -194,7 +197,7 @@ cizinec cizost clona cokoliv -couvnout +couvat ctitel ctnost cudnost @@ -215,6 +218,7 @@ debata dechovka decibel deficit +deflace dekl dekret demokrat @@ -224,8 +228,10 @@ deska detektiv dikobraz diktovat +dioda diplom disk +displej divadlo divoch dlaha @@ -287,7 +293,7 @@ drobnost drogerie drozd drsnost -drtivost +drtit drzost duben duchovno @@ -299,6 +305,7 @@ dusno dutost dvojice dvorec +dynamit ekolog ekonomie elektron @@ -309,6 +316,7 @@ emoce empatie epizoda epocha +epopej epos esej esence @@ -316,7 +324,6 @@ eskorta eskymo etiketa euforie -euro evoluce exekuce exkurze @@ -324,15 +331,11 @@ expedice exploze export extrakt -fabrika facka fajfka fakulta -falzum fanatik -fanda fantazie -fara farmacie favorit fazole @@ -340,18 +343,16 @@ federace fejeton fenka fialka -fiflena figurant filozof filtr finance finta -fixa +fixace fjord flanel flirt flotila -folklor fond fosfor fotbal @@ -360,6 +361,7 @@ foton frakce freska fronta +fukar funkce fyzika galeje @@ -367,11 +369,10 @@ garant genetika geolog gilotina +glazura glejt -globus golem golfista -gondola gotika graf gramofon @@ -383,7 +384,6 @@ groteska guma hadice hadr -hafan hala halenka hanba @@ -397,11 +397,11 @@ hejno hejtman hektar helma +hematom herec herna heslo hezky -hezoun historik hladovka hlasivky @@ -411,8 +411,9 @@ hlen hlodavec hloh hloupost -hltan +hltat hlubina +hluchota hmat hmota hmyz @@ -428,13 +429,14 @@ hodnota hodovat hojnost hokej -holinky +holinka holka holub homole honitba honorace horal +horda horizont horko horlivec @@ -539,7 +541,6 @@ kabinet kachna kadet kadidlo -kafe kahan kajak kajuta @@ -563,13 +564,11 @@ kaple kapota kapr kapusta +kapybara karamel -karfiol karotka karton kasa -kasino -kastrol katalog katedra kauce @@ -595,6 +594,7 @@ klenba klepat klesnout klid +klima klisna klobouk klokan @@ -604,10 +604,11 @@ klubovna klusat kluzkost kmen -kmit +kmitat kmotr kniha knot +koalice koberec kobka kobliha @@ -654,6 +655,7 @@ kosatec kostka kotel kotleta +kotoul koukat koupelna kousek @@ -680,6 +682,7 @@ krmivo krocan krok kronika +kropit kroupa krovka krtek @@ -718,9 +721,9 @@ kyvadlo labrador lachtan ladnost -lahev laik lakomec +lamela lampa lanovka lasice @@ -742,19 +745,16 @@ lehkost lehnout lektvar lenochod -lentilky +lentilka lepenka lepidlo -lepra letadlo letec letmo letokruh levhart levitace -levnost levobok -lhota libra lichotka lidojed @@ -763,17 +763,19 @@ lihovina lijavec lilek limetka -limitace linie linka +linoleum listopad litina litovat lobista -logicky +lodivod +logika logoped lokalita loket +lomcovat lopata lopuch lord @@ -788,17 +790,16 @@ lstivost lucerna lucifer lump -luneta lusk lustrace lvice lyra lyrika lysina -machr madam madlo magistr +mahagon majetek majitel majorita @@ -811,8 +812,8 @@ malovat malvice maminka mandle +manko marnost -marodka masakr maskot masopust @@ -827,7 +828,7 @@ mdloba mechanik meditace medovina -mejdan +melasa meloun mentolka metla @@ -849,7 +850,7 @@ minomet minulost miska mistr -mixer +mixovat mladost mlha mlhovina @@ -859,12 +860,11 @@ mluvit mnich mnohem mobil -mocensky mocnost modelka modlitba mohyla -mokrost +mokro molekula momentka monarcha @@ -880,7 +880,6 @@ motorka motyka moucha moudrost -movitost mozaika mozek mozol @@ -899,7 +898,7 @@ munice muset mutace muzeum -muzika +muzikant myslivec mzda nabourat @@ -907,14 +906,11 @@ nachytat nadace nadbytek nadhoz -nadlouho nadobro nadpis -nadrobno -naftalen nahlas nahnat -nahota +nahodile nahradit naivita najednou @@ -926,6 +922,7 @@ nakrmit nalevo namazat namluvit +nanometr naoko naopak naostro @@ -943,8 +940,6 @@ nasekat naslepo nastat natolik -natrvalo -natvrdo navenek navrch navzdory @@ -955,7 +950,6 @@ necky nedaleko nedbat neduh -nefrit negace nehet nehoda @@ -972,7 +966,6 @@ nerost nerv nesmysl nesoulad -nestor netvor neuron nevina @@ -983,13 +976,12 @@ nikam nikdy nikl nikterak -nimrod nitro nocleh nohavice nominace +nora norek -normativ nositel nosnost nouze @@ -1001,6 +993,7 @@ nudle nuget nutit nutnost +nutrie nymfa obal obarvit @@ -1097,7 +1090,6 @@ okrasa okres okrsek okruh -oktet okupant okurka okusit @@ -1111,7 +1103,6 @@ omlouvat omluva omyl onehdy -onkolog opakovat opasek operace @@ -1122,7 +1113,6 @@ opora opozice opravdu oproti -opuka orbital orchestr orgie @@ -1161,11 +1151,9 @@ ovlivnit ovoce oxid ozdoba -ozon pachatel pacient padouch -pagoda pahorek pakt palanda @@ -1187,7 +1175,7 @@ parta paruka paryba paseka -pasivum +pasivita pastelka patent patrona @@ -1198,9 +1186,11 @@ pecka pedagog pejsek peklo +peloton penalta pendrek penze +periskop pero pestrost petarda @@ -1213,8 +1203,7 @@ piha pijavice pikle piknik -piksla -piliny +pilina pilnost pilulka pinzeta @@ -1258,7 +1247,6 @@ podepsat podhled podivit podklad -podle podmanit podnik podoba @@ -1295,7 +1283,6 @@ pomsta pomyslet ponechat ponorka -ponton ponurost popadat popel @@ -1343,6 +1330,7 @@ poznatek pozor pozvat pracovat +prahory praktika prales praotec @@ -1369,6 +1357,7 @@ provaz prskavka prsten prudkost +prut prvek prvohory psanec @@ -1401,10 +1390,12 @@ radon raft ragby raketa +rakovina rameno rampouch rande rarach +rarita rasovna rastr ratolest @@ -1414,11 +1405,14 @@ reagovat reakce recept redaktor +referent +reflex rejnok reklama rekord rekrut rektor +reputace revize revma revolver @@ -1437,6 +1431,7 @@ rorejs rosol rostlina rotmistr +rotoped rotunda roubenka roucho @@ -1460,19 +1455,19 @@ rozsah roztok rozum rozvod +rubrika ruchadlo rukavice rukopis ryba rybolov rychlost +rydlo rypadlo -rytec rytina ryzost sadista sahat -sahel sako samec samizdat @@ -1488,6 +1483,7 @@ schovat sebranka secese sedadlo +sediment sedlo sehnat sejmout @@ -1520,6 +1516,7 @@ sirka sirotek sirup situace +skafandr skalisko skanzen skaut @@ -1527,7 +1524,6 @@ skeptik skica skladba sklenice -sklivec sklo skluz skoba @@ -1563,7 +1559,6 @@ smetana smilstvo smlouva smog -smola smrad smrk smrtka @@ -1595,9 +1590,9 @@ splav spodek spojenec spolu +sponzor spornost spousta -spratek sprcha spustit sranda @@ -1633,19 +1628,20 @@ subjekt subtropy suchar sudost -suknice +sukno sundat sunout +surikata surovina -sutana svah svalstvo +svetr svatba svazek -svetr svisle svitek svoboda +svodidlo svorka svrab sykavka @@ -1670,9 +1666,8 @@ tamhle tampon tancovat tanec -tankista +tanker tapeta -tatarka tavenina tazatel technika @@ -1687,8 +1682,10 @@ teplota tepna teprve terapie +termoska textil ticho +tiskopis titulek tkadlec tkanina @@ -1707,6 +1704,7 @@ tradice traktor tramp trasa +traverza trefit trest trezor @@ -1728,7 +1726,6 @@ tudy tuhnout tuhost tundra -tunika turista turnaj tuzemsko @@ -1760,6 +1757,7 @@ ulice ulita ulovit umyvadlo +unavit uniforma uniknout upadnout @@ -1800,7 +1798,6 @@ vanilka varan varhany varovat -vatra vcelku vchod vdova @@ -1813,11 +1810,12 @@ velitel velmoc velryba venkov +veranda verze veselka veskrze vesnice -vespod +vespodu vesta veterina veverka @@ -1829,7 +1827,7 @@ vidle vila vinice viset -vitamin +vitalita vize vizitka vjezd @@ -1842,11 +1840,11 @@ vlevo vlhkost vliv vlnovka -vloni vloupat vnucovat vnuk voda +vodivost vodoznak vodstvo vojensky @@ -1903,6 +1901,7 @@ vypravit vypustit vyrazit vyrovnat +vyrvat vyslovit vysoko vystavit @@ -1979,7 +1978,6 @@ zdatnost zdivo zdobit zdroj -zdrtit zdvih zdymadlo zelenina @@ -1988,6 +1986,7 @@ zemina zeptat zezadu zezdola +zhatit zhltnout zhluboka zhotovit @@ -2013,6 +2012,7 @@ zmizet zmocnit zmodrat zmrzlina +zmutovat znak znalost znamenat @@ -2034,7 +2034,7 @@ zrnitost zrno zrovna zrychlit -zrzek +zrzavost zticha ztratit zubovina -- cgit v1.2.3 From 5ad92f06de1036b650bf0bd40c7f89eb5059cb2d Mon Sep 17 00:00:00 2001 From: Jeff Garzik Date: Wed, 8 Mar 2017 11:46:12 -0500 Subject: Add BIP 100, as currently implemented. --- bip-0100.mediawiki | 66 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 66 insertions(+) create mode 100644 bip-0100.mediawiki diff --git a/bip-0100.mediawiki b/bip-0100.mediawiki new file mode 100644 index 0000000..67c2b4c --- /dev/null +++ b/bip-0100.mediawiki @@ -0,0 +1,66 @@ +
+  BIP: 100
+  Title: Dynamic maximum block size by miner vote
+  Author: Jeff Garzik 
+          Tom Harding 
+          Dagur Valberg Johannsson 
+  Status: Draft
+  Type: Standards Track
+  Created: 2015-06-11
+  License: BSD-2-Clause
+  Comments-Summary: No comments yet.
+  Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0100
+
+ +==Abstract== + +Replace the static 1M block size hard limit with a hard limit set by coinbase vote, conducted on the same schedule as difficulty retargeting. + +==Motivation== + +Miners directly feel the effects, both positive and negative, of any maximum block size change imposed by their peers. Larger blocks allow more growth in the on-chain ecosystem, while smaller blocks reduce resource requirements network-wide. Miners also act as an efficient proxy for the rest of the ecosystem, since they are paid in the tokens collected for the blocks they create. + +A simple deterministic system is specified, whereby a 75% mining supermajority may activate a change to the maximum block size each 2016 blocks. Each change is limited to a 5% increase from the previous block size hard limit, or a decrease of similar magnitude. Among adopting nodes, there will be no disagreement on the evolution of the maximum block size. + +The system is compatible with emergent consensus, but whereas under that system a miner may choose to accept any size block, a miner following BIP100 observes the 75% supermajority rule, and the 5% change limit rule. Excessive-block values signaled by emergent consensus blocks are considered in the calculation of the BIP100 block size hard limit, and the BIP100 calculated maximum block size is signaled as an excessive-block value for the benefit of all observers. + +==Specification== + +===Dynamic Maximum Block Size=== +# Initial value of hardLimit is 1000000 bytes, preserving current system. +# Changing hardLimit is accomplished by encoding a proposed value, a vote, within a block's coinbase scriptSig, and by processing the votes contained in the previous retargeting period.

+## Vote encoding +### A vote is represented as a positive megabyte value using the BIP100 pattern

/BIP100/B[0-9]+/

Example: /BIP100/B8/ is a vote for a 8000000-byte hardLimit.

+### If the block height is encoded at the start of the coinbase scriptSig, as per BIP34, it is ignored. +### Only the first BIP100 pattern match is processed in "Maximum block size recalculation" below. +### A valid megabyte value is represented by consecutive base-ten digits. +### If no BIP100 pattern is matched, the first matching emergent consensus pattern /EB[0-9]+/, if any, is accepted as the megabyte vote.

+## Maximum block size recalculation +### A new hardLimit is calculated after each difficulty adjustment period of 2016 blocks, and applies to the next 2016 blocks. +### Absent/invalid votes are counted as votes for the current hardLimit. +### The votes of the previous 2016 blocks are sorted by megabyte vote. +### Raising hardLimit

+#### The raise value is defined as the vote of the 1512th highest block, converted to bytes. +#### If the resultant raise value is greater than (current hardLimit * 1.05) rounded down to the nearest byte, it is set to that value. +#### If the resultant raise value is greater than current hardLimit, the raise value becomes the new hardLimit and the recalculation is complete.

+### Lowering hardlimit

+#### The lower value is defined as the vote of the 1512th lowest block, converted to bytes. +#### If the resultant lower value is less than (current hardLimit / 1.05) rounded down to the nearest byte, it is set to that value. +#### If the resultant lower value is less than current hardLimit, the lower value becomes the new hardLimit and the recalculation is complete.

+### Otherwise, new hardLimit remains the same as current hardLimit. + +===Signature Hashing Operations Limits=== +# The per-block signature hashing operations limit is scaled to (hardLimit rounded up to nearest megabyte)/50. +# A maximum serialized transaction size of 1000000 bytes is imposed. + +===Publication of hardLimit=== +# For the benefit of emergent consensus nodes, hardLimit is published. Example: a complete coinbase string might read

/BIP100/B8/EB2.123456/

which indicates a vote for 8M maximum block size, and an enforced hardLimit of 2.123456 megabytes for the block containing the coinbase string. + +==Deployment== + +This BIP is presumed deployed and activated as of block height 449568 by implementing nodes on the bitcoin mainnet. It has no effect until a raise value different from 1M is observed, which requires at least 1512 of 2016 blocks to vote differently from 1M. + +==Backward compatibility== + +The first block larger than 1M will create a network partition, as nodes with a fixed 1M hard limit reject that block. + -- cgit v1.2.3 From ac9c8b73ed7ee7d5b2055bf926b1da5e76025c49 Mon Sep 17 00:00:00 2001 From: Tom Harding Date: Thu, 4 May 2017 18:47:32 -0400 Subject: bip100: Improve header metadata. Vote search edge cases. jgarzik/bip100.git commits: b82d840733b09104fca3f8805ffa6f2459193201 4ed2a0e4523b06d97e32259e6e180734a794e24f --- bip-0100.mediawiki | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/bip-0100.mediawiki b/bip-0100.mediawiki index 67c2b4c..fdb62a4 100644 --- a/bip-0100.mediawiki +++ b/bip-0100.mediawiki @@ -1,5 +1,6 @@
   BIP: 100
+  Layer: Consensus (hard fork)
   Title: Dynamic maximum block size by miner vote
   Author: Jeff Garzik 
           Tom Harding 
@@ -8,8 +9,6 @@
   Type: Standards Track
   Created: 2015-06-11
   License: BSD-2-Clause
-  Comments-Summary: No comments yet.
-  Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0100
 
==Abstract== @@ -30,14 +29,14 @@ The system is compatible with emergent consensus, but whereas under that system # Initial value of hardLimit is 1000000 bytes, preserving current system. # Changing hardLimit is accomplished by encoding a proposed value, a vote, within a block's coinbase scriptSig, and by processing the votes contained in the previous retargeting period.

## Vote encoding -### A vote is represented as a positive megabyte value using the BIP100 pattern

/BIP100/B[0-9]+/

Example: /BIP100/B8/ is a vote for a 8000000-byte hardLimit.

+### A vote is represented as a megabyte value using the BIP100 pattern

/BIP100/B[0-9]+/

Example: /BIP100/B8/ is a vote for a 8000000-byte hardLimit.

### If the block height is encoded at the start of the coinbase scriptSig, as per BIP34, it is ignored. ### Only the first BIP100 pattern match is processed in "Maximum block size recalculation" below. -### A valid megabyte value is represented by consecutive base-ten digits. +### A megabyte value is represented by consecutive base-ten digits. ### If no BIP100 pattern is matched, the first matching emergent consensus pattern /EB[0-9]+/, if any, is accepted as the megabyte vote.

## Maximum block size recalculation ### A new hardLimit is calculated after each difficulty adjustment period of 2016 blocks, and applies to the next 2016 blocks. -### Absent/invalid votes are counted as votes for the current hardLimit. +### Absent/zero-valued votes are counted as votes for the current hardLimit. ### The votes of the previous 2016 blocks are sorted by megabyte vote. ### Raising hardLimit

#### The raise value is defined as the vote of the 1512th highest block, converted to bytes. @@ -50,7 +49,7 @@ The system is compatible with emergent consensus, but whereas under that system ### Otherwise, new hardLimit remains the same as current hardLimit. ===Signature Hashing Operations Limits=== -# The per-block signature hashing operations limit is scaled to (hardLimit rounded up to nearest megabyte)/50. +# The per-block signature hashing operations limit is scaled to (actual block size rounded up to nearest megabyte)/50. # A maximum serialized transaction size of 1000000 bytes is imposed. ===Publication of hardLimit=== @@ -64,3 +63,9 @@ This BIP is presumed deployed and activated as of block height 449568 by impleme The first block larger than 1M will create a network partition, as nodes with a fixed 1M hard limit reject that block. +==Reference Implementation== +https://github.com/bitcoinxt/bitcoinxt/pull/188 + +==Copyright== +This document is licensed under the BSD 2-clause license. + -- cgit v1.2.3 From 7ea5bfccde202b523738a26278afc746fdbf5aa4 Mon Sep 17 00:00:00 2001 From: Tom Harding Date: Fri, 5 May 2017 07:41:57 -0400 Subject: bip100: restore Comments header --- bip-0100.mediawiki | 1 + 1 file changed, 1 insertion(+) diff --git a/bip-0100.mediawiki b/bip-0100.mediawiki index fdb62a4..ceb24ad 100644 --- a/bip-0100.mediawiki +++ b/bip-0100.mediawiki @@ -5,6 +5,7 @@ Author: Jeff Garzik Tom Harding Dagur Valberg Johannsson + Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0100 Status: Draft Type: Standards Track Created: 2015-06-11 -- cgit v1.2.3 From 7c23bf14ee14beb8339b26914475cb50c95465e8 Mon Sep 17 00:00:00 2001 From: Tom Harding Date: Sat, 6 May 2017 12:29:54 -0400 Subject: bip100: Clarifications --- README.mediawiki | 7 +++++++ bip-0100.mediawiki | 19 ++++++++++++------- 2 files changed, 19 insertions(+), 7 deletions(-) diff --git a/README.mediawiki b/README.mediawiki index 1398ca3..aa6c50e 100644 --- a/README.mediawiki +++ b/README.mediawiki @@ -413,6 +413,13 @@ Those proposing changes should consider that ultimately consent may rest with th | Jorge TimΓ³n | Informational | Draft +|- +| [[bip-0100.mediawiki|100]] +| Consensus (hard fork) +| Dynamic block size by miner vote +| Jeff Garzik, Tom Harding, Dagur Valberg +| Standard +| Draft |- style="background-color: #ffcfcf" | [[bip-0101.mediawiki|101]] | Consensus (hard fork) diff --git a/bip-0100.mediawiki b/bip-0100.mediawiki index ceb24ad..6b322d0 100644 --- a/bip-0100.mediawiki +++ b/bip-0100.mediawiki @@ -5,6 +5,7 @@ Author: Jeff Garzik Tom Harding Dagur Valberg Johannsson + Comments-Summary: No comments yet. Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0100 Status: Draft Type: Standards Track @@ -41,20 +42,22 @@ The system is compatible with emergent consensus, but whereas under that system ### The votes of the previous 2016 blocks are sorted by megabyte vote. ### Raising hardLimit

#### The raise value is defined as the vote of the 1512th highest block, converted to bytes. -#### If the resultant raise value is greater than (current hardLimit * 1.05) rounded down to the nearest byte, it is set to that value. +#### If the resultant raise value is greater than (current hardLimit * 1.05) rounded down, it is set to that value. #### If the resultant raise value is greater than current hardLimit, the raise value becomes the new hardLimit and the recalculation is complete.

-### Lowering hardlimit

+### Lowering hardLimit

#### The lower value is defined as the vote of the 1512th lowest block, converted to bytes. -#### If the resultant lower value is less than (current hardLimit / 1.05) rounded down to the nearest byte, it is set to that value. +#### If the resultant lower value is less than (current hardLimit / 1.05) rounded down, it is set to that value. #### If the resultant lower value is less than current hardLimit, the lower value becomes the new hardLimit and the recalculation is complete.

### Otherwise, new hardLimit remains the same as current hardLimit. ===Signature Hashing Operations Limits=== -# The per-block signature hashing operations limit is scaled to (actual block size rounded up to nearest megabyte)/50. +# The per-block signature hashing operations limit is scaled to (actual block size, fractional megabyte rounded to next higher megabyte) / 50. # A maximum serialized transaction size of 1000000 bytes is imposed. +==Recommendations== + ===Publication of hardLimit=== -# For the benefit of emergent consensus nodes, hardLimit is published. Example: a complete coinbase string might read

/BIP100/B8/EB2.123456/

which indicates a vote for 8M maximum block size, and an enforced hardLimit of 2.123456 megabytes for the block containing the coinbase string. +# For the benefit of all observers, it is recommended that hardLimit be published. Example: a complete coinbase string might read

/BIP100/B8/EB2.123456/

which indicates a vote for 8M maximum block size, and an enforced hardLimit of 2.123456 megabytes for the block containing the coinbase string. ==Deployment== @@ -64,8 +67,10 @@ This BIP is presumed deployed and activated as of block height 449568 by impleme The first block larger than 1M will create a network partition, as nodes with a fixed 1M hard limit reject that block. -==Reference Implementation== -https://github.com/bitcoinxt/bitcoinxt/pull/188 +==Implementations== +https://github.com/bitcoinxt/bitcoinxt/pull/188
+https://github.com/bitcoinxt/bitcoin/pull/1
+https://github.com/BitcoinUnlimited/BitcoinUnlimited/pull/398
==Copyright== This document is licensed under the BSD 2-clause license. -- cgit v1.2.3 From 0a1d35877a993b460c14358324cf545a53d852ff Mon Sep 17 00:00:00 2001 From: Paul Sztorc Date: Sat, 3 Feb 2018 21:30:10 -0500 Subject: Create images.txt --- bip-hashrate-escrows/images.txt | 1 + 1 file changed, 1 insertion(+) create mode 100644 bip-hashrate-escrows/images.txt diff --git a/bip-hashrate-escrows/images.txt b/bip-hashrate-escrows/images.txt new file mode 100644 index 0000000..ece458d --- /dev/null +++ b/bip-hashrate-escrows/images.txt @@ -0,0 +1 @@ +Images used in this bip. -- cgit v1.2.3 From db86b890f92015ba50d6b1b49ba742a4b28663fc Mon Sep 17 00:00:00 2001 From: Paul Sztorc Date: Sat, 3 Feb 2018 21:30:22 -0500 Subject: Add files via upload --- bip-hashrate-escrows/two-groups.png | Bin 0 -> 39695 bytes 1 file changed, 0 insertions(+), 0 deletions(-) create mode 100644 bip-hashrate-escrows/two-groups.png diff --git a/bip-hashrate-escrows/two-groups.png b/bip-hashrate-escrows/two-groups.png new file mode 100644 index 0000000..c8a3ffa Binary files /dev/null and b/bip-hashrate-escrows/two-groups.png differ -- cgit v1.2.3 From 2580d6522831bd8d3967515bee4429fd6a6351a6 Mon Sep 17 00:00:00 2001 From: Paul Sztorc Date: Sat, 3 Feb 2018 21:32:48 -0500 Subject: Add files via upload --- bip-hashrate-escrows.md | 408 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 408 insertions(+) create mode 100644 bip-hashrate-escrows.md diff --git a/bip-hashrate-escrows.md b/bip-hashrate-escrows.md new file mode 100644 index 0000000..3b5d61e --- /dev/null +++ b/bip-hashrate-escrows.md @@ -0,0 +1,408 @@ + +Header +======= + + BIP: ???? + Layer: Consensus (soft fork) + Title: Hashrate Escrows (Consensus layer) + Author: Paul Sztorc + CryptAxe + Comments-Summary: No comments yet. + Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-??????? + Status: Draft + Type: Standards Track + Created: 2017-08-14 + License: BSD-2-Clause + Post-History: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-May/014364.html + + +Abstract +========== + +A "Hashrate Escrow" is a clearer term for the concept of "locked to an SPV Proof", which is itself a restatement of the phrase "within a sidechain" as described in [a famous Oct 2014 paper](https://blockstream.com/sidechains.pdf) written partially by some Blockstream co-founders. + +A Hashrate Escrow resembles a 2-of-3 multisig escrow, where the 3rd party (who will arbitrate any disputes) is a decentralized group of people: the dynamic-membership set of Bitcoin Miners. However, the 3rd party does not sign escrow-withdrawal transactions with a private key. Instead, these are "signed" by directing hashpower over them for a period of time. + +This project has [a website](http://www.drivechain.info/) which includes [a FAQ](http://www.drivechain.info/faq/index.html). + + +Motivation +============ + +In practice these escrows are likely to be "asymmetric sidechains" of Bitcoin (such as [Rootstock](http://www.rsk.co/)) or "virtual chains" within Bitcoin (such as [proposed by Blockstack](https://github.com/blockstack/virtualchain) in mid-2016). + +Sidechains have many potential benefits, including: + +1. Protect Bitcoin from competition from altcoins and spinoffs. Safely allow competing implementations (of *sidechains*). +2. Protect Bitcoin from hard fork campaigns. (Such campaigns represent an existential threat to Bitcoin, as well as an avenue for developer corruption.) +3. Help with review, by making it much easier for reviewers to ignore bad ideas. +4. Provide an avenue for good-but-confusing ideas to prove their value safely. + + + +Specification +============== + + +#### Components + +Hashrate Escrows are built of two types of component: [1] new databases, and [2] new message-interpretations. + +##### 1. New Databases + +* D1. "Escrow_DB" -- a database of "accounts" and their attributes. +* D2. "Withdrawal_DB" -- a database of pending withdrawals from these accounts, and their statuses. + +Please note that these structures (D1 and D2) will not literally exist anywhere in the blockchain. Instead they are constructed from messages...these messages, in contrast, *will* exist in the blockchain (with the exception of M4). + +##### 2. New Messages + +* M1. "Propose New Escrow" +* M2. "ACK Escrow Proposal" +* M3. "Propose Withdrawal" +* M4. (implied) "ACK Withdrawal" +* M5. "Execute Deposit" -- a transfer of BTC from-main-to-side +* M6. "Execute Withdrawal" -- a transfer of BTC from-side-to-main + + +#### On the Resource Requirements of New Databases + +The "new" databases are simply reinterpretations of data that are already contained elsewhere in the blockchain. Specifically, M1 M2 and M3 are all located in the block's coinbase txn, and M5 and M6 might be found in any regular txn. M4 is a special case and does not actually need to be included anywhere, so it is not. If you like, you can imagine that the M4s reside in an optional extension block. + +In other words, we just rearrange what is already there. Because of this, even though "new databases" are created and stored in memory, the existing bandwidth and storage limits are respected (although, see "M4" below). + + + + +### Adding Sidechains and Tracking Them (D1, M1, M2) + +#### D1 -- "Escrow_DB" + +The table below enumerates the new database fields, their size in bytes, and their purpose. In general, an escrow designer (for example, a sidechain-designer), is free to choose any value for these. + +Note: Fields 6 through 9 have been intentionally removed. Previously, this section allowed miners to set and commit to voting/waiting periods. However, I have since standardized the periods: withdrawals expire after 6 months (26298 blocks), and they succeed if they ever achieve an ACK score of 13140 or higher. I have removed the waiting period, because anyone who adopts a policy of ignoring all withdrawals with fewer than 400 ACKs will automatically gain all of the benefits of the waiting period. The justification for this change is that it strongly implies that an attack on any one sidechain is an attack on all of them (in a sense, this change makes the "victimhood" of each sidechain "fungible"). + +Field No. | Label | Bytes | Type | Description / Purpose +----------|-------|------|------|-------- +1 | Escrow Number | 1 | uInt | A number assigned to the entire escrow. Used to make it easy to refer to each escrow. +2 | Active\* | 2 | uInt | This counter starts at zero when a new entry is added, and increments as a new entry is supported by "M2". The entry either reaches the max value of 4032 (at which point the counter freezes), or else the entry is removed. This enforces the requirement "a soft fork is required to add a new escrow". +3 | Escrow Name/Description | 120 | string | A human-readable name and description of the sidechain. More than enough space to hold a 32 byte hash. Helps prevent destructive interference among sidechains (see below). +4 | Critical Private Key | 32 | hex | This is the Double-SHA256 of the binary release of the reference software. It ties the sidechain to a specific software release, and doubles as a way of signing withdrawals (see below). +5 | Critical Address\* | 32 | string | This is derived by taking f(x) = ImportPrivateKey(HexToBase58Check(x)) of field #3. It is intentionally in the human-readable format {{although it could be 25 bytes of binary}}. +10 | "CTIP" -- Part 1 "TxID"\* | 32 | hex | The CTIP, or "Critical (TxID, Index) Pair" is a variable for keeping track of where the escrow's money is (ie, which member of the UTXO set). +11 | "CTIP" -- Part 2 "Index"\* | 4 | hex | Of the CTIP, this is second element of the pair: the Index. See #10 above. + + +\* Denotes a "convenience field", the entry for this field is derived from other fields, or from the blockchain-state itself. The escrow-creator does not need to provide these values in M1 (or anywhere). + +Escrow_DB requires 223 bytes [1+120+32+32+2+32+4] for each escrow in the main blockchain. Of these, 70 bytes [2+32+32+4] are merely for convenience. Therefore, a sidechain is defined (see "M1") by 153 bytes of information. + +#### Notes on D1 + +1. D1 will always exist. +2. D1 will always have a unique sort (first by "Escrow Number" (field #1), and second by "Active" (field #2)). There is only ever one (escrow number, Active) pair. +3. D1 is updated according to M1 and M2 (below). +4. If a new entry is added to D1 with an "Escrow Number" that is already in use, then this entry will either eventually be removed (because it was not supported with an M2), or it will eventually overwrite the old entry (if it *was* supported via M2). + + +#### Notes on D1 + +##### Obligations Placed on Miners + +Miners have always upgraded their software according to criteria that are known only to them (in other words, "whenever they want"). + +However, this soft fork imposes two new criteria upon them. First: miners should only upgrade their software, if any modification to the portfolio of sidechains [that are added/removed in the upgrade] can be expected to increase miner wealth. Trivially, this implies that miners should make sure that the upgrade doesn't overwrite (and destroy) an existing sidechain that they like! But, more seriously, it implies that miners should take an interest in what the sidechain is doing to the mainchain and other sidechains (see below). + +##### Destructive Sidechain Interference + +People frequently emphasize that miners should have "as little control" as possible. It is a very safe claim to make, and a very easy sentence to write. Much harder is to determine exactly what this minimum value is, and how to achieve it. Harder still is to untie the knot of who is actually controlling what, in a decentralized, interacting system. + +Certainly, miners can not have "zero control" -- for that is the same as to just remove them from the system altogether. Some rules are enforced "on miners by nodes" (such as the infamous blocksize limit); other rules are enforced by nodes but are narrowly-controlled by miners (such as the proof-of-work itself, or the block's timestamp). Thirdly, some rules are enforced by both against each other (such as the rule against including invalid txns or double-spent txns), for mutual benefit. + +Some pause should be given, after one considers that the sidechain design goal is literally a piece of software that can do *anything*. Anything includes a great many things, many of which I demonstrate to be undesirable. Bitcoin itself does not allow "anything" -- it allows any person to transact, but, in contrast, it does not permit any person to double-spend. This is because "allowing anyone to do anything" is not viable in a world that contains undesirable interactions (what a libertarian might call "aggression") -- in the case of money, these are theft and counterfeiting. + +I have produced a comprehensive quantity of written material [1], presentations [2], etc [3] on exactly what the level of miner-control should be, and why. Specifically, I claim that **miners should be aware of the purpose of the sidechain, and they should reject sidechains which have an unclear purpose or which have a purpose that will lead to decrease in miner-wealth** (where wealth measured explicitly as: the estimated present value of the purchasing power of the blockchain's coinbase txns). I claim that this criterion is necessary because, just Original Bitcoin filters unwanted interactions among different BTC txns, so too much "Sidechain Bitcoin" filter out unwanted interactions among sidechain. + +* [1] http://www.truthcoin.info/blog/wise-contracts/ +* [2] https://www.youtube.com/watch?v=xGu0o8HH10U&index=1&list=PLw8-6ARlyVciMH79ZyLOpImsMug3LgNc4 +* [3] http://www.drivechain.info/literature/index.html + +Call it a "sidechain non-aggression principle", if you want. + +To the best of my knowledge, everyone who *has* reviewed this information as found the arguments to be acceptable. It has, also, changed a few minds (from "unacceptable" to "acceptable"). + + +##### ISSUE: "Signing" BTC Txns + +Currently, we use a process which may be suboptimal. It is that we *literally sign* a txn with a globally and publicly known private key. But this is for convenience purposes -- the signature that is produced is not doing anything, and is therefore wasteful. Instead we may use OP_TRUE, but this might interfere with how we detect the sidechain's balance. I'm not sure what the best way is. Someone needs to investigate how to do this -- removing OP_CheckSig, etc. This is a TODO for sure, and an opportunity for someone to help. + + + +(The following messages were modeled on SegWit -- https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki#commitment-structure ) + + + +#### M1 -- "Propose New Sidechain" + + 1-byte - OP_RETURN (0x6a) + 1-byte - Push the following 157 bytes (0x9d) + 4-byte - Commitment header (0x53707243) + 153-byte - the critical bytes mentioned above (fields #1, #3, and #4, to populate a new D1 entry) + + +#### New Block Validation Rules + +1. If the network detects a properly-formatted M1, it must add an entry to D1, into the very next block, with the following initial values: +* Field #5 will be calculated as per [version 1 Bitcoin addresses](https://en.bitcoin.it/wiki/Technical_background_of_version_1_Bitcoin_addresses), but with a prefix of "4" instead of "1". +* Field #9 will be derived from #7 and #8 using math. +* The initial values of Fields #10, #11, and #12 are set to zero. +2. Only one M1 (of any kind) can be added into a block at a time. + +#### Notes on M1 + +The escrow will "reuse" the same address over and over. But notice that there is no privacy loss due to address reuse, because the address in question does not "belong" to any specific individual. Furthermore, the transactions in question could already be linked by just watching the Hashrate Escrow for deposits ...this, in fact, is exactly what a sidechain must do in order to make main-to-side transfers. + + +#### M2 -- "ACK Sidechain Proposal" + + 1-byte - OP_RETURN (0x6a) + 1-byte - Push the following 29 bytes (0x1D) + 4-byte - Commitment header (0x53616343) + 25-byte - Commitment hash: RIPMD-160 of Sha256 of a given M1 (above) + +#### New Block Validation Rules + +1. An entry will be removed from D1, unless that block contains an M2 supporting it (this will increment field #2), or the entry has accumulated 4032 M2s. Therefore, each entry needs to have 4032 *consecutive* M2s supporting it, else it will be removed from D1 and must start over. This requires all miners to agree to "soft fork" the sidechain into existence. This makes it easier to curate the list of active sidechains, and prevents destructive interference (see above) among contradictory sidechains. +2. Once the entry has 4032 M2s, then a miner must make a 0 btc payment to the Critical Address, thus activating the chain. This payment becomes the first "critical (TxID, index) pair" (or "CTIP"). +3. It is possible to "overwrite" a D1 slot -- this destroys the sidechain that is there, and replaces it with a new one. This is defined as requiring 6 months (26298 blocks) of M2s, instead of 4 weeks (4032). This possibility does not change the security assumptions, because we already assume that users have access to "magically true" bandwidth at a rate of 1 bit per ACK-cycle. (This is why the voting cycles should be large, 1 month Waiting, 2+ months Voting). + + + +### Adding Withdrawal-Attempts and Tracking Their Success (D2, M3, M4) + +#### D2 -- "Withdrawal_DB" + +The table below enumerates the database fields, their size (in bytes), type and purpose. + + +Field No. | Label | Bytes | Type | Description / Purpose +----------|-------|------|------|----------------------- +1 | Escrow Number | 1 | uInt | Links the withdrawal-request to a specific escrow. +2 | WT^ | 32 | hex | This is a "blinded transaction id" (ie, the double-Sha256 of a txn that has had two fields zeroed out, see M6) of a withdrawal-attempt. +3 | ACKs\* | 2 | uInt | The current total number of "votes", this starts at 0 and remains there throughout the waiting period. +4 | Age\* | 3 | uInt | Total duration of time, in blocks, that this WT^ has been inside of D2. +5 | Waiting Period\* | 2 | uInt | Total duration of time, in blocks, that this entry must sit idle, before it can begin to accumulate ACKs/NACKs. Pulled from D1's field #6. +6 | Max Age\* | 3 | uInt | Determined by summing (D1's field #6) and (D1's field #7). +7 | Threshold\* | 2 | uInt | Total ACKs needed, this is pulled from D1's field #9. +8 | Approved\* | 1 | boolean | True while ACKs > Threshold, False otherwise. + +\* Denotes a "convenience field" (see above). + +Withdrawal_DB requires 46 bytes [1+32+2+3+2+3+2+1] per entry. Of these, 13 bytes ([2+3+2+3+2+1], all fields except #1 and #2) can be generated locally, leaving 33 critical bytes [1+32]. + +#### New Block Validation Rules for D2 + +1. In each block, a hash commitment to D2 must always exist (even if D2 is blank). +2. D2 must always be sorted first by field #1 (Escrow Number) and second by field #4 (Age). This imposes a unique sort. +3. From one block to the next, every entry's "Age" field must increase by exactly 1. +4. From one block to the next, entries are only removed from D2 (in the very next block) if: +* * "Age" = "MaxAge". +* * If the block contains a txn who's blinded txID matches WT^. {{ This might be unnecessary, and a lot of work. }} +5. In addition, there are special rules for the allowed values in the "ACKs" field (field #3). See M4 below. + +#### M3 -- "Propose Withdrawal" + + 1-byte - OP_RETURN (0x6a) + 1-byte - Push the following 37 bytes (0x25) + 4-byte - Commitment header (0xD45AA943) + 33-byte - the critical bytes mentioned above (fields #1 and #2, to populate a new D2 entry) + + +#### New Block Validation Rules for M3 + +1. If the network detects a properly-formatted M3, it must add an entry to D2 in the very next block. The starting values of fields #3 and #4 are zero, and #5 is pulled over by extracting the relevant value from D1. +2. Each block can only contain one M3 per sidechain. + + +#### M4 -- "ACK Withdrawal" + +#### Very Little Info, Probably Calculable in Advance + +M4 is exceptional (in comparison to the other M's) in a few ways. First, its content is not stored anywhere, only the *hash* of its *effect* is stored (in a leaf of a merkle tree who's root is inserted into a mainchain coinbase). M4 alters the contents of D2 -- the *contents* of D2 are consensus critical, but M4 (the process by which nodes reach a new valid D2) can be anything. + +In fact, M4 can also be *nothing*. In other words, it may be optional. This is precisely because, from one block to the next, we have constrained D2 such that it is only allowed to change in a few ways. Therefore, the exhaustive set of "candidate D2s" can be precomputed by full nodes in advance. + +#### Two Withdrawals at Once + +In general, only one withdrawal (per sidechain) can make progress (toward being included in a block) at a time. In other words, as WT^s are proposed, only one can make progress toward the finish line. As a result, a given side-to-main transfer will always take between 3 and 6 months. Instead, with more simultaneous withdrawals, the worst-case transfer duration would improve. + +![dots-image](/bip-hashrate-escrows/two-groups.png?raw=true) + +The worst-case withdrawal time obeys f(n)=3+(3/n) months, where n is the number of simultaneous withdrawals. + +N=2 is the most desirable choice for several reasons. First, it delievers the greatest marginal benefit (of 1.5 months). Later choices only deliver 0.5 and 0.25 marginal months. + +Second, n=2 can be implemented in a clever way: by allowing a withdrawal to freely advance, if and only if has an ACK-score of 6575 or greater, and if it also has the largest ACK score. In other words, the withdrawal that is furthest along can advance (or retreat) for free, if it has already made it at least halfway to the finish line. With this change, our new M4, is either an "abstain" for the sidechain (in which case nothing happens to any ACK scores), or else it will be in one of two cases: old_M4 + "the largest advances", or new_M4 + "the largest retreats". As a result the number of M4 possibilities (of which the next section is concerned) only increases by a factor of two (instead of exponentially). + +It is possible to troll this rule, by getting two (or even three) withdrawals to have 6575+ ACK scores, and then getting them to *tie* for first place. So, if there are any ties, the ability to "bonus move" is disabled until all ties are broken. + +#### How Hard is it to Guess M4? + +If there are n Escrows and m Withdrawals-per-escrow1, then there are (m+2)^n total candidates for the next D2. This is because, [per block per escrow], one of three things can happen: (1) one of the m withdrawal-candidates can be "ACK"ed (or "upvoted" or "promoted"), which automatically downvotes the others; or (2) all withdrawal-candidates can be downvoted, or finally (3) the miners can abstain from voting on the escrow's withdrawals altogether, leaving the tallies the same. + +First, for nodes which validate all sidechains (assuming these escrows are sidechains), this simplifies to 2^n -- these nodes only have to choose between the single honest choice (on one hand) or an abstention (on the other). Second, even for nodes that don't validate any sidechains, the number of candidates might be reduced from m^n to 3^n, by making a simplifying assumption: whichever withdrawal was most recently added/upvoted, is likely to be the one which is upvoted next. + +Of course, that is still O(k^n) for n sidechains, which isn't great2. If the "D2 update" cannot be guessed, it must be transmitted in some way. + +#### Giving Up and Getting M4 the Old Fashioned Way + +Two examples for transmitting it are below: + +"Short Form" (Assumes there are no more than 254 active withdrawal-attempts per account) + + 4-byte - Message identifier (0x????????) + 1-byte - Version of this message + N-byte - N is the total number of active accounts ("sidechains"), each byte specifies the position of the single WT that was "upvoted". A value of 0 indicates "downvote everything", a value of 255 indicates abstention. + +"Long Form" (Makes no assumptions about anything) + + 4-byte - Message identifier (0x????????) + 1-byte - Version of this message + 1-byte - Length (in bytes) of this message; total number of withdrawal attempts; y = ceiling( sum_i(m_i +2)/8 ). Nodes should already know what length to expect, because they know the sequence of M3s and therefore the vector of WT^s. + Y-byte - stream of bits (not bytes), with a 1 indicating the position of the chosen action [downvote all, abstain, upvote1, upvote2, ...] + + +If the message is very very large, then nodes may not want to broadcast it. This opens up an "exhaustion attack"2, in which many miners create bad WT^s, vote on these randomly, and then refuse to broadcast their votes. Fortunately, even for a worst-case scenario of 200 sidechains and 1,000 withdrawal-attempts per sidechain, honest nodes can communicate a long form M4 with each other by using just 25,056 bytes per block [4+1+1+(200\*(1000+1+1)/8)]. + +Today's pre-drivechain miners can already carry out a similar attack, by creating and including txns and then not broadcasting that part of the block to anyone. This is often characterized as a ["block publication incentive"](https://petertodd.org/2016/block-publication-incentives-for-miners), because in that case the prospect of exhaustively computing all possible transactions (to uncover the missing ones) is completely out of the question. + +However, message M4 is different from a withheld-txn, because M4 operates outside of the block's mandated information-processing limits (ie, outside the infamous 1 MB nonwitness blocksize limit). So we should examine the conditions under which M4 grows and shrinks, to ensure that we are not smuggling in a tremendous burden on full nodes. + +Under adversarial conditions, to lengthen a long-form M4 by one bit per block, for C blocks, the attacker must pay 312 bits (39 bytes) one time (to embed a new M3 message). The value C is the length of the sidechain's voting period, which varies but which I expect to be approximately 8,064 (and which could theoretically be as high as 65,536). Thus the attacker can burden nodes disproportionately, if (s)he wishes. + +Fortunately, the attack in question has no motivation (as far as I can tell). If the miner's goal is to trick rivals into mining on top of invalid blocks, he can already do this much more effectively with the unpublished-txn method (above). If instead he is just trying to harass nodes, then nodes may freely "downgrade" to earlier versions of the protocol, and simply ignore all drivechain-related messages. It seems that the attack could best be used in order to: make a large D2, make D2 confusing, sneak in votes for evil WT^ lurking in D2. Thus, the attack disables the transparency of the drivechain system, to some extent. The cost of the attack is forgone transaction fees, due to block space wasted on useless M3s. + +In practice, n is already capped, and miners may impose [on each other] a "soft cap" on m for their mutual protection. Thus, n and m might never get above 10 and 30, respectfully. In this case, the [Short Form, this time] M4 can never require more than 15 bytes per block, no matter what the attacker tries. + +In practice, m should always be 1 or 2, else something fishy is going on; and m can only inch up by 1 unit per block. So the system as a whole is still quite transparent, in that users are warned appropriately and well in advance. Attackers must invest upfront and they face an uphill climb, in order to eventually make things more expensive for a few others; defenders can wait-and-see if the attack looks like it will ever amount to anything before lifting a finger. + + +##### New Block Validation Rules (for D2 and, by implication, M4) + +From one block to the next, D2 can only be edited in a few strict ways: + +* Entries can only be added/removed from D2 if they meet the criteria above (in M3, and implicitly M1 and M2). +* The ACK-counter of any individual entry can only change by (-1,0,+1) relative to its previous entry. +* Within a sidechain group, upvoting one withdrawal (ACK=ACK+1) requires you to downvote all other withdrawals in that group. However, the minimum ACK value is zero (and, therefore, downvotes cannot reduce it below zero). + +##### Footnotes for M4 + +1 This represents the worst-case scenario is one where all the Withdrawals are spread evenly over each Sidechain. Under normal operations, there is no reason to expect the all sidechains will have the same number of withdrawals at any given time. In fact, under normal operations, the very *concept* of counting the withdrawals-per-sidechain should be a purposeless one, because there should only be *one* withdrawal at a time. Nonetheless we consider the worst case scenario here. + +2 Guessing becomes more computationally intensive in a highly adversarial situation where the "limited range" is intentionally expanded. In such a scenario, [a] there are many sidechains, and [b] miners voluntarily sacrifice their scarce block-space by creating a high number of (mutually-exclusive, and hence ultimately invalid) withdrawal attempts and putting these into coinbase transactions; and then agree to all [c] vote on these randomly (guaranteeing that all withdrawals fail, including any true withdrawals) and [d] successfully withhold their random voting strategies from nodes (even including spy-miner-nodes). Under this bizarre scenario, nodes may require computing resources which increase near-exponentially with the number of withdrawals, and it may take a long time for an ignorant node to exhaustively work out the underlying state of Withdrawal_DB. In this case, nodes may decide to temporarily stop validating such transactions (as if they had not yet upgraded to support this soft fork). + + + +### Depositing and Withdrawing (M5, M6) + + +Both M5 and M6 are regular Bitcoin txns. They are identified by meeting an important criteria: they select a one of the Critical TxID-index Pairs (a "CTIP") as one of their inputs. Deposits ("M5") are distinguished from withdrawals ("M6") by simply checking to see if money is "going in", or "out". In other words, we compare the BTC value of the original CTIP to that of new CTIP. If original <= new it is a deposit, if original > new then it is a withdrawal. + +The code that identifies sidechain withdrawal / deposit txns (by calculating how much value is being put into or taken out of a sidechain) can be seen here: https://github.com/drivechain-project/bitcoin/blob/mainchainBMM/src/validation.cpp#L351-L386 + +Such txns are forced (by consensus) to obey two additional criteria: + +1. They must contain an output paying "to" the Critical Address [probably in TxOut0]. +2. They must be accompanied by an update to this sidechain's Critical TxID-index Pair (CTIP). The new CTIP must be "this" txn itself. + +These criteria are enforced here https://github.com/drivechain-project/bitcoin/blob/mainchainBMM/src/validation.cpp#L440-L473 by checking that a deposit is paying back to the sidechain more than it is taking out, and completely rejecting any withdrawal from the mempool. And here https://github.com/drivechain-project/bitcoin/blob/mainchainBMM/src/validation.cpp#L1747-L1757 we allow for a withdrawal only once it has attained sufficient work score (ACKs). + +The purpose of this is to have all of the escrow's money (ie all of the sidechain's money) in one TxID, so that depositors immediately undo any UTXO bloat they may cause. This simplifies the withdrawal process, as there is no need to worry about cleaning up "dust deposits" (...and such cleaning can often result in headaches, for example where a withdrawal-txn is larger than 1MB in size, or else may only withdraw an arbitrarily limited amount of BTC). Notice that, unless we assume that an account will last forever, all utxos which are deposited must eventually be withdrawn by someone. Therefore, the relevant design criterion is not "efficiency" (total network cost) but rather "who should pay" (allocation of costs). + +#### M5. "Make a Deposit" -- a transfer of BTC from-main-to-side + +As far as mainchain consensus is concerned, there are no additional requirements. + +However, in practice there *are* additional mainchain requirements...specified by the escrow account, (ie specified by the "sidechain" or "virtual chain"). These requirements are not part of mainchain consensus and are allowed to be anything. In other words, the sidechain is free to invent any way to credit depositor's money -- M5 is fully customizable. + +One method, is for mainchain depositors to append a zero-value OP Return to a Deposit txn, so that the sidechain knows how to credit funds. Mainchain users must upgrade their wallet software, of course, (on an individual basis) in order to become aware of and take advantage of new deposit-methods. + +##### Inconvenient Race Condition + +The requirement that each hashrate escrow be linked to a single TxID does create an interesting inconvenience for depositors. If a user is slow to sign a txn after constructing it (perhaps because the user employs an air-gapped computer, etc), then the signed txn may no longer be valid. This is because the input it selects, may no longer be the Critical TxID (as "the" Critical TxID changes with each deposit). **Only one user can deposit at a time** (although many can deposit per block). As a result, the transaction must fail, and the user would need to be prompted to remake and resign the txn. If this is problem is too frustrating, users can always make main-to-side transfers using atomic cross chain swaps (or, the LN, if they already have a channel open on both chains). + +Fortunately, it is already a part of mainchain consensus that no two txns can spend the same TxID. The only new issue here is the confusion it might create for the user (hence the need for error messages and alternative deposit-methods). + + +#### M6. "Execute Withdrawal" -- a transfer of BTC from-side-to-main + +We come, finally, to the critical matter: where users can take their money *out* of the escrow account, and return it to the "regular" UTXO set. As previously mentioned, this txn is one which (a) spends from a CTIP and (b) reduces the quantity of BTC in an account's CTIP. Most of the work has already been done by D1, M3, M4, and D2. Furthermore, existing Bitcoin tx-rules prevent the sidechain from ever withdrawing more money than has been placed into it. + +From there, we merely introduce two final concepts: + +1. In each block, an entry in D2 is considered an "approved candidate" if the "ACKs" value is above 13140. +2. A "blinded TxID" is way of hashing the txn, in which we first overwrite some parts of the txn with zeros. Specifically, the first 36 bytes of "TxIn0" (the first input, including TxOutHash and TxOutIndex), as well as the first 8 bytes of "TxOut0" (the first output). + +Blinding is necessary because we allow each sidechain only one UTXO at a time. + +of our restriction of the account to a single UTXO-member. Because of this, during the ACKing process the withdrawal-txn (which is currently being ACKed) may change in two ways: the CTIP (which changes with each deposit), and the total quantity of BTC stored in the account (which arbitrarily increases with each new deposit). In other words, a withdrawal-attempt is created via M3, but this takes place many blocks before the withdrawal is actually included via M6. During this time, a single new deposit to the account would change its CTIP and its value. So, what do we ACK? Well, we ACK a "blinded" version of the withdrawal. This blinded version is stable because the dynamic parts are always overwritten with zeros. + +While we ACK a blinded WT^, what is actually included in the blockchain ("M6") is an unblinded WT^. Since each blinded WT^ could correspond to many different unblinded WT^s, we need to impose further restrictions on those unblinded WT^s that are finally included. First, we will force the final unblinded WT^ to spend the entire sidechain balance (by forcing sum(input_values) to equal sum(output_values)). To avoid withdrawing the entire sidechain balance with every withdrawal, we will, secondly, force the unblinded WT^ to create a new output which is itself a deposit to the sidechain it withdrew from (which nodes can check using D1's CTIP field). Unfortunately, these requirements eliminate the possibility of including a transaction fee, as traditionally calculated. So, finally, to compensate for *that*, txn fees are encoded explicitly as a withdrawal to OP_TRUE (which the main:block's miner can immediately claim). + +With all of this in place, the only requirements for inclusion in a block are these: + +1. "Be ACKed" -- The "blinded TxID" of this txn must be member of the "approved candidate" set in the D2 of this block. +2. "Return Change to Account" -- TxOut0 must pay to the "critical account" (see D1) that corresponds to the CTIP that was selected as a TxIn. +3. "Return *all* Change to Account" -- Sum of inputs must equal the sum of outputs. No traditional tx fee is possible. + +Finally, don't forget that M6 inherits the requirement (common to both M5 and M6) that the CTIP be selected as an input, and that the CTIP then be updated. In this case, we know that the critical index will be zero, so the new CTIP will be ("this TxID" (NOT blinded), 0). The TxID is NOT blinded because blinding is only for accumulating ACKs. + +As a result of these requirements, every single withdrawal-attempt will fail, unless an entry has been added to D2 and "ACKed" a sufficient number of times. + + + +Backward compatibility +======================== + +As a soft fork, older software will continue to operate without modification. Non-upgraded nodes will see a number of phenomena that they don't understand -- coinbase txns with non-txn data, value accumulating in anyone-can-spend UTXOs for months at a time, and then random amounts leaving the UTXO in single, infrequent bursts. However, this phenomena doesn't affect them or the validity of the money that they receive. + +( As a nice bonus, note that the sidechains themselves inherit a resistance to hard forks. The only way to guarantee that the WT^s reported by different clients will continue to match identically, is to upgrade sidechains via soft forks of themselves. ) + + +Deployment +=========== + +This BIP will be deployed by "version bits" BIP9 with the name "hrescrow" and using bit 4. + +``` +// Deployment of Drivechains (BIPX, BIPY) +consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].bit = 4; +consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].nStartTime = 1515974401; // January 15th, 2018. +consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].nTimeout = 1547510401; // January 15th, 2019. +``` + +Reference Implementation +========================== + +See: https://github.com/drivechain-project/bitcoin/tree/mainchainBMM + +Also, for interest, see an example sidechain here: https://github.com/drivechain-project/bitcoin/tree/sidechainBMM + + +References +============ + +See http://www.drivechain.info/literature/index.html + + +Credits +========= + +Thanks to everyone who contributed to the discussion, especially: ZmnSCPxj, Adam Back, Peter Todd, Dan Anderson, Sergio Demian Lerner, Chris Stewart, Matt Corallo, Sjors Provoost, Tier Nolan, Erik Aronesty, Jason Dreyzehner, Joe Miyamoto, Ben Goldhaber. + + + +Copyright +========== + +This BIP is licensed under the BSD 2-clause license. -- cgit v1.2.3 From da2e5745f3dcf801e9ab50939791d6cecf434c11 Mon Sep 17 00:00:00 2001 From: Paul Sztorc Date: Sat, 3 Feb 2018 21:36:40 -0500 Subject: Add files via upload --- bip-blind-merged-mining.md | 329 ++++++++++++++++++++++++ bip-blind-merged-mining/bmm-dots-examples.png | Bin 0 -> 41116 bytes bip-blind-merged-mining/images.txt | 1 + bip-blind-merged-mining/witness-vs-critical.png | Bin 0 -> 67570 bytes 4 files changed, 330 insertions(+) create mode 100644 bip-blind-merged-mining.md create mode 100644 bip-blind-merged-mining/bmm-dots-examples.png create mode 100644 bip-blind-merged-mining/images.txt create mode 100644 bip-blind-merged-mining/witness-vs-critical.png diff --git a/bip-blind-merged-mining.md b/bip-blind-merged-mining.md new file mode 100644 index 0000000..ee81f1f --- /dev/null +++ b/bip-blind-merged-mining.md @@ -0,0 +1,329 @@ + Drivechain Documentation -- Blind Merged Mining BIP + Paul Sztorc + November 17, 2017 + Document 3 of 3 + v4.1 + + +Header +======= + + BIP: ???? + Layer: Consensus (soft fork) + Title: Blind Merged Mining (Consensus layer) + Author: Paul Sztorc + CryptAxe + Chris Stewart + Comments-Summary: No comments yet. + Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-??????? + Status: Draft + Type: Standards Track + Created: 2017-10-24 + License: BSD-2-Clause + + +Abstract +========== + +Blind Merged Mining (BMM) is a way of mining special extension blocks, ie "sidechains". It produces strong guarantees that the block is valid, for *any* arbitrary set of rules; and yet it does so without requiring miners to actually do any validation on the block whatsoever. + +BMM actually is a process that spans two or more chains. For an explanation of the "whole picture", please see [this post](http://www.truthcoin.info/blog/blind-merged-mining/). Here we focus on the modifications to mainchain Bitcoin. + +To support BMM, the mainchain is asked to accomplish two goals: +1. Track a set of ordered hashes (the merged-mining). +2. Allow miners to "sell" the act of finding a sidechain block (through the use of a new extended serialization transaction type). + +These goals are accomplished by forcing nodes to validate two new messages (M7, M8), and track data in one new database (D3). + + +Motivation +============ + +Regular "Merged-Mining" (MM) allows miners to reuse their hashing work to secure other chains (for example, as in Namecoin). However, traditional MM has two drawbacks: + +1. Miners must run a full node of the other chain. (This is because [while miners can effortlessly create the block] miners will not create a valid payment to themselves, unless the block that they MM is a valid one. Therefore, miners must assemble a *valid* block first, then MM it.) +2. Miners are paid on the other chain, not on the regular BTC mainchain. For example, miners who MM Namecoin will earn NMC (and they will need to sell the NMC for BTC, before selling the BTC in order to pay for electricity). + +Blind Merged-Mining (BMM) attempts to address those shortcomings. + + +Specification +============ + +Note: This document uses the notation side:\* and main:\* in front of otherwise-ambiguous words (such as "block", "node", or "chain"), to distinguish the mainchain version from its sidechain counterpart. + +As stated above, we have two goals: [1] create and monitor an alt-chain (defined only by a deterministic list of hashes), and [2] allow miners to "sell" the act of finding a sidechain block (through the use of a new extended serialization transaction type). + +### Sidechain Critical Data ("Sidechain Mini-Header") + +Specifically, per side:block per side:chain, we track the following 35 bytes of information: + + 1-byte - ChainIndex (known as "Account Number" in hashrate-escrows.md , or as "Sidechain Number") + 32-bytes - sideHeaderHash (also known as "h*" / hashCritical, the hash of the sidechain block) + 2-bytes - prevBlockRef (an index which points to this side:block's parent side:block) + +The **ChainIndex** indicates which sidechain this critical data is relevant to. As we may eventually have more than one sidechain, this serves as an identifier similar to the Bitcoin network's magic bytes (0xF9BEB4D9). Drivechains however only need to use 1 byte for the identifier (there is a hard limit of 256 sidechains identified as 0-255). The **sideHeaderHash** is the hash of a side:block which will receive PoW via BMM. It serves a similar function to Bitcoin's "hashMerkleRoot", in that it contains the data for its blockchain. The **prevBlockRef** forms the set of headers into a blockchain structure by making each headers refer to one parent header. It is most similar to Bitcoin's hashPrevBlock. + +Where does this data come from, and how does it get around? + +#### Creating / Broadcasting This Data + +##### Creation + +By the time Blind Merged Mining can take place, the ChainIndex is globally known (it is the "Account Number" in D1 [see previous BIP], and "nSidechain" in the code). Each sidechain, when activated by soft fork, will take one of the 0-255 available indexes. + +The other two items, sideHeaderHash and prevBlockRef, are created by sidechain nodes. sideHeaderHash is quite straightforward -- side:nodes build side:blocks, and take the hash of these. + +The final item, prevBlockRef, is a little more complicated. It is an integer that counts the number of "skips" one must take in the side:chain in order to find the current side:block's parent block. In practice, this value will usually be zero. It will only be a value other than zero, in cases where invalid sidechain blocks have been mined, or when a side:node intentionally wants to orphan some side:blocks (if a side:node wants to orphan the most-recent N blocks, the value of the current block will be equal to N ; in the block after that it will be back to zero). + +![dots-image](/bip-blind-merged-mining/bmm-dots-examples.png?raw=true) + +Since the hashes themselves are already ordered by the mainchain, tracing the blockchain's path by index (prevBlockRef) will be the same as tracing it by identifying a list of hashes. In other words, the ordering given via each side:block's "prevBlockRef" will be isomorphic to an ordering given by each side:block's "prevSideHeaderHash" ... if "prevSideHeaderHash is defined to be the sidechain's equivalent of the mainchain's "prevBlockHash". It will be possible to freely convert from one to the other. See M8 to learn more about how these hashes are requested by sidechain block creators to be included in the mainchain. + +Now that we know what our critical data is, and how it is made, how is this data broadcast and stored? + +##### Broadcast + +Mainchain nodes are going to need this data later, so it must be easy to find. We will put it into the coinbase via OP RETURN. + +#### M7 -- "Blind-Mine the Sidechain(s)" + +Thus, (for n sidechains) we have a coinbase output with: + + 1-byte - OP_RETURN (0x6a) + 1-byte - Push the following (4+(n*35)) bytes (0x??) + 4-byte - Message header (0xD3407053) + (n*(32+5))-byte - A sequence of bytes, of the three Mini-Header items (h*, prevBlockRef, ChainIndex). + +( We assume that 5 bytes are used for the Critical Data bytes (non h* parts of the Sidechain Mini-Header). For 256 sidechains, a total of 9,478 bytes [1+1+4+256\*(32+5)] are required, conveniently just below the 10 KB scriptPubKey size limit.) + +This data is parsed by laying it in sequential 37-byte chunks (any remaining data --ie, some final chunk that is less than 37 bytes in length-- has no consensus meaning). + +Each 37-byte chunk is then parsed to obtain the data outlined above (in "Description"). If two 35-byte chunks being with the same "Sidechain number" (ie, if the two chunks have the same first byte), then only the first chunk has consensus meaning. + +We are left with, at most, one (h*, prevBlockRef) pair per sidechain per block. This data is added directly to D3, a new database. + +#### D3 -- "RecentSidechains_DB" + +To suit our purposes, the mainchain full nodes will need to keep track of the most recent 8000 (h\*, prevBlockRef) pairs. + +( This 8,000 figure is a tradeoff between decentralization (costs of running the main:node) and sidechain security -- it requires attackers to merged-mine 8,000 invalid blocks consecutively, in order to cause the sidechain to fail. The mainchain burden is minimal, so this figure might be raised to 12,000 or higher. ) + +Therefore, D3 would look something like this: + + + BlockHeight CB_Index SC_1 Blks_Atop_1 SC 2 Blks_Atop_2 SC 3 Blks_Atop_3 + --------- ------ ------ --------- ------ --------- ------ --------- + 1. 401,005 2 (h*, 0) 7985 (h*, 0) 1 (h*, 0) 0 + 2. 401,006 4 (h*, 0) 7984 (h*, 0) 0 (h*, 1) 7801 + 3. 401,007 2 (h*, 0) 7983 (h*, 5) 2027 (h*, 0) 0 + 4. 401,008 2 (h*, 0) 7982 (h*, 0) 2028 (h*, 1) 7800 + ... ... ) + 7999. 409,003 3 (h*, 0) 1 (h*, 0) 0 (h*, 0) 1 + 8000. 409,004 2 (h*, 0) 0 (h*, 1) 0 (h*, 0) 0 + + +When new sidechains (or "hashrate escrows") are soft-forked into existence, a new column is added to D3 (to contain any BMMing that might be done on it). + +For each sidechain we also track the field "Blocks Atop". This is the number of side:blocks that are "on top" of the specified side:block. These might be regarded as "side:confirmations" (pseudo-confirmations that are specific to each sidechain). + +D3 also contains a column (not shown) for each sidechain containing "prevSideBlockHash". This value is is either derived from prevBlockRef; or else it is given explicitly (in which case it is the converse: prevBlockRef is derived from prevSideBlockHash). + + +#### Coinbase Cache + +As mentioned above, M7s cause data to be added to D3. Recent D3 data is tracked by all mainchain nodes for a period of time. + +To efficiently keep track of the above data, without having to constantly load and process entire blocks from disk, we temporarily cache enough coinbases in the chain index to maintain D3. + + +### M8 -- Paying miners to include BMM data in their coinbase outputs + +This section introduces a new type of transaction, which allows sidechain block creators to request, and pay for, a critical hash to be included in a specific block by mainchain miners. See [the Blind Merged Mining spec](http://www.truthcoin.info/blog/blind-merged-mining/). This txn allows miners to "sell" the act of mining a sidechain block. By taking advantage of this option, miners earn tx fees for mining sidechains...truly "for free". They do not even need to run sidechain nodes, and the tx-fees they earn are in mainchain BTC. As a result, sidechains affect all miners equally and do not affect the mining ecosystem. + +M8 will ultimately come in two versions. The second version will be specialized for use in the Lightning Network and must use the full 32-byte prevBlockHash (ironically, this larger transaction is cheaper for the Bitcoin network to process, as it is completely off-chain). The first version of M8, in contrast, cannot be used inside the Lightning Network, but is slightly more space-efficient (using the 2 prevBlockRef bytes to maintain chain order). It is important to make both options available to the user, because some side:nodes may be unwilling or unable to open a payment channels with each main:miner. However, in the long run we expect the lightning version to be preferred. + +#### Setup + +We define **"Mary"** as a mainchain miner, and **"Simon"** as a sidechain node. + +The goal is to construct a payment from Simon to Mary, such that: + +1. If the critical data conditions are met, **Mary** can claim the outputs of the transaction with finality. +2. If the critical data conditions are not met, the outputs become immediately available again to **Simon**. + + +#### Goals (this is rather philosophical, and skippable) + +##### Immediate Expiration ("Fill-or-Kill") + +We would like to make special guarantees to the counterparties of this transaction. Specifically, instead of Simon making a "payment" to Mary, we prefer that Simon give Mary an "offer" (which she can either accept or decline). + +Crucially, we want Simon to safely make many offers to several different Mary's, in realtime (ie, quickly and off-chain). However, we ultimately want only one offer to be accepted, at most. In other words, we want Simon's offers to *immediately expire*. If only one offer can become a bona fide transaction, then Simon will feel comfortable making offers all day long. Because all of the Simons are making many offers, the Marys collectively gain access to a large set of offers to choose from. + +##### Forward Progress (The Need for a "Ratchet") + +The "ratchet" concept is an attempt to harmonize incentives among the main and side chain(s). +We need to ensure that a sidechain is making "forward progress", without tracking too much about the sidechain such that we burden Bitcoin (see [1] and [2]) all while still allowing the sidechain to reorganize [3]. + +* [1] https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-July/014789.html +* [2] http://www.drivechain.info/faq/index.html#what-is-the-difference-between-drivechain-and-extension-blocks +* [3] http://www.truthcoin.info/blog/blind-merged-mining/#handling-reorganizations + +The ratchet system must keep track of sidechain "mini-headers" (see Sidechain Critical Data ("Sidechain Mini-Header")) and count the "blocks atop" maturity of the related side:blocks. + +Simon's offer to Mary to include a critical hash in exchange for payment must be *atomic*. The "ratchet" concept helps to construct a very tight connection between two things: + +1. The sidechain-block-generator "Simon" paying himself the side:block's side:tx-fees (which he receives in 100 sidechain blocks (blocks atop) hence). +2. "Simon" making a mainchain main:btc payment to a mainchain miner "Mary". + +Either both of the two should succeed, or else both should jointly fail. + +However, absent our intervention, there are cases in which [2, the payment to Mary] succeeds but [1, side:tx-fees] fails. One such case is when a side:block contains unusually high side:tx-fees. Here, there will be many requests to include a critical hash in exchange for payment submitted to Mary, but only one can be included in each main:block per sidechain. Without an incentive to make "forward progress", Mary is likely to include one of the highest paying requests in the next main:block (and the main:block after that, and so on). Mary will "blindly" include high-paying requests for *older* blocks, unless something prevents her from doing so. + +To address these potential issues, we utilize the concept of "Blocks_Atop" (the "side:confirmations") that we mentioned earlier. As previously mentioned, Mary will not be able to spend Simon's M8 payment until satisfying the critical data requirements as well as the blocks atop (side:confirmations) requirement. + + +#### M8 -- The two forms of M8 transactions + +As previously mentioned, M8 can take two forms. The first does not require the Lightning Network, but it does have new requirements for Immediate Expiration (see above). The second inherits Immediate Expiration from the Lightning Network itself, but requires extra preparation and a different/larger message. + +Both forms require that certain Critical Data will be committed to within the coinbase of the block that the transaction is included in. For the non Lightning version, we have created a new extended serialization transaction type (very similar to how segwit handles witness data (the witness stack)). + +##### M8_V1 - No Lightning Network + +M8_V1 does not require the Lightning network but does have new requirements for validation. + +A M8_V1 TxOut is expected to contain: + + 1-byte - OP_RETURN (0x6a) + 1-byte - Push the following 36 bytes (0x24) + 4-byte - Message header (0xD1617368) + 32-bytes - h* side:block hash + 5~7-bytes - BMM request identifying bytes (0x00bf00) + prevBlockRef & ChainIndex (sidechain mini-header) + + +In the first version of M8, we need to introduce the concept of Immediate Expiration (see above). In other words, we need a way for Simon to construct many payments to multiple Marys, such that only one of these is ever included; and only then if Simon's txn is expected to coincide with the finding of Simon's side:block. + +We do this by imposing validity-rules on the txn itself: + +1. The txn's content, when examined, must match part of the main:block's content. Specifically, the (ChainIndex, h\*) pair of the txn, must match one of the (ChainIndex, h\*) pairs in the M7 of this main:block. +2. Only one payment per sidechain per main:block is valid. In other words, if 400 people all try to bm-mine the sidechain with ChainIndex==4, then not only is it the case that only one side_4:block can be found, but it is also the case that only the corresponding M8 txn can be included (out of all of the 400 M8s which are for ChainIndex==4). +3. Simon's txns must only be valid for the current block; afterward, they immediately expire. This is because Simon's intended prevBlockRef & side:block contents will most likely change from one main:block to the next. + +To impose new requirements on the transaction level (not the block level nor the TxOutput level), we borrow the "flag" trick from SegWit style transactions. If the flag is present, the transaction is examined for extra data, and if this data does not pass certain requirements, the transaction is invalid. With SegWit, this extra data is the signatures, and the extra requirements are the signatures' locations and validity. In the BMM-transactions, the extra data is the (ChainIndex, h\*) pair, which must meet the first two requirements (above) as well as the main:blocknumber, which must meet the third requirement (above). + +To impose new requirements at the transaction level, we borrow the dummy vin & "flag" trick from SegWit style transactions. If the flag is set to 2 (0010), the transaction contains Critical Data and requires that our new validation rules be met in order for the txn to be valid in a block. Unless all of the requirements for sidechain critical data transactions are met by the block it is included in, the transaction is invalid. With SegWit, this extra data is the segwit signature stack, and the extra requirements are the signatures' locations and validity. In the sidechain BMM critical data transactions, the extra data is the (ChainIndex, h\*) pair, which must meet the first two requirements (above) as well as the main:blocknumber, which must meet the third requirement (above). Note The main:blocknumber does not take up any additional space compared to a normal txn, as we reuse the locktime field for our purposes. + + + + +![extra-data-image](/bip-blind-merged-mining/witness-vs-critical.png?raw=true) + +This txn structure conserves main:blockspace, because it is the easiest way to refer to a previous sidechain block in 4 bytes, (prevBlockRef + FoK_nLockTime). Instead, we would need to use at least 32 bytes (prevSideBlockHash). + +These types of transactions have slightly different mempool behavior, and should probably be kept in a second mempool. These txns are received, checked immediately, and if valid they are evaluated for inclusion in a block. If they are not able to be included in the specific requested block (if the block height requested has been surpassed by the chain tip), they are discarded. In fact, after any main:block is found, everything in this "second mempool" can be discarded as new payments will be created immediately for the next block height. (This includes cases where the blockchain reorganizes.) There is no re-evaluation of the txns in this mempool ever -- they are evaluated once and then either included or discarded. To be clear, when the transaction is received we are able to evaluate its validity, and do not need to rescan these transactions again. + +Interestingly, these payments (M8) will *always* be directed to miners from non-miners. Therefore, non-mining nodes do not need to keep them in any mempool at all. Non-miner nodes can just wait for a block to be found, and check the txn then. These transactions more resemble a stock market's pit trades (in contrast, regular Bitcoin txns remind me more of paper checks). + +##### M8_V2 With Lightning + +M8_V2 requires having a LN-channel open with a miner. This may not always be practical (or even possible), especially today. + +A M8_V1 TxOut is expected to contain: + + 1-byte - OP_RETURN (0x6a) + 1-byte - Push the following 68 bytes (0x44) + 4-byte - Message header (0xD0520C6E) + 32-bytes - h* side:block hash + 32-bytes - prevSideBlockHash + 5~7-bytes - BMM request identifying bytes (0x00bf00) + prevBlockRef & ChainIndex (sidechain mini-header) + + +Notice that, in M8_V1, Simon could reuse the same h\* all he wanted, because only one M8_V1 could be included per main:block per sidechain. However, on the LN no such rule can be enforced, as the goal is to push everything off-chain and include *zero* M8s. So, we will never know what the M8s were or how many had an effect on anything. + +Therefore, Simon will need to ensure that he **gives each Mary a different h\***. Simon can easily do this, as he controls the side:block's contents and can simply increment a nonce -- this changes the side:block, and changes its hash (ie, changes h\*). + +With a unique h\* per Mary, and at most 1 h\* making it into a block (per sidechain), we can guarantee that only one of the M8_V2's critical data can be committed to in a single main:block. By giving each miner (who Simon has a payment channel open with) a different h*, Simon can figure out which miner followed through with the commit, and know that only one such commit went through. Furthermore, if this Simon's requested critical data is not found in a block, none of the M8_V2 payments will be spendable by the Mary(s), because none of the h\* in question have ever made it into D3 (which is always on-chain) and no blocks atop will be accumulated. + +That's probably confusing, so here is an example, in which: Simon starts with 13 BTC, Mary starts with 40 BTC, the side:block's tx-fees currently total 7.1 BTC, and Simon is keeping 0.1 BTC for himself and paying 7 BTC to Mary. + +We start with (I): + + Simon 13 in, Mary 40 in ; 53 in total + Simon's version [signed by Mary] + 13 ; to Simon if TimeLock=over; OR to Mary if SimonSig + 40 ; to Mary + Mary's version [signed by Simon] + 40 ; to me if TimeLock=over; OR to Simon if MarySig + 13 ; to Simon + + +And both parties move, from there to "M8_V2" (II): + + Simon 13 in, Mary 40 in ; 53 in total + Simon's version [signed by Mary] + 6 ; to Simon if TimeLock=over; OR to Mary if SimonSig + 40 ; to Mary + 7 ; to Mary if critical data requirements met; OR to Simon if LongTimeLock=over + Mary's version [signed by Simon] + 40 ; to Mary if TimeLock=over; OR to Simon if MarySig + 6 ; to Simon + 7 ; to Mary if critical data requirements met; OR to Simon if LongTimeLock=over + +From here, if the h\* side:block in question is BMMed, they can proceed to (III): + + Simon 13 in, Mary 40 in ; 53 in total + Simon's version [signed by Mary] + 6 ; to Simon if TimeLock=over; OR to Mary if SimonSig + 47 ; to Mary + Mary's version [signed by Simon] + 47 ; to me if TimeLock=over; OR to Simon if MarySig + 6 ; to Simon + +Although, if Simon proceeds immediately, he removes the protection of the 'ratchet'. Ie, Simon removes Mary's incentive to care about blocks being built on this side:block. If Simon's side:block is orphaned, he loses his 7 BTC. Simon can either play it safe, and wait the full 100 side:blocks before moving on (ie, moving on to the third LN txn, above); or else Simon can take the risk if he feels comfortable with it. + +If the h\* side:block is not found, then (II) and (III) are basically equivalent to each other. Simon and Mary could jointly reconstruct (I) and go back there, or they could proceed to a new version of II (with a different h\*, trying again with new side:block in the next main:block). + + + + +Deployment +=========== + +This BIP will be deployed by "version bits" BIP9 with the name "blindmm" and using bit 4. + +``` +// Deployment of Drivechains (BIPX, BIPY) +consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].bit = 4; +consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].nStartTime = 1515974401; // January 15th, 2018. +consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].nTimeout = 1547510401; // January 15th, 2019. +``` + +Reference Implementation +========================== + +See: https://github.com/drivechain-project/bitcoin/tree/mainchainBMM + +Also, for interest, see an example sidechain here: https://github.com/drivechain-project/bitcoin/tree/sidechainBMM + + +References +============ + +* http://www.drivechain.info/literature/index.html +* http://www.truthcoin.info/blog/blind-merged-mining/ +* https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-July/014789.html +* http://www.truthcoin.info/images/bmm-outline.txt + + +Thanks +========= + +Thanks to everyone who contributed to the discussion, especially: ZmnSCPxj, Adam Back, Peter Todd, Dan Anderson, Sergio Demian Lerner, Matt Corallo, Sjors Provoost, Tier Nolan, Erik Aronesty, Jason Dreyzehner, Joe Miyamoto, Ben Goldhaber. + + +Copyright +========== + +This BIP is licensed under the BSD 2-clause license. diff --git a/bip-blind-merged-mining/bmm-dots-examples.png b/bip-blind-merged-mining/bmm-dots-examples.png new file mode 100644 index 0000000..70f11f6 Binary files /dev/null and b/bip-blind-merged-mining/bmm-dots-examples.png differ diff --git a/bip-blind-merged-mining/images.txt b/bip-blind-merged-mining/images.txt new file mode 100644 index 0000000..00b3932 --- /dev/null +++ b/bip-blind-merged-mining/images.txt @@ -0,0 +1 @@ +Images used as reference in the documentation. diff --git a/bip-blind-merged-mining/witness-vs-critical.png b/bip-blind-merged-mining/witness-vs-critical.png new file mode 100644 index 0000000..1a2458d Binary files /dev/null and b/bip-blind-merged-mining/witness-vs-critical.png differ -- cgit v1.2.3 From 1af547cf728d27070b3c90ce02c3a7731bb5dcd6 Mon Sep 17 00:00:00 2001 From: Paul Sztorc Date: Sat, 3 Feb 2018 21:38:23 -0500 Subject: Delete bip-hashrate-escrows.md --- bip-hashrate-escrows.md | 408 ------------------------------------------------ 1 file changed, 408 deletions(-) delete mode 100644 bip-hashrate-escrows.md diff --git a/bip-hashrate-escrows.md b/bip-hashrate-escrows.md deleted file mode 100644 index 3b5d61e..0000000 --- a/bip-hashrate-escrows.md +++ /dev/null @@ -1,408 +0,0 @@ - -Header -======= - - BIP: ???? - Layer: Consensus (soft fork) - Title: Hashrate Escrows (Consensus layer) - Author: Paul Sztorc - CryptAxe - Comments-Summary: No comments yet. - Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-??????? - Status: Draft - Type: Standards Track - Created: 2017-08-14 - License: BSD-2-Clause - Post-History: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-May/014364.html - - -Abstract -========== - -A "Hashrate Escrow" is a clearer term for the concept of "locked to an SPV Proof", which is itself a restatement of the phrase "within a sidechain" as described in [a famous Oct 2014 paper](https://blockstream.com/sidechains.pdf) written partially by some Blockstream co-founders. - -A Hashrate Escrow resembles a 2-of-3 multisig escrow, where the 3rd party (who will arbitrate any disputes) is a decentralized group of people: the dynamic-membership set of Bitcoin Miners. However, the 3rd party does not sign escrow-withdrawal transactions with a private key. Instead, these are "signed" by directing hashpower over them for a period of time. - -This project has [a website](http://www.drivechain.info/) which includes [a FAQ](http://www.drivechain.info/faq/index.html). - - -Motivation -============ - -In practice these escrows are likely to be "asymmetric sidechains" of Bitcoin (such as [Rootstock](http://www.rsk.co/)) or "virtual chains" within Bitcoin (such as [proposed by Blockstack](https://github.com/blockstack/virtualchain) in mid-2016). - -Sidechains have many potential benefits, including: - -1. Protect Bitcoin from competition from altcoins and spinoffs. Safely allow competing implementations (of *sidechains*). -2. Protect Bitcoin from hard fork campaigns. (Such campaigns represent an existential threat to Bitcoin, as well as an avenue for developer corruption.) -3. Help with review, by making it much easier for reviewers to ignore bad ideas. -4. Provide an avenue for good-but-confusing ideas to prove their value safely. - - - -Specification -============== - - -#### Components - -Hashrate Escrows are built of two types of component: [1] new databases, and [2] new message-interpretations. - -##### 1. New Databases - -* D1. "Escrow_DB" -- a database of "accounts" and their attributes. -* D2. "Withdrawal_DB" -- a database of pending withdrawals from these accounts, and their statuses. - -Please note that these structures (D1 and D2) will not literally exist anywhere in the blockchain. Instead they are constructed from messages...these messages, in contrast, *will* exist in the blockchain (with the exception of M4). - -##### 2. New Messages - -* M1. "Propose New Escrow" -* M2. "ACK Escrow Proposal" -* M3. "Propose Withdrawal" -* M4. (implied) "ACK Withdrawal" -* M5. "Execute Deposit" -- a transfer of BTC from-main-to-side -* M6. "Execute Withdrawal" -- a transfer of BTC from-side-to-main - - -#### On the Resource Requirements of New Databases - -The "new" databases are simply reinterpretations of data that are already contained elsewhere in the blockchain. Specifically, M1 M2 and M3 are all located in the block's coinbase txn, and M5 and M6 might be found in any regular txn. M4 is a special case and does not actually need to be included anywhere, so it is not. If you like, you can imagine that the M4s reside in an optional extension block. - -In other words, we just rearrange what is already there. Because of this, even though "new databases" are created and stored in memory, the existing bandwidth and storage limits are respected (although, see "M4" below). - - - - -### Adding Sidechains and Tracking Them (D1, M1, M2) - -#### D1 -- "Escrow_DB" - -The table below enumerates the new database fields, their size in bytes, and their purpose. In general, an escrow designer (for example, a sidechain-designer), is free to choose any value for these. - -Note: Fields 6 through 9 have been intentionally removed. Previously, this section allowed miners to set and commit to voting/waiting periods. However, I have since standardized the periods: withdrawals expire after 6 months (26298 blocks), and they succeed if they ever achieve an ACK score of 13140 or higher. I have removed the waiting period, because anyone who adopts a policy of ignoring all withdrawals with fewer than 400 ACKs will automatically gain all of the benefits of the waiting period. The justification for this change is that it strongly implies that an attack on any one sidechain is an attack on all of them (in a sense, this change makes the "victimhood" of each sidechain "fungible"). - -Field No. | Label | Bytes | Type | Description / Purpose -----------|-------|------|------|-------- -1 | Escrow Number | 1 | uInt | A number assigned to the entire escrow. Used to make it easy to refer to each escrow. -2 | Active\* | 2 | uInt | This counter starts at zero when a new entry is added, and increments as a new entry is supported by "M2". The entry either reaches the max value of 4032 (at which point the counter freezes), or else the entry is removed. This enforces the requirement "a soft fork is required to add a new escrow". -3 | Escrow Name/Description | 120 | string | A human-readable name and description of the sidechain. More than enough space to hold a 32 byte hash. Helps prevent destructive interference among sidechains (see below). -4 | Critical Private Key | 32 | hex | This is the Double-SHA256 of the binary release of the reference software. It ties the sidechain to a specific software release, and doubles as a way of signing withdrawals (see below). -5 | Critical Address\* | 32 | string | This is derived by taking f(x) = ImportPrivateKey(HexToBase58Check(x)) of field #3. It is intentionally in the human-readable format {{although it could be 25 bytes of binary}}. -10 | "CTIP" -- Part 1 "TxID"\* | 32 | hex | The CTIP, or "Critical (TxID, Index) Pair" is a variable for keeping track of where the escrow's money is (ie, which member of the UTXO set). -11 | "CTIP" -- Part 2 "Index"\* | 4 | hex | Of the CTIP, this is second element of the pair: the Index. See #10 above. - - -\* Denotes a "convenience field", the entry for this field is derived from other fields, or from the blockchain-state itself. The escrow-creator does not need to provide these values in M1 (or anywhere). - -Escrow_DB requires 223 bytes [1+120+32+32+2+32+4] for each escrow in the main blockchain. Of these, 70 bytes [2+32+32+4] are merely for convenience. Therefore, a sidechain is defined (see "M1") by 153 bytes of information. - -#### Notes on D1 - -1. D1 will always exist. -2. D1 will always have a unique sort (first by "Escrow Number" (field #1), and second by "Active" (field #2)). There is only ever one (escrow number, Active) pair. -3. D1 is updated according to M1 and M2 (below). -4. If a new entry is added to D1 with an "Escrow Number" that is already in use, then this entry will either eventually be removed (because it was not supported with an M2), or it will eventually overwrite the old entry (if it *was* supported via M2). - - -#### Notes on D1 - -##### Obligations Placed on Miners - -Miners have always upgraded their software according to criteria that are known only to them (in other words, "whenever they want"). - -However, this soft fork imposes two new criteria upon them. First: miners should only upgrade their software, if any modification to the portfolio of sidechains [that are added/removed in the upgrade] can be expected to increase miner wealth. Trivially, this implies that miners should make sure that the upgrade doesn't overwrite (and destroy) an existing sidechain that they like! But, more seriously, it implies that miners should take an interest in what the sidechain is doing to the mainchain and other sidechains (see below). - -##### Destructive Sidechain Interference - -People frequently emphasize that miners should have "as little control" as possible. It is a very safe claim to make, and a very easy sentence to write. Much harder is to determine exactly what this minimum value is, and how to achieve it. Harder still is to untie the knot of who is actually controlling what, in a decentralized, interacting system. - -Certainly, miners can not have "zero control" -- for that is the same as to just remove them from the system altogether. Some rules are enforced "on miners by nodes" (such as the infamous blocksize limit); other rules are enforced by nodes but are narrowly-controlled by miners (such as the proof-of-work itself, or the block's timestamp). Thirdly, some rules are enforced by both against each other (such as the rule against including invalid txns or double-spent txns), for mutual benefit. - -Some pause should be given, after one considers that the sidechain design goal is literally a piece of software that can do *anything*. Anything includes a great many things, many of which I demonstrate to be undesirable. Bitcoin itself does not allow "anything" -- it allows any person to transact, but, in contrast, it does not permit any person to double-spend. This is because "allowing anyone to do anything" is not viable in a world that contains undesirable interactions (what a libertarian might call "aggression") -- in the case of money, these are theft and counterfeiting. - -I have produced a comprehensive quantity of written material [1], presentations [2], etc [3] on exactly what the level of miner-control should be, and why. Specifically, I claim that **miners should be aware of the purpose of the sidechain, and they should reject sidechains which have an unclear purpose or which have a purpose that will lead to decrease in miner-wealth** (where wealth measured explicitly as: the estimated present value of the purchasing power of the blockchain's coinbase txns). I claim that this criterion is necessary because, just Original Bitcoin filters unwanted interactions among different BTC txns, so too much "Sidechain Bitcoin" filter out unwanted interactions among sidechain. - -* [1] http://www.truthcoin.info/blog/wise-contracts/ -* [2] https://www.youtube.com/watch?v=xGu0o8HH10U&index=1&list=PLw8-6ARlyVciMH79ZyLOpImsMug3LgNc4 -* [3] http://www.drivechain.info/literature/index.html - -Call it a "sidechain non-aggression principle", if you want. - -To the best of my knowledge, everyone who *has* reviewed this information as found the arguments to be acceptable. It has, also, changed a few minds (from "unacceptable" to "acceptable"). - - -##### ISSUE: "Signing" BTC Txns - -Currently, we use a process which may be suboptimal. It is that we *literally sign* a txn with a globally and publicly known private key. But this is for convenience purposes -- the signature that is produced is not doing anything, and is therefore wasteful. Instead we may use OP_TRUE, but this might interfere with how we detect the sidechain's balance. I'm not sure what the best way is. Someone needs to investigate how to do this -- removing OP_CheckSig, etc. This is a TODO for sure, and an opportunity for someone to help. - - - -(The following messages were modeled on SegWit -- https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki#commitment-structure ) - - - -#### M1 -- "Propose New Sidechain" - - 1-byte - OP_RETURN (0x6a) - 1-byte - Push the following 157 bytes (0x9d) - 4-byte - Commitment header (0x53707243) - 153-byte - the critical bytes mentioned above (fields #1, #3, and #4, to populate a new D1 entry) - - -#### New Block Validation Rules - -1. If the network detects a properly-formatted M1, it must add an entry to D1, into the very next block, with the following initial values: -* Field #5 will be calculated as per [version 1 Bitcoin addresses](https://en.bitcoin.it/wiki/Technical_background_of_version_1_Bitcoin_addresses), but with a prefix of "4" instead of "1". -* Field #9 will be derived from #7 and #8 using math. -* The initial values of Fields #10, #11, and #12 are set to zero. -2. Only one M1 (of any kind) can be added into a block at a time. - -#### Notes on M1 - -The escrow will "reuse" the same address over and over. But notice that there is no privacy loss due to address reuse, because the address in question does not "belong" to any specific individual. Furthermore, the transactions in question could already be linked by just watching the Hashrate Escrow for deposits ...this, in fact, is exactly what a sidechain must do in order to make main-to-side transfers. - - -#### M2 -- "ACK Sidechain Proposal" - - 1-byte - OP_RETURN (0x6a) - 1-byte - Push the following 29 bytes (0x1D) - 4-byte - Commitment header (0x53616343) - 25-byte - Commitment hash: RIPMD-160 of Sha256 of a given M1 (above) - -#### New Block Validation Rules - -1. An entry will be removed from D1, unless that block contains an M2 supporting it (this will increment field #2), or the entry has accumulated 4032 M2s. Therefore, each entry needs to have 4032 *consecutive* M2s supporting it, else it will be removed from D1 and must start over. This requires all miners to agree to "soft fork" the sidechain into existence. This makes it easier to curate the list of active sidechains, and prevents destructive interference (see above) among contradictory sidechains. -2. Once the entry has 4032 M2s, then a miner must make a 0 btc payment to the Critical Address, thus activating the chain. This payment becomes the first "critical (TxID, index) pair" (or "CTIP"). -3. It is possible to "overwrite" a D1 slot -- this destroys the sidechain that is there, and replaces it with a new one. This is defined as requiring 6 months (26298 blocks) of M2s, instead of 4 weeks (4032). This possibility does not change the security assumptions, because we already assume that users have access to "magically true" bandwidth at a rate of 1 bit per ACK-cycle. (This is why the voting cycles should be large, 1 month Waiting, 2+ months Voting). - - - -### Adding Withdrawal-Attempts and Tracking Their Success (D2, M3, M4) - -#### D2 -- "Withdrawal_DB" - -The table below enumerates the database fields, their size (in bytes), type and purpose. - - -Field No. | Label | Bytes | Type | Description / Purpose -----------|-------|------|------|----------------------- -1 | Escrow Number | 1 | uInt | Links the withdrawal-request to a specific escrow. -2 | WT^ | 32 | hex | This is a "blinded transaction id" (ie, the double-Sha256 of a txn that has had two fields zeroed out, see M6) of a withdrawal-attempt. -3 | ACKs\* | 2 | uInt | The current total number of "votes", this starts at 0 and remains there throughout the waiting period. -4 | Age\* | 3 | uInt | Total duration of time, in blocks, that this WT^ has been inside of D2. -5 | Waiting Period\* | 2 | uInt | Total duration of time, in blocks, that this entry must sit idle, before it can begin to accumulate ACKs/NACKs. Pulled from D1's field #6. -6 | Max Age\* | 3 | uInt | Determined by summing (D1's field #6) and (D1's field #7). -7 | Threshold\* | 2 | uInt | Total ACKs needed, this is pulled from D1's field #9. -8 | Approved\* | 1 | boolean | True while ACKs > Threshold, False otherwise. - -\* Denotes a "convenience field" (see above). - -Withdrawal_DB requires 46 bytes [1+32+2+3+2+3+2+1] per entry. Of these, 13 bytes ([2+3+2+3+2+1], all fields except #1 and #2) can be generated locally, leaving 33 critical bytes [1+32]. - -#### New Block Validation Rules for D2 - -1. In each block, a hash commitment to D2 must always exist (even if D2 is blank). -2. D2 must always be sorted first by field #1 (Escrow Number) and second by field #4 (Age). This imposes a unique sort. -3. From one block to the next, every entry's "Age" field must increase by exactly 1. -4. From one block to the next, entries are only removed from D2 (in the very next block) if: -* * "Age" = "MaxAge". -* * If the block contains a txn who's blinded txID matches WT^. {{ This might be unnecessary, and a lot of work. }} -5. In addition, there are special rules for the allowed values in the "ACKs" field (field #3). See M4 below. - -#### M3 -- "Propose Withdrawal" - - 1-byte - OP_RETURN (0x6a) - 1-byte - Push the following 37 bytes (0x25) - 4-byte - Commitment header (0xD45AA943) - 33-byte - the critical bytes mentioned above (fields #1 and #2, to populate a new D2 entry) - - -#### New Block Validation Rules for M3 - -1. If the network detects a properly-formatted M3, it must add an entry to D2 in the very next block. The starting values of fields #3 and #4 are zero, and #5 is pulled over by extracting the relevant value from D1. -2. Each block can only contain one M3 per sidechain. - - -#### M4 -- "ACK Withdrawal" - -#### Very Little Info, Probably Calculable in Advance - -M4 is exceptional (in comparison to the other M's) in a few ways. First, its content is not stored anywhere, only the *hash* of its *effect* is stored (in a leaf of a merkle tree who's root is inserted into a mainchain coinbase). M4 alters the contents of D2 -- the *contents* of D2 are consensus critical, but M4 (the process by which nodes reach a new valid D2) can be anything. - -In fact, M4 can also be *nothing*. In other words, it may be optional. This is precisely because, from one block to the next, we have constrained D2 such that it is only allowed to change in a few ways. Therefore, the exhaustive set of "candidate D2s" can be precomputed by full nodes in advance. - -#### Two Withdrawals at Once - -In general, only one withdrawal (per sidechain) can make progress (toward being included in a block) at a time. In other words, as WT^s are proposed, only one can make progress toward the finish line. As a result, a given side-to-main transfer will always take between 3 and 6 months. Instead, with more simultaneous withdrawals, the worst-case transfer duration would improve. - -![dots-image](/bip-hashrate-escrows/two-groups.png?raw=true) - -The worst-case withdrawal time obeys f(n)=3+(3/n) months, where n is the number of simultaneous withdrawals. - -N=2 is the most desirable choice for several reasons. First, it delievers the greatest marginal benefit (of 1.5 months). Later choices only deliver 0.5 and 0.25 marginal months. - -Second, n=2 can be implemented in a clever way: by allowing a withdrawal to freely advance, if and only if has an ACK-score of 6575 or greater, and if it also has the largest ACK score. In other words, the withdrawal that is furthest along can advance (or retreat) for free, if it has already made it at least halfway to the finish line. With this change, our new M4, is either an "abstain" for the sidechain (in which case nothing happens to any ACK scores), or else it will be in one of two cases: old_M4 + "the largest advances", or new_M4 + "the largest retreats". As a result the number of M4 possibilities (of which the next section is concerned) only increases by a factor of two (instead of exponentially). - -It is possible to troll this rule, by getting two (or even three) withdrawals to have 6575+ ACK scores, and then getting them to *tie* for first place. So, if there are any ties, the ability to "bonus move" is disabled until all ties are broken. - -#### How Hard is it to Guess M4? - -If there are n Escrows and m Withdrawals-per-escrow1, then there are (m+2)^n total candidates for the next D2. This is because, [per block per escrow], one of three things can happen: (1) one of the m withdrawal-candidates can be "ACK"ed (or "upvoted" or "promoted"), which automatically downvotes the others; or (2) all withdrawal-candidates can be downvoted, or finally (3) the miners can abstain from voting on the escrow's withdrawals altogether, leaving the tallies the same. - -First, for nodes which validate all sidechains (assuming these escrows are sidechains), this simplifies to 2^n -- these nodes only have to choose between the single honest choice (on one hand) or an abstention (on the other). Second, even for nodes that don't validate any sidechains, the number of candidates might be reduced from m^n to 3^n, by making a simplifying assumption: whichever withdrawal was most recently added/upvoted, is likely to be the one which is upvoted next. - -Of course, that is still O(k^n) for n sidechains, which isn't great2. If the "D2 update" cannot be guessed, it must be transmitted in some way. - -#### Giving Up and Getting M4 the Old Fashioned Way - -Two examples for transmitting it are below: - -"Short Form" (Assumes there are no more than 254 active withdrawal-attempts per account) - - 4-byte - Message identifier (0x????????) - 1-byte - Version of this message - N-byte - N is the total number of active accounts ("sidechains"), each byte specifies the position of the single WT that was "upvoted". A value of 0 indicates "downvote everything", a value of 255 indicates abstention. - -"Long Form" (Makes no assumptions about anything) - - 4-byte - Message identifier (0x????????) - 1-byte - Version of this message - 1-byte - Length (in bytes) of this message; total number of withdrawal attempts; y = ceiling( sum_i(m_i +2)/8 ). Nodes should already know what length to expect, because they know the sequence of M3s and therefore the vector of WT^s. - Y-byte - stream of bits (not bytes), with a 1 indicating the position of the chosen action [downvote all, abstain, upvote1, upvote2, ...] - - -If the message is very very large, then nodes may not want to broadcast it. This opens up an "exhaustion attack"2, in which many miners create bad WT^s, vote on these randomly, and then refuse to broadcast their votes. Fortunately, even for a worst-case scenario of 200 sidechains and 1,000 withdrawal-attempts per sidechain, honest nodes can communicate a long form M4 with each other by using just 25,056 bytes per block [4+1+1+(200\*(1000+1+1)/8)]. - -Today's pre-drivechain miners can already carry out a similar attack, by creating and including txns and then not broadcasting that part of the block to anyone. This is often characterized as a ["block publication incentive"](https://petertodd.org/2016/block-publication-incentives-for-miners), because in that case the prospect of exhaustively computing all possible transactions (to uncover the missing ones) is completely out of the question. - -However, message M4 is different from a withheld-txn, because M4 operates outside of the block's mandated information-processing limits (ie, outside the infamous 1 MB nonwitness blocksize limit). So we should examine the conditions under which M4 grows and shrinks, to ensure that we are not smuggling in a tremendous burden on full nodes. - -Under adversarial conditions, to lengthen a long-form M4 by one bit per block, for C blocks, the attacker must pay 312 bits (39 bytes) one time (to embed a new M3 message). The value C is the length of the sidechain's voting period, which varies but which I expect to be approximately 8,064 (and which could theoretically be as high as 65,536). Thus the attacker can burden nodes disproportionately, if (s)he wishes. - -Fortunately, the attack in question has no motivation (as far as I can tell). If the miner's goal is to trick rivals into mining on top of invalid blocks, he can already do this much more effectively with the unpublished-txn method (above). If instead he is just trying to harass nodes, then nodes may freely "downgrade" to earlier versions of the protocol, and simply ignore all drivechain-related messages. It seems that the attack could best be used in order to: make a large D2, make D2 confusing, sneak in votes for evil WT^ lurking in D2. Thus, the attack disables the transparency of the drivechain system, to some extent. The cost of the attack is forgone transaction fees, due to block space wasted on useless M3s. - -In practice, n is already capped, and miners may impose [on each other] a "soft cap" on m for their mutual protection. Thus, n and m might never get above 10 and 30, respectfully. In this case, the [Short Form, this time] M4 can never require more than 15 bytes per block, no matter what the attacker tries. - -In practice, m should always be 1 or 2, else something fishy is going on; and m can only inch up by 1 unit per block. So the system as a whole is still quite transparent, in that users are warned appropriately and well in advance. Attackers must invest upfront and they face an uphill climb, in order to eventually make things more expensive for a few others; defenders can wait-and-see if the attack looks like it will ever amount to anything before lifting a finger. - - -##### New Block Validation Rules (for D2 and, by implication, M4) - -From one block to the next, D2 can only be edited in a few strict ways: - -* Entries can only be added/removed from D2 if they meet the criteria above (in M3, and implicitly M1 and M2). -* The ACK-counter of any individual entry can only change by (-1,0,+1) relative to its previous entry. -* Within a sidechain group, upvoting one withdrawal (ACK=ACK+1) requires you to downvote all other withdrawals in that group. However, the minimum ACK value is zero (and, therefore, downvotes cannot reduce it below zero). - -##### Footnotes for M4 - -1 This represents the worst-case scenario is one where all the Withdrawals are spread evenly over each Sidechain. Under normal operations, there is no reason to expect the all sidechains will have the same number of withdrawals at any given time. In fact, under normal operations, the very *concept* of counting the withdrawals-per-sidechain should be a purposeless one, because there should only be *one* withdrawal at a time. Nonetheless we consider the worst case scenario here. - -2 Guessing becomes more computationally intensive in a highly adversarial situation where the "limited range" is intentionally expanded. In such a scenario, [a] there are many sidechains, and [b] miners voluntarily sacrifice their scarce block-space by creating a high number of (mutually-exclusive, and hence ultimately invalid) withdrawal attempts and putting these into coinbase transactions; and then agree to all [c] vote on these randomly (guaranteeing that all withdrawals fail, including any true withdrawals) and [d] successfully withhold their random voting strategies from nodes (even including spy-miner-nodes). Under this bizarre scenario, nodes may require computing resources which increase near-exponentially with the number of withdrawals, and it may take a long time for an ignorant node to exhaustively work out the underlying state of Withdrawal_DB. In this case, nodes may decide to temporarily stop validating such transactions (as if they had not yet upgraded to support this soft fork). - - - -### Depositing and Withdrawing (M5, M6) - - -Both M5 and M6 are regular Bitcoin txns. They are identified by meeting an important criteria: they select a one of the Critical TxID-index Pairs (a "CTIP") as one of their inputs. Deposits ("M5") are distinguished from withdrawals ("M6") by simply checking to see if money is "going in", or "out". In other words, we compare the BTC value of the original CTIP to that of new CTIP. If original <= new it is a deposit, if original > new then it is a withdrawal. - -The code that identifies sidechain withdrawal / deposit txns (by calculating how much value is being put into or taken out of a sidechain) can be seen here: https://github.com/drivechain-project/bitcoin/blob/mainchainBMM/src/validation.cpp#L351-L386 - -Such txns are forced (by consensus) to obey two additional criteria: - -1. They must contain an output paying "to" the Critical Address [probably in TxOut0]. -2. They must be accompanied by an update to this sidechain's Critical TxID-index Pair (CTIP). The new CTIP must be "this" txn itself. - -These criteria are enforced here https://github.com/drivechain-project/bitcoin/blob/mainchainBMM/src/validation.cpp#L440-L473 by checking that a deposit is paying back to the sidechain more than it is taking out, and completely rejecting any withdrawal from the mempool. And here https://github.com/drivechain-project/bitcoin/blob/mainchainBMM/src/validation.cpp#L1747-L1757 we allow for a withdrawal only once it has attained sufficient work score (ACKs). - -The purpose of this is to have all of the escrow's money (ie all of the sidechain's money) in one TxID, so that depositors immediately undo any UTXO bloat they may cause. This simplifies the withdrawal process, as there is no need to worry about cleaning up "dust deposits" (...and such cleaning can often result in headaches, for example where a withdrawal-txn is larger than 1MB in size, or else may only withdraw an arbitrarily limited amount of BTC). Notice that, unless we assume that an account will last forever, all utxos which are deposited must eventually be withdrawn by someone. Therefore, the relevant design criterion is not "efficiency" (total network cost) but rather "who should pay" (allocation of costs). - -#### M5. "Make a Deposit" -- a transfer of BTC from-main-to-side - -As far as mainchain consensus is concerned, there are no additional requirements. - -However, in practice there *are* additional mainchain requirements...specified by the escrow account, (ie specified by the "sidechain" or "virtual chain"). These requirements are not part of mainchain consensus and are allowed to be anything. In other words, the sidechain is free to invent any way to credit depositor's money -- M5 is fully customizable. - -One method, is for mainchain depositors to append a zero-value OP Return to a Deposit txn, so that the sidechain knows how to credit funds. Mainchain users must upgrade their wallet software, of course, (on an individual basis) in order to become aware of and take advantage of new deposit-methods. - -##### Inconvenient Race Condition - -The requirement that each hashrate escrow be linked to a single TxID does create an interesting inconvenience for depositors. If a user is slow to sign a txn after constructing it (perhaps because the user employs an air-gapped computer, etc), then the signed txn may no longer be valid. This is because the input it selects, may no longer be the Critical TxID (as "the" Critical TxID changes with each deposit). **Only one user can deposit at a time** (although many can deposit per block). As a result, the transaction must fail, and the user would need to be prompted to remake and resign the txn. If this is problem is too frustrating, users can always make main-to-side transfers using atomic cross chain swaps (or, the LN, if they already have a channel open on both chains). - -Fortunately, it is already a part of mainchain consensus that no two txns can spend the same TxID. The only new issue here is the confusion it might create for the user (hence the need for error messages and alternative deposit-methods). - - -#### M6. "Execute Withdrawal" -- a transfer of BTC from-side-to-main - -We come, finally, to the critical matter: where users can take their money *out* of the escrow account, and return it to the "regular" UTXO set. As previously mentioned, this txn is one which (a) spends from a CTIP and (b) reduces the quantity of BTC in an account's CTIP. Most of the work has already been done by D1, M3, M4, and D2. Furthermore, existing Bitcoin tx-rules prevent the sidechain from ever withdrawing more money than has been placed into it. - -From there, we merely introduce two final concepts: - -1. In each block, an entry in D2 is considered an "approved candidate" if the "ACKs" value is above 13140. -2. A "blinded TxID" is way of hashing the txn, in which we first overwrite some parts of the txn with zeros. Specifically, the first 36 bytes of "TxIn0" (the first input, including TxOutHash and TxOutIndex), as well as the first 8 bytes of "TxOut0" (the first output). - -Blinding is necessary because we allow each sidechain only one UTXO at a time. - -of our restriction of the account to a single UTXO-member. Because of this, during the ACKing process the withdrawal-txn (which is currently being ACKed) may change in two ways: the CTIP (which changes with each deposit), and the total quantity of BTC stored in the account (which arbitrarily increases with each new deposit). In other words, a withdrawal-attempt is created via M3, but this takes place many blocks before the withdrawal is actually included via M6. During this time, a single new deposit to the account would change its CTIP and its value. So, what do we ACK? Well, we ACK a "blinded" version of the withdrawal. This blinded version is stable because the dynamic parts are always overwritten with zeros. - -While we ACK a blinded WT^, what is actually included in the blockchain ("M6") is an unblinded WT^. Since each blinded WT^ could correspond to many different unblinded WT^s, we need to impose further restrictions on those unblinded WT^s that are finally included. First, we will force the final unblinded WT^ to spend the entire sidechain balance (by forcing sum(input_values) to equal sum(output_values)). To avoid withdrawing the entire sidechain balance with every withdrawal, we will, secondly, force the unblinded WT^ to create a new output which is itself a deposit to the sidechain it withdrew from (which nodes can check using D1's CTIP field). Unfortunately, these requirements eliminate the possibility of including a transaction fee, as traditionally calculated. So, finally, to compensate for *that*, txn fees are encoded explicitly as a withdrawal to OP_TRUE (which the main:block's miner can immediately claim). - -With all of this in place, the only requirements for inclusion in a block are these: - -1. "Be ACKed" -- The "blinded TxID" of this txn must be member of the "approved candidate" set in the D2 of this block. -2. "Return Change to Account" -- TxOut0 must pay to the "critical account" (see D1) that corresponds to the CTIP that was selected as a TxIn. -3. "Return *all* Change to Account" -- Sum of inputs must equal the sum of outputs. No traditional tx fee is possible. - -Finally, don't forget that M6 inherits the requirement (common to both M5 and M6) that the CTIP be selected as an input, and that the CTIP then be updated. In this case, we know that the critical index will be zero, so the new CTIP will be ("this TxID" (NOT blinded), 0). The TxID is NOT blinded because blinding is only for accumulating ACKs. - -As a result of these requirements, every single withdrawal-attempt will fail, unless an entry has been added to D2 and "ACKed" a sufficient number of times. - - - -Backward compatibility -======================== - -As a soft fork, older software will continue to operate without modification. Non-upgraded nodes will see a number of phenomena that they don't understand -- coinbase txns with non-txn data, value accumulating in anyone-can-spend UTXOs for months at a time, and then random amounts leaving the UTXO in single, infrequent bursts. However, this phenomena doesn't affect them or the validity of the money that they receive. - -( As a nice bonus, note that the sidechains themselves inherit a resistance to hard forks. The only way to guarantee that the WT^s reported by different clients will continue to match identically, is to upgrade sidechains via soft forks of themselves. ) - - -Deployment -=========== - -This BIP will be deployed by "version bits" BIP9 with the name "hrescrow" and using bit 4. - -``` -// Deployment of Drivechains (BIPX, BIPY) -consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].bit = 4; -consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].nStartTime = 1515974401; // January 15th, 2018. -consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].nTimeout = 1547510401; // January 15th, 2019. -``` - -Reference Implementation -========================== - -See: https://github.com/drivechain-project/bitcoin/tree/mainchainBMM - -Also, for interest, see an example sidechain here: https://github.com/drivechain-project/bitcoin/tree/sidechainBMM - - -References -============ - -See http://www.drivechain.info/literature/index.html - - -Credits -========= - -Thanks to everyone who contributed to the discussion, especially: ZmnSCPxj, Adam Back, Peter Todd, Dan Anderson, Sergio Demian Lerner, Chris Stewart, Matt Corallo, Sjors Provoost, Tier Nolan, Erik Aronesty, Jason Dreyzehner, Joe Miyamoto, Ben Goldhaber. - - - -Copyright -========== - -This BIP is licensed under the BSD 2-clause license. -- cgit v1.2.3 From 4bea2ef2b67a996c040d026df61cdaf470c9de53 Mon Sep 17 00:00:00 2001 From: Paul Sztorc Date: Sat, 3 Feb 2018 21:40:43 -0500 Subject: one bip at a time --- bip-hashrate-escrows.md | 408 ------------------------------------ bip-hashrate-escrows/images.txt | 1 - bip-hashrate-escrows/two-groups.png | Bin 39695 -> 0 bytes 3 files changed, 409 deletions(-) delete mode 100644 bip-hashrate-escrows.md delete mode 100644 bip-hashrate-escrows/images.txt delete mode 100644 bip-hashrate-escrows/two-groups.png diff --git a/bip-hashrate-escrows.md b/bip-hashrate-escrows.md deleted file mode 100644 index 3b5d61e..0000000 --- a/bip-hashrate-escrows.md +++ /dev/null @@ -1,408 +0,0 @@ - -Header -======= - - BIP: ???? - Layer: Consensus (soft fork) - Title: Hashrate Escrows (Consensus layer) - Author: Paul Sztorc - CryptAxe - Comments-Summary: No comments yet. - Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-??????? - Status: Draft - Type: Standards Track - Created: 2017-08-14 - License: BSD-2-Clause - Post-History: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-May/014364.html - - -Abstract -========== - -A "Hashrate Escrow" is a clearer term for the concept of "locked to an SPV Proof", which is itself a restatement of the phrase "within a sidechain" as described in [a famous Oct 2014 paper](https://blockstream.com/sidechains.pdf) written partially by some Blockstream co-founders. - -A Hashrate Escrow resembles a 2-of-3 multisig escrow, where the 3rd party (who will arbitrate any disputes) is a decentralized group of people: the dynamic-membership set of Bitcoin Miners. However, the 3rd party does not sign escrow-withdrawal transactions with a private key. Instead, these are "signed" by directing hashpower over them for a period of time. - -This project has [a website](http://www.drivechain.info/) which includes [a FAQ](http://www.drivechain.info/faq/index.html). - - -Motivation -============ - -In practice these escrows are likely to be "asymmetric sidechains" of Bitcoin (such as [Rootstock](http://www.rsk.co/)) or "virtual chains" within Bitcoin (such as [proposed by Blockstack](https://github.com/blockstack/virtualchain) in mid-2016). - -Sidechains have many potential benefits, including: - -1. Protect Bitcoin from competition from altcoins and spinoffs. Safely allow competing implementations (of *sidechains*). -2. Protect Bitcoin from hard fork campaigns. (Such campaigns represent an existential threat to Bitcoin, as well as an avenue for developer corruption.) -3. Help with review, by making it much easier for reviewers to ignore bad ideas. -4. Provide an avenue for good-but-confusing ideas to prove their value safely. - - - -Specification -============== - - -#### Components - -Hashrate Escrows are built of two types of component: [1] new databases, and [2] new message-interpretations. - -##### 1. New Databases - -* D1. "Escrow_DB" -- a database of "accounts" and their attributes. -* D2. "Withdrawal_DB" -- a database of pending withdrawals from these accounts, and their statuses. - -Please note that these structures (D1 and D2) will not literally exist anywhere in the blockchain. Instead they are constructed from messages...these messages, in contrast, *will* exist in the blockchain (with the exception of M4). - -##### 2. New Messages - -* M1. "Propose New Escrow" -* M2. "ACK Escrow Proposal" -* M3. "Propose Withdrawal" -* M4. (implied) "ACK Withdrawal" -* M5. "Execute Deposit" -- a transfer of BTC from-main-to-side -* M6. "Execute Withdrawal" -- a transfer of BTC from-side-to-main - - -#### On the Resource Requirements of New Databases - -The "new" databases are simply reinterpretations of data that are already contained elsewhere in the blockchain. Specifically, M1 M2 and M3 are all located in the block's coinbase txn, and M5 and M6 might be found in any regular txn. M4 is a special case and does not actually need to be included anywhere, so it is not. If you like, you can imagine that the M4s reside in an optional extension block. - -In other words, we just rearrange what is already there. Because of this, even though "new databases" are created and stored in memory, the existing bandwidth and storage limits are respected (although, see "M4" below). - - - - -### Adding Sidechains and Tracking Them (D1, M1, M2) - -#### D1 -- "Escrow_DB" - -The table below enumerates the new database fields, their size in bytes, and their purpose. In general, an escrow designer (for example, a sidechain-designer), is free to choose any value for these. - -Note: Fields 6 through 9 have been intentionally removed. Previously, this section allowed miners to set and commit to voting/waiting periods. However, I have since standardized the periods: withdrawals expire after 6 months (26298 blocks), and they succeed if they ever achieve an ACK score of 13140 or higher. I have removed the waiting period, because anyone who adopts a policy of ignoring all withdrawals with fewer than 400 ACKs will automatically gain all of the benefits of the waiting period. The justification for this change is that it strongly implies that an attack on any one sidechain is an attack on all of them (in a sense, this change makes the "victimhood" of each sidechain "fungible"). - -Field No. | Label | Bytes | Type | Description / Purpose -----------|-------|------|------|-------- -1 | Escrow Number | 1 | uInt | A number assigned to the entire escrow. Used to make it easy to refer to each escrow. -2 | Active\* | 2 | uInt | This counter starts at zero when a new entry is added, and increments as a new entry is supported by "M2". The entry either reaches the max value of 4032 (at which point the counter freezes), or else the entry is removed. This enforces the requirement "a soft fork is required to add a new escrow". -3 | Escrow Name/Description | 120 | string | A human-readable name and description of the sidechain. More than enough space to hold a 32 byte hash. Helps prevent destructive interference among sidechains (see below). -4 | Critical Private Key | 32 | hex | This is the Double-SHA256 of the binary release of the reference software. It ties the sidechain to a specific software release, and doubles as a way of signing withdrawals (see below). -5 | Critical Address\* | 32 | string | This is derived by taking f(x) = ImportPrivateKey(HexToBase58Check(x)) of field #3. It is intentionally in the human-readable format {{although it could be 25 bytes of binary}}. -10 | "CTIP" -- Part 1 "TxID"\* | 32 | hex | The CTIP, or "Critical (TxID, Index) Pair" is a variable for keeping track of where the escrow's money is (ie, which member of the UTXO set). -11 | "CTIP" -- Part 2 "Index"\* | 4 | hex | Of the CTIP, this is second element of the pair: the Index. See #10 above. - - -\* Denotes a "convenience field", the entry for this field is derived from other fields, or from the blockchain-state itself. The escrow-creator does not need to provide these values in M1 (or anywhere). - -Escrow_DB requires 223 bytes [1+120+32+32+2+32+4] for each escrow in the main blockchain. Of these, 70 bytes [2+32+32+4] are merely for convenience. Therefore, a sidechain is defined (see "M1") by 153 bytes of information. - -#### Notes on D1 - -1. D1 will always exist. -2. D1 will always have a unique sort (first by "Escrow Number" (field #1), and second by "Active" (field #2)). There is only ever one (escrow number, Active) pair. -3. D1 is updated according to M1 and M2 (below). -4. If a new entry is added to D1 with an "Escrow Number" that is already in use, then this entry will either eventually be removed (because it was not supported with an M2), or it will eventually overwrite the old entry (if it *was* supported via M2). - - -#### Notes on D1 - -##### Obligations Placed on Miners - -Miners have always upgraded their software according to criteria that are known only to them (in other words, "whenever they want"). - -However, this soft fork imposes two new criteria upon them. First: miners should only upgrade their software, if any modification to the portfolio of sidechains [that are added/removed in the upgrade] can be expected to increase miner wealth. Trivially, this implies that miners should make sure that the upgrade doesn't overwrite (and destroy) an existing sidechain that they like! But, more seriously, it implies that miners should take an interest in what the sidechain is doing to the mainchain and other sidechains (see below). - -##### Destructive Sidechain Interference - -People frequently emphasize that miners should have "as little control" as possible. It is a very safe claim to make, and a very easy sentence to write. Much harder is to determine exactly what this minimum value is, and how to achieve it. Harder still is to untie the knot of who is actually controlling what, in a decentralized, interacting system. - -Certainly, miners can not have "zero control" -- for that is the same as to just remove them from the system altogether. Some rules are enforced "on miners by nodes" (such as the infamous blocksize limit); other rules are enforced by nodes but are narrowly-controlled by miners (such as the proof-of-work itself, or the block's timestamp). Thirdly, some rules are enforced by both against each other (such as the rule against including invalid txns or double-spent txns), for mutual benefit. - -Some pause should be given, after one considers that the sidechain design goal is literally a piece of software that can do *anything*. Anything includes a great many things, many of which I demonstrate to be undesirable. Bitcoin itself does not allow "anything" -- it allows any person to transact, but, in contrast, it does not permit any person to double-spend. This is because "allowing anyone to do anything" is not viable in a world that contains undesirable interactions (what a libertarian might call "aggression") -- in the case of money, these are theft and counterfeiting. - -I have produced a comprehensive quantity of written material [1], presentations [2], etc [3] on exactly what the level of miner-control should be, and why. Specifically, I claim that **miners should be aware of the purpose of the sidechain, and they should reject sidechains which have an unclear purpose or which have a purpose that will lead to decrease in miner-wealth** (where wealth measured explicitly as: the estimated present value of the purchasing power of the blockchain's coinbase txns). I claim that this criterion is necessary because, just Original Bitcoin filters unwanted interactions among different BTC txns, so too much "Sidechain Bitcoin" filter out unwanted interactions among sidechain. - -* [1] http://www.truthcoin.info/blog/wise-contracts/ -* [2] https://www.youtube.com/watch?v=xGu0o8HH10U&index=1&list=PLw8-6ARlyVciMH79ZyLOpImsMug3LgNc4 -* [3] http://www.drivechain.info/literature/index.html - -Call it a "sidechain non-aggression principle", if you want. - -To the best of my knowledge, everyone who *has* reviewed this information as found the arguments to be acceptable. It has, also, changed a few minds (from "unacceptable" to "acceptable"). - - -##### ISSUE: "Signing" BTC Txns - -Currently, we use a process which may be suboptimal. It is that we *literally sign* a txn with a globally and publicly known private key. But this is for convenience purposes -- the signature that is produced is not doing anything, and is therefore wasteful. Instead we may use OP_TRUE, but this might interfere with how we detect the sidechain's balance. I'm not sure what the best way is. Someone needs to investigate how to do this -- removing OP_CheckSig, etc. This is a TODO for sure, and an opportunity for someone to help. - - - -(The following messages were modeled on SegWit -- https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki#commitment-structure ) - - - -#### M1 -- "Propose New Sidechain" - - 1-byte - OP_RETURN (0x6a) - 1-byte - Push the following 157 bytes (0x9d) - 4-byte - Commitment header (0x53707243) - 153-byte - the critical bytes mentioned above (fields #1, #3, and #4, to populate a new D1 entry) - - -#### New Block Validation Rules - -1. If the network detects a properly-formatted M1, it must add an entry to D1, into the very next block, with the following initial values: -* Field #5 will be calculated as per [version 1 Bitcoin addresses](https://en.bitcoin.it/wiki/Technical_background_of_version_1_Bitcoin_addresses), but with a prefix of "4" instead of "1". -* Field #9 will be derived from #7 and #8 using math. -* The initial values of Fields #10, #11, and #12 are set to zero. -2. Only one M1 (of any kind) can be added into a block at a time. - -#### Notes on M1 - -The escrow will "reuse" the same address over and over. But notice that there is no privacy loss due to address reuse, because the address in question does not "belong" to any specific individual. Furthermore, the transactions in question could already be linked by just watching the Hashrate Escrow for deposits ...this, in fact, is exactly what a sidechain must do in order to make main-to-side transfers. - - -#### M2 -- "ACK Sidechain Proposal" - - 1-byte - OP_RETURN (0x6a) - 1-byte - Push the following 29 bytes (0x1D) - 4-byte - Commitment header (0x53616343) - 25-byte - Commitment hash: RIPMD-160 of Sha256 of a given M1 (above) - -#### New Block Validation Rules - -1. An entry will be removed from D1, unless that block contains an M2 supporting it (this will increment field #2), or the entry has accumulated 4032 M2s. Therefore, each entry needs to have 4032 *consecutive* M2s supporting it, else it will be removed from D1 and must start over. This requires all miners to agree to "soft fork" the sidechain into existence. This makes it easier to curate the list of active sidechains, and prevents destructive interference (see above) among contradictory sidechains. -2. Once the entry has 4032 M2s, then a miner must make a 0 btc payment to the Critical Address, thus activating the chain. This payment becomes the first "critical (TxID, index) pair" (or "CTIP"). -3. It is possible to "overwrite" a D1 slot -- this destroys the sidechain that is there, and replaces it with a new one. This is defined as requiring 6 months (26298 blocks) of M2s, instead of 4 weeks (4032). This possibility does not change the security assumptions, because we already assume that users have access to "magically true" bandwidth at a rate of 1 bit per ACK-cycle. (This is why the voting cycles should be large, 1 month Waiting, 2+ months Voting). - - - -### Adding Withdrawal-Attempts and Tracking Their Success (D2, M3, M4) - -#### D2 -- "Withdrawal_DB" - -The table below enumerates the database fields, their size (in bytes), type and purpose. - - -Field No. | Label | Bytes | Type | Description / Purpose -----------|-------|------|------|----------------------- -1 | Escrow Number | 1 | uInt | Links the withdrawal-request to a specific escrow. -2 | WT^ | 32 | hex | This is a "blinded transaction id" (ie, the double-Sha256 of a txn that has had two fields zeroed out, see M6) of a withdrawal-attempt. -3 | ACKs\* | 2 | uInt | The current total number of "votes", this starts at 0 and remains there throughout the waiting period. -4 | Age\* | 3 | uInt | Total duration of time, in blocks, that this WT^ has been inside of D2. -5 | Waiting Period\* | 2 | uInt | Total duration of time, in blocks, that this entry must sit idle, before it can begin to accumulate ACKs/NACKs. Pulled from D1's field #6. -6 | Max Age\* | 3 | uInt | Determined by summing (D1's field #6) and (D1's field #7). -7 | Threshold\* | 2 | uInt | Total ACKs needed, this is pulled from D1's field #9. -8 | Approved\* | 1 | boolean | True while ACKs > Threshold, False otherwise. - -\* Denotes a "convenience field" (see above). - -Withdrawal_DB requires 46 bytes [1+32+2+3+2+3+2+1] per entry. Of these, 13 bytes ([2+3+2+3+2+1], all fields except #1 and #2) can be generated locally, leaving 33 critical bytes [1+32]. - -#### New Block Validation Rules for D2 - -1. In each block, a hash commitment to D2 must always exist (even if D2 is blank). -2. D2 must always be sorted first by field #1 (Escrow Number) and second by field #4 (Age). This imposes a unique sort. -3. From one block to the next, every entry's "Age" field must increase by exactly 1. -4. From one block to the next, entries are only removed from D2 (in the very next block) if: -* * "Age" = "MaxAge". -* * If the block contains a txn who's blinded txID matches WT^. {{ This might be unnecessary, and a lot of work. }} -5. In addition, there are special rules for the allowed values in the "ACKs" field (field #3). See M4 below. - -#### M3 -- "Propose Withdrawal" - - 1-byte - OP_RETURN (0x6a) - 1-byte - Push the following 37 bytes (0x25) - 4-byte - Commitment header (0xD45AA943) - 33-byte - the critical bytes mentioned above (fields #1 and #2, to populate a new D2 entry) - - -#### New Block Validation Rules for M3 - -1. If the network detects a properly-formatted M3, it must add an entry to D2 in the very next block. The starting values of fields #3 and #4 are zero, and #5 is pulled over by extracting the relevant value from D1. -2. Each block can only contain one M3 per sidechain. - - -#### M4 -- "ACK Withdrawal" - -#### Very Little Info, Probably Calculable in Advance - -M4 is exceptional (in comparison to the other M's) in a few ways. First, its content is not stored anywhere, only the *hash* of its *effect* is stored (in a leaf of a merkle tree who's root is inserted into a mainchain coinbase). M4 alters the contents of D2 -- the *contents* of D2 are consensus critical, but M4 (the process by which nodes reach a new valid D2) can be anything. - -In fact, M4 can also be *nothing*. In other words, it may be optional. This is precisely because, from one block to the next, we have constrained D2 such that it is only allowed to change in a few ways. Therefore, the exhaustive set of "candidate D2s" can be precomputed by full nodes in advance. - -#### Two Withdrawals at Once - -In general, only one withdrawal (per sidechain) can make progress (toward being included in a block) at a time. In other words, as WT^s are proposed, only one can make progress toward the finish line. As a result, a given side-to-main transfer will always take between 3 and 6 months. Instead, with more simultaneous withdrawals, the worst-case transfer duration would improve. - -![dots-image](/bip-hashrate-escrows/two-groups.png?raw=true) - -The worst-case withdrawal time obeys f(n)=3+(3/n) months, where n is the number of simultaneous withdrawals. - -N=2 is the most desirable choice for several reasons. First, it delievers the greatest marginal benefit (of 1.5 months). Later choices only deliver 0.5 and 0.25 marginal months. - -Second, n=2 can be implemented in a clever way: by allowing a withdrawal to freely advance, if and only if has an ACK-score of 6575 or greater, and if it also has the largest ACK score. In other words, the withdrawal that is furthest along can advance (or retreat) for free, if it has already made it at least halfway to the finish line. With this change, our new M4, is either an "abstain" for the sidechain (in which case nothing happens to any ACK scores), or else it will be in one of two cases: old_M4 + "the largest advances", or new_M4 + "the largest retreats". As a result the number of M4 possibilities (of which the next section is concerned) only increases by a factor of two (instead of exponentially). - -It is possible to troll this rule, by getting two (or even three) withdrawals to have 6575+ ACK scores, and then getting them to *tie* for first place. So, if there are any ties, the ability to "bonus move" is disabled until all ties are broken. - -#### How Hard is it to Guess M4? - -If there are n Escrows and m Withdrawals-per-escrow1, then there are (m+2)^n total candidates for the next D2. This is because, [per block per escrow], one of three things can happen: (1) one of the m withdrawal-candidates can be "ACK"ed (or "upvoted" or "promoted"), which automatically downvotes the others; or (2) all withdrawal-candidates can be downvoted, or finally (3) the miners can abstain from voting on the escrow's withdrawals altogether, leaving the tallies the same. - -First, for nodes which validate all sidechains (assuming these escrows are sidechains), this simplifies to 2^n -- these nodes only have to choose between the single honest choice (on one hand) or an abstention (on the other). Second, even for nodes that don't validate any sidechains, the number of candidates might be reduced from m^n to 3^n, by making a simplifying assumption: whichever withdrawal was most recently added/upvoted, is likely to be the one which is upvoted next. - -Of course, that is still O(k^n) for n sidechains, which isn't great2. If the "D2 update" cannot be guessed, it must be transmitted in some way. - -#### Giving Up and Getting M4 the Old Fashioned Way - -Two examples for transmitting it are below: - -"Short Form" (Assumes there are no more than 254 active withdrawal-attempts per account) - - 4-byte - Message identifier (0x????????) - 1-byte - Version of this message - N-byte - N is the total number of active accounts ("sidechains"), each byte specifies the position of the single WT that was "upvoted". A value of 0 indicates "downvote everything", a value of 255 indicates abstention. - -"Long Form" (Makes no assumptions about anything) - - 4-byte - Message identifier (0x????????) - 1-byte - Version of this message - 1-byte - Length (in bytes) of this message; total number of withdrawal attempts; y = ceiling( sum_i(m_i +2)/8 ). Nodes should already know what length to expect, because they know the sequence of M3s and therefore the vector of WT^s. - Y-byte - stream of bits (not bytes), with a 1 indicating the position of the chosen action [downvote all, abstain, upvote1, upvote2, ...] - - -If the message is very very large, then nodes may not want to broadcast it. This opens up an "exhaustion attack"2, in which many miners create bad WT^s, vote on these randomly, and then refuse to broadcast their votes. Fortunately, even for a worst-case scenario of 200 sidechains and 1,000 withdrawal-attempts per sidechain, honest nodes can communicate a long form M4 with each other by using just 25,056 bytes per block [4+1+1+(200\*(1000+1+1)/8)]. - -Today's pre-drivechain miners can already carry out a similar attack, by creating and including txns and then not broadcasting that part of the block to anyone. This is often characterized as a ["block publication incentive"](https://petertodd.org/2016/block-publication-incentives-for-miners), because in that case the prospect of exhaustively computing all possible transactions (to uncover the missing ones) is completely out of the question. - -However, message M4 is different from a withheld-txn, because M4 operates outside of the block's mandated information-processing limits (ie, outside the infamous 1 MB nonwitness blocksize limit). So we should examine the conditions under which M4 grows and shrinks, to ensure that we are not smuggling in a tremendous burden on full nodes. - -Under adversarial conditions, to lengthen a long-form M4 by one bit per block, for C blocks, the attacker must pay 312 bits (39 bytes) one time (to embed a new M3 message). The value C is the length of the sidechain's voting period, which varies but which I expect to be approximately 8,064 (and which could theoretically be as high as 65,536). Thus the attacker can burden nodes disproportionately, if (s)he wishes. - -Fortunately, the attack in question has no motivation (as far as I can tell). If the miner's goal is to trick rivals into mining on top of invalid blocks, he can already do this much more effectively with the unpublished-txn method (above). If instead he is just trying to harass nodes, then nodes may freely "downgrade" to earlier versions of the protocol, and simply ignore all drivechain-related messages. It seems that the attack could best be used in order to: make a large D2, make D2 confusing, sneak in votes for evil WT^ lurking in D2. Thus, the attack disables the transparency of the drivechain system, to some extent. The cost of the attack is forgone transaction fees, due to block space wasted on useless M3s. - -In practice, n is already capped, and miners may impose [on each other] a "soft cap" on m for their mutual protection. Thus, n and m might never get above 10 and 30, respectfully. In this case, the [Short Form, this time] M4 can never require more than 15 bytes per block, no matter what the attacker tries. - -In practice, m should always be 1 or 2, else something fishy is going on; and m can only inch up by 1 unit per block. So the system as a whole is still quite transparent, in that users are warned appropriately and well in advance. Attackers must invest upfront and they face an uphill climb, in order to eventually make things more expensive for a few others; defenders can wait-and-see if the attack looks like it will ever amount to anything before lifting a finger. - - -##### New Block Validation Rules (for D2 and, by implication, M4) - -From one block to the next, D2 can only be edited in a few strict ways: - -* Entries can only be added/removed from D2 if they meet the criteria above (in M3, and implicitly M1 and M2). -* The ACK-counter of any individual entry can only change by (-1,0,+1) relative to its previous entry. -* Within a sidechain group, upvoting one withdrawal (ACK=ACK+1) requires you to downvote all other withdrawals in that group. However, the minimum ACK value is zero (and, therefore, downvotes cannot reduce it below zero). - -##### Footnotes for M4 - -1 This represents the worst-case scenario is one where all the Withdrawals are spread evenly over each Sidechain. Under normal operations, there is no reason to expect the all sidechains will have the same number of withdrawals at any given time. In fact, under normal operations, the very *concept* of counting the withdrawals-per-sidechain should be a purposeless one, because there should only be *one* withdrawal at a time. Nonetheless we consider the worst case scenario here. - -2 Guessing becomes more computationally intensive in a highly adversarial situation where the "limited range" is intentionally expanded. In such a scenario, [a] there are many sidechains, and [b] miners voluntarily sacrifice their scarce block-space by creating a high number of (mutually-exclusive, and hence ultimately invalid) withdrawal attempts and putting these into coinbase transactions; and then agree to all [c] vote on these randomly (guaranteeing that all withdrawals fail, including any true withdrawals) and [d] successfully withhold their random voting strategies from nodes (even including spy-miner-nodes). Under this bizarre scenario, nodes may require computing resources which increase near-exponentially with the number of withdrawals, and it may take a long time for an ignorant node to exhaustively work out the underlying state of Withdrawal_DB. In this case, nodes may decide to temporarily stop validating such transactions (as if they had not yet upgraded to support this soft fork). - - - -### Depositing and Withdrawing (M5, M6) - - -Both M5 and M6 are regular Bitcoin txns. They are identified by meeting an important criteria: they select a one of the Critical TxID-index Pairs (a "CTIP") as one of their inputs. Deposits ("M5") are distinguished from withdrawals ("M6") by simply checking to see if money is "going in", or "out". In other words, we compare the BTC value of the original CTIP to that of new CTIP. If original <= new it is a deposit, if original > new then it is a withdrawal. - -The code that identifies sidechain withdrawal / deposit txns (by calculating how much value is being put into or taken out of a sidechain) can be seen here: https://github.com/drivechain-project/bitcoin/blob/mainchainBMM/src/validation.cpp#L351-L386 - -Such txns are forced (by consensus) to obey two additional criteria: - -1. They must contain an output paying "to" the Critical Address [probably in TxOut0]. -2. They must be accompanied by an update to this sidechain's Critical TxID-index Pair (CTIP). The new CTIP must be "this" txn itself. - -These criteria are enforced here https://github.com/drivechain-project/bitcoin/blob/mainchainBMM/src/validation.cpp#L440-L473 by checking that a deposit is paying back to the sidechain more than it is taking out, and completely rejecting any withdrawal from the mempool. And here https://github.com/drivechain-project/bitcoin/blob/mainchainBMM/src/validation.cpp#L1747-L1757 we allow for a withdrawal only once it has attained sufficient work score (ACKs). - -The purpose of this is to have all of the escrow's money (ie all of the sidechain's money) in one TxID, so that depositors immediately undo any UTXO bloat they may cause. This simplifies the withdrawal process, as there is no need to worry about cleaning up "dust deposits" (...and such cleaning can often result in headaches, for example where a withdrawal-txn is larger than 1MB in size, or else may only withdraw an arbitrarily limited amount of BTC). Notice that, unless we assume that an account will last forever, all utxos which are deposited must eventually be withdrawn by someone. Therefore, the relevant design criterion is not "efficiency" (total network cost) but rather "who should pay" (allocation of costs). - -#### M5. "Make a Deposit" -- a transfer of BTC from-main-to-side - -As far as mainchain consensus is concerned, there are no additional requirements. - -However, in practice there *are* additional mainchain requirements...specified by the escrow account, (ie specified by the "sidechain" or "virtual chain"). These requirements are not part of mainchain consensus and are allowed to be anything. In other words, the sidechain is free to invent any way to credit depositor's money -- M5 is fully customizable. - -One method, is for mainchain depositors to append a zero-value OP Return to a Deposit txn, so that the sidechain knows how to credit funds. Mainchain users must upgrade their wallet software, of course, (on an individual basis) in order to become aware of and take advantage of new deposit-methods. - -##### Inconvenient Race Condition - -The requirement that each hashrate escrow be linked to a single TxID does create an interesting inconvenience for depositors. If a user is slow to sign a txn after constructing it (perhaps because the user employs an air-gapped computer, etc), then the signed txn may no longer be valid. This is because the input it selects, may no longer be the Critical TxID (as "the" Critical TxID changes with each deposit). **Only one user can deposit at a time** (although many can deposit per block). As a result, the transaction must fail, and the user would need to be prompted to remake and resign the txn. If this is problem is too frustrating, users can always make main-to-side transfers using atomic cross chain swaps (or, the LN, if they already have a channel open on both chains). - -Fortunately, it is already a part of mainchain consensus that no two txns can spend the same TxID. The only new issue here is the confusion it might create for the user (hence the need for error messages and alternative deposit-methods). - - -#### M6. "Execute Withdrawal" -- a transfer of BTC from-side-to-main - -We come, finally, to the critical matter: where users can take their money *out* of the escrow account, and return it to the "regular" UTXO set. As previously mentioned, this txn is one which (a) spends from a CTIP and (b) reduces the quantity of BTC in an account's CTIP. Most of the work has already been done by D1, M3, M4, and D2. Furthermore, existing Bitcoin tx-rules prevent the sidechain from ever withdrawing more money than has been placed into it. - -From there, we merely introduce two final concepts: - -1. In each block, an entry in D2 is considered an "approved candidate" if the "ACKs" value is above 13140. -2. A "blinded TxID" is way of hashing the txn, in which we first overwrite some parts of the txn with zeros. Specifically, the first 36 bytes of "TxIn0" (the first input, including TxOutHash and TxOutIndex), as well as the first 8 bytes of "TxOut0" (the first output). - -Blinding is necessary because we allow each sidechain only one UTXO at a time. - -of our restriction of the account to a single UTXO-member. Because of this, during the ACKing process the withdrawal-txn (which is currently being ACKed) may change in two ways: the CTIP (which changes with each deposit), and the total quantity of BTC stored in the account (which arbitrarily increases with each new deposit). In other words, a withdrawal-attempt is created via M3, but this takes place many blocks before the withdrawal is actually included via M6. During this time, a single new deposit to the account would change its CTIP and its value. So, what do we ACK? Well, we ACK a "blinded" version of the withdrawal. This blinded version is stable because the dynamic parts are always overwritten with zeros. - -While we ACK a blinded WT^, what is actually included in the blockchain ("M6") is an unblinded WT^. Since each blinded WT^ could correspond to many different unblinded WT^s, we need to impose further restrictions on those unblinded WT^s that are finally included. First, we will force the final unblinded WT^ to spend the entire sidechain balance (by forcing sum(input_values) to equal sum(output_values)). To avoid withdrawing the entire sidechain balance with every withdrawal, we will, secondly, force the unblinded WT^ to create a new output which is itself a deposit to the sidechain it withdrew from (which nodes can check using D1's CTIP field). Unfortunately, these requirements eliminate the possibility of including a transaction fee, as traditionally calculated. So, finally, to compensate for *that*, txn fees are encoded explicitly as a withdrawal to OP_TRUE (which the main:block's miner can immediately claim). - -With all of this in place, the only requirements for inclusion in a block are these: - -1. "Be ACKed" -- The "blinded TxID" of this txn must be member of the "approved candidate" set in the D2 of this block. -2. "Return Change to Account" -- TxOut0 must pay to the "critical account" (see D1) that corresponds to the CTIP that was selected as a TxIn. -3. "Return *all* Change to Account" -- Sum of inputs must equal the sum of outputs. No traditional tx fee is possible. - -Finally, don't forget that M6 inherits the requirement (common to both M5 and M6) that the CTIP be selected as an input, and that the CTIP then be updated. In this case, we know that the critical index will be zero, so the new CTIP will be ("this TxID" (NOT blinded), 0). The TxID is NOT blinded because blinding is only for accumulating ACKs. - -As a result of these requirements, every single withdrawal-attempt will fail, unless an entry has been added to D2 and "ACKed" a sufficient number of times. - - - -Backward compatibility -======================== - -As a soft fork, older software will continue to operate without modification. Non-upgraded nodes will see a number of phenomena that they don't understand -- coinbase txns with non-txn data, value accumulating in anyone-can-spend UTXOs for months at a time, and then random amounts leaving the UTXO in single, infrequent bursts. However, this phenomena doesn't affect them or the validity of the money that they receive. - -( As a nice bonus, note that the sidechains themselves inherit a resistance to hard forks. The only way to guarantee that the WT^s reported by different clients will continue to match identically, is to upgrade sidechains via soft forks of themselves. ) - - -Deployment -=========== - -This BIP will be deployed by "version bits" BIP9 with the name "hrescrow" and using bit 4. - -``` -// Deployment of Drivechains (BIPX, BIPY) -consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].bit = 4; -consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].nStartTime = 1515974401; // January 15th, 2018. -consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].nTimeout = 1547510401; // January 15th, 2019. -``` - -Reference Implementation -========================== - -See: https://github.com/drivechain-project/bitcoin/tree/mainchainBMM - -Also, for interest, see an example sidechain here: https://github.com/drivechain-project/bitcoin/tree/sidechainBMM - - -References -============ - -See http://www.drivechain.info/literature/index.html - - -Credits -========= - -Thanks to everyone who contributed to the discussion, especially: ZmnSCPxj, Adam Back, Peter Todd, Dan Anderson, Sergio Demian Lerner, Chris Stewart, Matt Corallo, Sjors Provoost, Tier Nolan, Erik Aronesty, Jason Dreyzehner, Joe Miyamoto, Ben Goldhaber. - - - -Copyright -========== - -This BIP is licensed under the BSD 2-clause license. diff --git a/bip-hashrate-escrows/images.txt b/bip-hashrate-escrows/images.txt deleted file mode 100644 index ece458d..0000000 --- a/bip-hashrate-escrows/images.txt +++ /dev/null @@ -1 +0,0 @@ -Images used in this bip. diff --git a/bip-hashrate-escrows/two-groups.png b/bip-hashrate-escrows/two-groups.png deleted file mode 100644 index c8a3ffa..0000000 Binary files a/bip-hashrate-escrows/two-groups.png and /dev/null differ -- cgit v1.2.3 From 39a8bec6b7b7ca7e7fa405e7f43703684f5fa1e8 Mon Sep 17 00:00:00 2001 From: Paul Sztorc Date: Sat, 3 Feb 2018 21:42:45 -0500 Subject: Delete images.txt --- bip-hashrate-escrows/images.txt | 1 - 1 file changed, 1 deletion(-) delete mode 100644 bip-hashrate-escrows/images.txt diff --git a/bip-hashrate-escrows/images.txt b/bip-hashrate-escrows/images.txt deleted file mode 100644 index ece458d..0000000 --- a/bip-hashrate-escrows/images.txt +++ /dev/null @@ -1 +0,0 @@ -Images used in this bip. -- cgit v1.2.3 From 3246096174020bdcfcdf464c43b0ec997410f580 Mon Sep 17 00:00:00 2001 From: Paul Sztorc Date: Sat, 3 Feb 2018 21:42:52 -0500 Subject: Delete two-groups.png --- bip-hashrate-escrows/two-groups.png | Bin 39695 -> 0 bytes 1 file changed, 0 insertions(+), 0 deletions(-) delete mode 100644 bip-hashrate-escrows/two-groups.png diff --git a/bip-hashrate-escrows/two-groups.png b/bip-hashrate-escrows/two-groups.png deleted file mode 100644 index c8a3ffa..0000000 Binary files a/bip-hashrate-escrows/two-groups.png and /dev/null differ -- cgit v1.2.3 From 8a8c6253bef170b57c91c8e129d977ae27e3f530 Mon Sep 17 00:00:00 2001 From: Paul Sztorc Date: Sat, 10 Feb 2018 14:01:25 -0500 Subject: re-reverse the bips this is what I originally intended, but I forked this branch at the wrong time --- bip-blind-merged-mining.md | 329 ------------------- bip-blind-merged-mining/bmm-dots-examples.png | Bin 41116 -> 0 bytes bip-blind-merged-mining/images.txt | 1 - bip-blind-merged-mining/witness-vs-critical.png | Bin 67570 -> 0 bytes bip-hashrate-escrows/images.txt | 1 + bip-hashrate-escrows/two-groups.png | Bin 0 -> 39695 bytes hashrate-escrows.md | 408 ++++++++++++++++++++++++ 7 files changed, 409 insertions(+), 330 deletions(-) delete mode 100644 bip-blind-merged-mining.md delete mode 100644 bip-blind-merged-mining/bmm-dots-examples.png delete mode 100644 bip-blind-merged-mining/images.txt delete mode 100644 bip-blind-merged-mining/witness-vs-critical.png create mode 100644 bip-hashrate-escrows/images.txt create mode 100644 bip-hashrate-escrows/two-groups.png create mode 100644 hashrate-escrows.md diff --git a/bip-blind-merged-mining.md b/bip-blind-merged-mining.md deleted file mode 100644 index ee81f1f..0000000 --- a/bip-blind-merged-mining.md +++ /dev/null @@ -1,329 +0,0 @@ - Drivechain Documentation -- Blind Merged Mining BIP - Paul Sztorc - November 17, 2017 - Document 3 of 3 - v4.1 - - -Header -======= - - BIP: ???? - Layer: Consensus (soft fork) - Title: Blind Merged Mining (Consensus layer) - Author: Paul Sztorc - CryptAxe - Chris Stewart - Comments-Summary: No comments yet. - Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-??????? - Status: Draft - Type: Standards Track - Created: 2017-10-24 - License: BSD-2-Clause - - -Abstract -========== - -Blind Merged Mining (BMM) is a way of mining special extension blocks, ie "sidechains". It produces strong guarantees that the block is valid, for *any* arbitrary set of rules; and yet it does so without requiring miners to actually do any validation on the block whatsoever. - -BMM actually is a process that spans two or more chains. For an explanation of the "whole picture", please see [this post](http://www.truthcoin.info/blog/blind-merged-mining/). Here we focus on the modifications to mainchain Bitcoin. - -To support BMM, the mainchain is asked to accomplish two goals: -1. Track a set of ordered hashes (the merged-mining). -2. Allow miners to "sell" the act of finding a sidechain block (through the use of a new extended serialization transaction type). - -These goals are accomplished by forcing nodes to validate two new messages (M7, M8), and track data in one new database (D3). - - -Motivation -============ - -Regular "Merged-Mining" (MM) allows miners to reuse their hashing work to secure other chains (for example, as in Namecoin). However, traditional MM has two drawbacks: - -1. Miners must run a full node of the other chain. (This is because [while miners can effortlessly create the block] miners will not create a valid payment to themselves, unless the block that they MM is a valid one. Therefore, miners must assemble a *valid* block first, then MM it.) -2. Miners are paid on the other chain, not on the regular BTC mainchain. For example, miners who MM Namecoin will earn NMC (and they will need to sell the NMC for BTC, before selling the BTC in order to pay for electricity). - -Blind Merged-Mining (BMM) attempts to address those shortcomings. - - -Specification -============ - -Note: This document uses the notation side:\* and main:\* in front of otherwise-ambiguous words (such as "block", "node", or "chain"), to distinguish the mainchain version from its sidechain counterpart. - -As stated above, we have two goals: [1] create and monitor an alt-chain (defined only by a deterministic list of hashes), and [2] allow miners to "sell" the act of finding a sidechain block (through the use of a new extended serialization transaction type). - -### Sidechain Critical Data ("Sidechain Mini-Header") - -Specifically, per side:block per side:chain, we track the following 35 bytes of information: - - 1-byte - ChainIndex (known as "Account Number" in hashrate-escrows.md , or as "Sidechain Number") - 32-bytes - sideHeaderHash (also known as "h*" / hashCritical, the hash of the sidechain block) - 2-bytes - prevBlockRef (an index which points to this side:block's parent side:block) - -The **ChainIndex** indicates which sidechain this critical data is relevant to. As we may eventually have more than one sidechain, this serves as an identifier similar to the Bitcoin network's magic bytes (0xF9BEB4D9). Drivechains however only need to use 1 byte for the identifier (there is a hard limit of 256 sidechains identified as 0-255). The **sideHeaderHash** is the hash of a side:block which will receive PoW via BMM. It serves a similar function to Bitcoin's "hashMerkleRoot", in that it contains the data for its blockchain. The **prevBlockRef** forms the set of headers into a blockchain structure by making each headers refer to one parent header. It is most similar to Bitcoin's hashPrevBlock. - -Where does this data come from, and how does it get around? - -#### Creating / Broadcasting This Data - -##### Creation - -By the time Blind Merged Mining can take place, the ChainIndex is globally known (it is the "Account Number" in D1 [see previous BIP], and "nSidechain" in the code). Each sidechain, when activated by soft fork, will take one of the 0-255 available indexes. - -The other two items, sideHeaderHash and prevBlockRef, are created by sidechain nodes. sideHeaderHash is quite straightforward -- side:nodes build side:blocks, and take the hash of these. - -The final item, prevBlockRef, is a little more complicated. It is an integer that counts the number of "skips" one must take in the side:chain in order to find the current side:block's parent block. In practice, this value will usually be zero. It will only be a value other than zero, in cases where invalid sidechain blocks have been mined, or when a side:node intentionally wants to orphan some side:blocks (if a side:node wants to orphan the most-recent N blocks, the value of the current block will be equal to N ; in the block after that it will be back to zero). - -![dots-image](/bip-blind-merged-mining/bmm-dots-examples.png?raw=true) - -Since the hashes themselves are already ordered by the mainchain, tracing the blockchain's path by index (prevBlockRef) will be the same as tracing it by identifying a list of hashes. In other words, the ordering given via each side:block's "prevBlockRef" will be isomorphic to an ordering given by each side:block's "prevSideHeaderHash" ... if "prevSideHeaderHash is defined to be the sidechain's equivalent of the mainchain's "prevBlockHash". It will be possible to freely convert from one to the other. See M8 to learn more about how these hashes are requested by sidechain block creators to be included in the mainchain. - -Now that we know what our critical data is, and how it is made, how is this data broadcast and stored? - -##### Broadcast - -Mainchain nodes are going to need this data later, so it must be easy to find. We will put it into the coinbase via OP RETURN. - -#### M7 -- "Blind-Mine the Sidechain(s)" - -Thus, (for n sidechains) we have a coinbase output with: - - 1-byte - OP_RETURN (0x6a) - 1-byte - Push the following (4+(n*35)) bytes (0x??) - 4-byte - Message header (0xD3407053) - (n*(32+5))-byte - A sequence of bytes, of the three Mini-Header items (h*, prevBlockRef, ChainIndex). - -( We assume that 5 bytes are used for the Critical Data bytes (non h* parts of the Sidechain Mini-Header). For 256 sidechains, a total of 9,478 bytes [1+1+4+256\*(32+5)] are required, conveniently just below the 10 KB scriptPubKey size limit.) - -This data is parsed by laying it in sequential 37-byte chunks (any remaining data --ie, some final chunk that is less than 37 bytes in length-- has no consensus meaning). - -Each 37-byte chunk is then parsed to obtain the data outlined above (in "Description"). If two 35-byte chunks being with the same "Sidechain number" (ie, if the two chunks have the same first byte), then only the first chunk has consensus meaning. - -We are left with, at most, one (h*, prevBlockRef) pair per sidechain per block. This data is added directly to D3, a new database. - -#### D3 -- "RecentSidechains_DB" - -To suit our purposes, the mainchain full nodes will need to keep track of the most recent 8000 (h\*, prevBlockRef) pairs. - -( This 8,000 figure is a tradeoff between decentralization (costs of running the main:node) and sidechain security -- it requires attackers to merged-mine 8,000 invalid blocks consecutively, in order to cause the sidechain to fail. The mainchain burden is minimal, so this figure might be raised to 12,000 or higher. ) - -Therefore, D3 would look something like this: - - - BlockHeight CB_Index SC_1 Blks_Atop_1 SC 2 Blks_Atop_2 SC 3 Blks_Atop_3 - --------- ------ ------ --------- ------ --------- ------ --------- - 1. 401,005 2 (h*, 0) 7985 (h*, 0) 1 (h*, 0) 0 - 2. 401,006 4 (h*, 0) 7984 (h*, 0) 0 (h*, 1) 7801 - 3. 401,007 2 (h*, 0) 7983 (h*, 5) 2027 (h*, 0) 0 - 4. 401,008 2 (h*, 0) 7982 (h*, 0) 2028 (h*, 1) 7800 - ... ... ) - 7999. 409,003 3 (h*, 0) 1 (h*, 0) 0 (h*, 0) 1 - 8000. 409,004 2 (h*, 0) 0 (h*, 1) 0 (h*, 0) 0 - - -When new sidechains (or "hashrate escrows") are soft-forked into existence, a new column is added to D3 (to contain any BMMing that might be done on it). - -For each sidechain we also track the field "Blocks Atop". This is the number of side:blocks that are "on top" of the specified side:block. These might be regarded as "side:confirmations" (pseudo-confirmations that are specific to each sidechain). - -D3 also contains a column (not shown) for each sidechain containing "prevSideBlockHash". This value is is either derived from prevBlockRef; or else it is given explicitly (in which case it is the converse: prevBlockRef is derived from prevSideBlockHash). - - -#### Coinbase Cache - -As mentioned above, M7s cause data to be added to D3. Recent D3 data is tracked by all mainchain nodes for a period of time. - -To efficiently keep track of the above data, without having to constantly load and process entire blocks from disk, we temporarily cache enough coinbases in the chain index to maintain D3. - - -### M8 -- Paying miners to include BMM data in their coinbase outputs - -This section introduces a new type of transaction, which allows sidechain block creators to request, and pay for, a critical hash to be included in a specific block by mainchain miners. See [the Blind Merged Mining spec](http://www.truthcoin.info/blog/blind-merged-mining/). This txn allows miners to "sell" the act of mining a sidechain block. By taking advantage of this option, miners earn tx fees for mining sidechains...truly "for free". They do not even need to run sidechain nodes, and the tx-fees they earn are in mainchain BTC. As a result, sidechains affect all miners equally and do not affect the mining ecosystem. - -M8 will ultimately come in two versions. The second version will be specialized for use in the Lightning Network and must use the full 32-byte prevBlockHash (ironically, this larger transaction is cheaper for the Bitcoin network to process, as it is completely off-chain). The first version of M8, in contrast, cannot be used inside the Lightning Network, but is slightly more space-efficient (using the 2 prevBlockRef bytes to maintain chain order). It is important to make both options available to the user, because some side:nodes may be unwilling or unable to open a payment channels with each main:miner. However, in the long run we expect the lightning version to be preferred. - -#### Setup - -We define **"Mary"** as a mainchain miner, and **"Simon"** as a sidechain node. - -The goal is to construct a payment from Simon to Mary, such that: - -1. If the critical data conditions are met, **Mary** can claim the outputs of the transaction with finality. -2. If the critical data conditions are not met, the outputs become immediately available again to **Simon**. - - -#### Goals (this is rather philosophical, and skippable) - -##### Immediate Expiration ("Fill-or-Kill") - -We would like to make special guarantees to the counterparties of this transaction. Specifically, instead of Simon making a "payment" to Mary, we prefer that Simon give Mary an "offer" (which she can either accept or decline). - -Crucially, we want Simon to safely make many offers to several different Mary's, in realtime (ie, quickly and off-chain). However, we ultimately want only one offer to be accepted, at most. In other words, we want Simon's offers to *immediately expire*. If only one offer can become a bona fide transaction, then Simon will feel comfortable making offers all day long. Because all of the Simons are making many offers, the Marys collectively gain access to a large set of offers to choose from. - -##### Forward Progress (The Need for a "Ratchet") - -The "ratchet" concept is an attempt to harmonize incentives among the main and side chain(s). -We need to ensure that a sidechain is making "forward progress", without tracking too much about the sidechain such that we burden Bitcoin (see [1] and [2]) all while still allowing the sidechain to reorganize [3]. - -* [1] https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-July/014789.html -* [2] http://www.drivechain.info/faq/index.html#what-is-the-difference-between-drivechain-and-extension-blocks -* [3] http://www.truthcoin.info/blog/blind-merged-mining/#handling-reorganizations - -The ratchet system must keep track of sidechain "mini-headers" (see Sidechain Critical Data ("Sidechain Mini-Header")) and count the "blocks atop" maturity of the related side:blocks. - -Simon's offer to Mary to include a critical hash in exchange for payment must be *atomic*. The "ratchet" concept helps to construct a very tight connection between two things: - -1. The sidechain-block-generator "Simon" paying himself the side:block's side:tx-fees (which he receives in 100 sidechain blocks (blocks atop) hence). -2. "Simon" making a mainchain main:btc payment to a mainchain miner "Mary". - -Either both of the two should succeed, or else both should jointly fail. - -However, absent our intervention, there are cases in which [2, the payment to Mary] succeeds but [1, side:tx-fees] fails. One such case is when a side:block contains unusually high side:tx-fees. Here, there will be many requests to include a critical hash in exchange for payment submitted to Mary, but only one can be included in each main:block per sidechain. Without an incentive to make "forward progress", Mary is likely to include one of the highest paying requests in the next main:block (and the main:block after that, and so on). Mary will "blindly" include high-paying requests for *older* blocks, unless something prevents her from doing so. - -To address these potential issues, we utilize the concept of "Blocks_Atop" (the "side:confirmations") that we mentioned earlier. As previously mentioned, Mary will not be able to spend Simon's M8 payment until satisfying the critical data requirements as well as the blocks atop (side:confirmations) requirement. - - -#### M8 -- The two forms of M8 transactions - -As previously mentioned, M8 can take two forms. The first does not require the Lightning Network, but it does have new requirements for Immediate Expiration (see above). The second inherits Immediate Expiration from the Lightning Network itself, but requires extra preparation and a different/larger message. - -Both forms require that certain Critical Data will be committed to within the coinbase of the block that the transaction is included in. For the non Lightning version, we have created a new extended serialization transaction type (very similar to how segwit handles witness data (the witness stack)). - -##### M8_V1 - No Lightning Network - -M8_V1 does not require the Lightning network but does have new requirements for validation. - -A M8_V1 TxOut is expected to contain: - - 1-byte - OP_RETURN (0x6a) - 1-byte - Push the following 36 bytes (0x24) - 4-byte - Message header (0xD1617368) - 32-bytes - h* side:block hash - 5~7-bytes - BMM request identifying bytes (0x00bf00) + prevBlockRef & ChainIndex (sidechain mini-header) - - -In the first version of M8, we need to introduce the concept of Immediate Expiration (see above). In other words, we need a way for Simon to construct many payments to multiple Marys, such that only one of these is ever included; and only then if Simon's txn is expected to coincide with the finding of Simon's side:block. - -We do this by imposing validity-rules on the txn itself: - -1. The txn's content, when examined, must match part of the main:block's content. Specifically, the (ChainIndex, h\*) pair of the txn, must match one of the (ChainIndex, h\*) pairs in the M7 of this main:block. -2. Only one payment per sidechain per main:block is valid. In other words, if 400 people all try to bm-mine the sidechain with ChainIndex==4, then not only is it the case that only one side_4:block can be found, but it is also the case that only the corresponding M8 txn can be included (out of all of the 400 M8s which are for ChainIndex==4). -3. Simon's txns must only be valid for the current block; afterward, they immediately expire. This is because Simon's intended prevBlockRef & side:block contents will most likely change from one main:block to the next. - -To impose new requirements on the transaction level (not the block level nor the TxOutput level), we borrow the "flag" trick from SegWit style transactions. If the flag is present, the transaction is examined for extra data, and if this data does not pass certain requirements, the transaction is invalid. With SegWit, this extra data is the signatures, and the extra requirements are the signatures' locations and validity. In the BMM-transactions, the extra data is the (ChainIndex, h\*) pair, which must meet the first two requirements (above) as well as the main:blocknumber, which must meet the third requirement (above). - -To impose new requirements at the transaction level, we borrow the dummy vin & "flag" trick from SegWit style transactions. If the flag is set to 2 (0010), the transaction contains Critical Data and requires that our new validation rules be met in order for the txn to be valid in a block. Unless all of the requirements for sidechain critical data transactions are met by the block it is included in, the transaction is invalid. With SegWit, this extra data is the segwit signature stack, and the extra requirements are the signatures' locations and validity. In the sidechain BMM critical data transactions, the extra data is the (ChainIndex, h\*) pair, which must meet the first two requirements (above) as well as the main:blocknumber, which must meet the third requirement (above). Note The main:blocknumber does not take up any additional space compared to a normal txn, as we reuse the locktime field for our purposes. - - - - -![extra-data-image](/bip-blind-merged-mining/witness-vs-critical.png?raw=true) - -This txn structure conserves main:blockspace, because it is the easiest way to refer to a previous sidechain block in 4 bytes, (prevBlockRef + FoK_nLockTime). Instead, we would need to use at least 32 bytes (prevSideBlockHash). - -These types of transactions have slightly different mempool behavior, and should probably be kept in a second mempool. These txns are received, checked immediately, and if valid they are evaluated for inclusion in a block. If they are not able to be included in the specific requested block (if the block height requested has been surpassed by the chain tip), they are discarded. In fact, after any main:block is found, everything in this "second mempool" can be discarded as new payments will be created immediately for the next block height. (This includes cases where the blockchain reorganizes.) There is no re-evaluation of the txns in this mempool ever -- they are evaluated once and then either included or discarded. To be clear, when the transaction is received we are able to evaluate its validity, and do not need to rescan these transactions again. - -Interestingly, these payments (M8) will *always* be directed to miners from non-miners. Therefore, non-mining nodes do not need to keep them in any mempool at all. Non-miner nodes can just wait for a block to be found, and check the txn then. These transactions more resemble a stock market's pit trades (in contrast, regular Bitcoin txns remind me more of paper checks). - -##### M8_V2 With Lightning - -M8_V2 requires having a LN-channel open with a miner. This may not always be practical (or even possible), especially today. - -A M8_V1 TxOut is expected to contain: - - 1-byte - OP_RETURN (0x6a) - 1-byte - Push the following 68 bytes (0x44) - 4-byte - Message header (0xD0520C6E) - 32-bytes - h* side:block hash - 32-bytes - prevSideBlockHash - 5~7-bytes - BMM request identifying bytes (0x00bf00) + prevBlockRef & ChainIndex (sidechain mini-header) - - -Notice that, in M8_V1, Simon could reuse the same h\* all he wanted, because only one M8_V1 could be included per main:block per sidechain. However, on the LN no such rule can be enforced, as the goal is to push everything off-chain and include *zero* M8s. So, we will never know what the M8s were or how many had an effect on anything. - -Therefore, Simon will need to ensure that he **gives each Mary a different h\***. Simon can easily do this, as he controls the side:block's contents and can simply increment a nonce -- this changes the side:block, and changes its hash (ie, changes h\*). - -With a unique h\* per Mary, and at most 1 h\* making it into a block (per sidechain), we can guarantee that only one of the M8_V2's critical data can be committed to in a single main:block. By giving each miner (who Simon has a payment channel open with) a different h*, Simon can figure out which miner followed through with the commit, and know that only one such commit went through. Furthermore, if this Simon's requested critical data is not found in a block, none of the M8_V2 payments will be spendable by the Mary(s), because none of the h\* in question have ever made it into D3 (which is always on-chain) and no blocks atop will be accumulated. - -That's probably confusing, so here is an example, in which: Simon starts with 13 BTC, Mary starts with 40 BTC, the side:block's tx-fees currently total 7.1 BTC, and Simon is keeping 0.1 BTC for himself and paying 7 BTC to Mary. - -We start with (I): - - Simon 13 in, Mary 40 in ; 53 in total - Simon's version [signed by Mary] - 13 ; to Simon if TimeLock=over; OR to Mary if SimonSig - 40 ; to Mary - Mary's version [signed by Simon] - 40 ; to me if TimeLock=over; OR to Simon if MarySig - 13 ; to Simon - - -And both parties move, from there to "M8_V2" (II): - - Simon 13 in, Mary 40 in ; 53 in total - Simon's version [signed by Mary] - 6 ; to Simon if TimeLock=over; OR to Mary if SimonSig - 40 ; to Mary - 7 ; to Mary if critical data requirements met; OR to Simon if LongTimeLock=over - Mary's version [signed by Simon] - 40 ; to Mary if TimeLock=over; OR to Simon if MarySig - 6 ; to Simon - 7 ; to Mary if critical data requirements met; OR to Simon if LongTimeLock=over - -From here, if the h\* side:block in question is BMMed, they can proceed to (III): - - Simon 13 in, Mary 40 in ; 53 in total - Simon's version [signed by Mary] - 6 ; to Simon if TimeLock=over; OR to Mary if SimonSig - 47 ; to Mary - Mary's version [signed by Simon] - 47 ; to me if TimeLock=over; OR to Simon if MarySig - 6 ; to Simon - -Although, if Simon proceeds immediately, he removes the protection of the 'ratchet'. Ie, Simon removes Mary's incentive to care about blocks being built on this side:block. If Simon's side:block is orphaned, he loses his 7 BTC. Simon can either play it safe, and wait the full 100 side:blocks before moving on (ie, moving on to the third LN txn, above); or else Simon can take the risk if he feels comfortable with it. - -If the h\* side:block is not found, then (II) and (III) are basically equivalent to each other. Simon and Mary could jointly reconstruct (I) and go back there, or they could proceed to a new version of II (with a different h\*, trying again with new side:block in the next main:block). - - - - -Deployment -=========== - -This BIP will be deployed by "version bits" BIP9 with the name "blindmm" and using bit 4. - -``` -// Deployment of Drivechains (BIPX, BIPY) -consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].bit = 4; -consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].nStartTime = 1515974401; // January 15th, 2018. -consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].nTimeout = 1547510401; // January 15th, 2019. -``` - -Reference Implementation -========================== - -See: https://github.com/drivechain-project/bitcoin/tree/mainchainBMM - -Also, for interest, see an example sidechain here: https://github.com/drivechain-project/bitcoin/tree/sidechainBMM - - -References -============ - -* http://www.drivechain.info/literature/index.html -* http://www.truthcoin.info/blog/blind-merged-mining/ -* https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-July/014789.html -* http://www.truthcoin.info/images/bmm-outline.txt - - -Thanks -========= - -Thanks to everyone who contributed to the discussion, especially: ZmnSCPxj, Adam Back, Peter Todd, Dan Anderson, Sergio Demian Lerner, Matt Corallo, Sjors Provoost, Tier Nolan, Erik Aronesty, Jason Dreyzehner, Joe Miyamoto, Ben Goldhaber. - - -Copyright -========== - -This BIP is licensed under the BSD 2-clause license. diff --git a/bip-blind-merged-mining/bmm-dots-examples.png b/bip-blind-merged-mining/bmm-dots-examples.png deleted file mode 100644 index 70f11f6..0000000 Binary files a/bip-blind-merged-mining/bmm-dots-examples.png and /dev/null differ diff --git a/bip-blind-merged-mining/images.txt b/bip-blind-merged-mining/images.txt deleted file mode 100644 index 00b3932..0000000 --- a/bip-blind-merged-mining/images.txt +++ /dev/null @@ -1 +0,0 @@ -Images used as reference in the documentation. diff --git a/bip-blind-merged-mining/witness-vs-critical.png b/bip-blind-merged-mining/witness-vs-critical.png deleted file mode 100644 index 1a2458d..0000000 Binary files a/bip-blind-merged-mining/witness-vs-critical.png and /dev/null differ diff --git a/bip-hashrate-escrows/images.txt b/bip-hashrate-escrows/images.txt new file mode 100644 index 0000000..2fbbf63 --- /dev/null +++ b/bip-hashrate-escrows/images.txt @@ -0,0 +1 @@ +Images used as reference in the documentation. diff --git a/bip-hashrate-escrows/two-groups.png b/bip-hashrate-escrows/two-groups.png new file mode 100644 index 0000000..c8a3ffa Binary files /dev/null and b/bip-hashrate-escrows/two-groups.png differ diff --git a/hashrate-escrows.md b/hashrate-escrows.md new file mode 100644 index 0000000..a594b26 --- /dev/null +++ b/hashrate-escrows.md @@ -0,0 +1,408 @@ + +Header +======= + + BIP: ???? + Layer: Consensus (soft fork) + Title: Hashrate Escrows (Consensus layer) + Author: Paul Sztorc + CryptAxe + Comments-Summary: No comments yet. + Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-??????? + Status: Draft + Type: Standards Track + Created: 2017-08-14 + License: BSD-2-Clause + Post-History: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-May/014364.html + + +Abstract +========== + +A "Hashrate Escrow" is a clearer term for the concept of "locked to an SPV Proof", which is itself a restatement of the phrase "within a sidechain" as described in [a famous Oct 2014 paper](https://blockstream.com/sidechains.pdf) written partially by some Blockstream co-founders. + +A Hashrate Escrow resembles a 2-of-3 multisig escrow, where the 3rd party (who will arbitrate any disputes) is a decentralized group of people: the dynamic-membership set of Bitcoin Miners. However, the 3rd party does not sign escrow-withdrawal transactions with a private key. Instead, these are "signed" by directing hashpower over them for a period of time. + +This project has [a website](http://www.drivechain.info/) which includes [a FAQ](http://www.drivechain.info/faq/index.html). + + +Motivation +============ + +In practice these escrows are likely to be "asymmetric sidechains" of Bitcoin (such as [Rootstock](http://www.rsk.co/)) or "virtual chains" within Bitcoin (such as [proposed by Blockstack](https://github.com/blockstack/virtualchain) in mid-2016). + +Sidechains have many potential benefits, including: + +1. Protect Bitcoin from competition from altcoins and spinoffs. Safely allow competing implementations (of *sidechains*). +2. Protect Bitcoin from hard fork campaigns. (Such campaigns represent an existential threat to Bitcoin, as well as an avenue for developer corruption.) +3. Help with review, by making it much easier for reviewers to ignore bad ideas. +4. Provide an avenue for good-but-confusing ideas to prove their value safely. + + + +Specification +============== + + +#### Components + +Hashrate Escrows are built of two types of component: [1] new databases, and [2] new message-interpretations. + +##### 1. New Databases + +* D1. "Escrow_DB" -- a database of "accounts" and their attributes. +* D2. "Withdrawal_DB" -- a database of pending withdrawals from these accounts, and their statuses. + +Please note that these structures (D1 and D2) will not literally exist anywhere in the blockchain. Instead they are constructed from messages...these messages, in contrast, *will* exist in the blockchain (with the exception of M4). + +##### 2. New Messages + +* M1. "Propose New Escrow" +* M2. "ACK Escrow Proposal" +* M3. "Propose Withdrawal" +* M4. (implied) "ACK Withdrawal" +* M5. "Execute Deposit" -- a transfer of BTC from-main-to-side +* M6. "Execute Withdrawal" -- a transfer of BTC from-side-to-main + + +#### On the Resource Requirements of New Databases + +The "new" databases are simply reinterpretations of data that are already contained elsewhere in the blockchain. Specifically, M1 M2 and M3 are all located in the block's coinbase txn, and M5 and M6 might be found in any regular txn. M4 is a special case and does not actually need to be included anywhere, so it is not. If you like, you can imagine that the M4s reside in an optional extension block. + +In other words, we just rearrange what is already there. Because of this, even though "new databases" are created and stored in memory, the existing bandwidth and storage limits are respected (although, see "M4" below). + + + + +### Adding Sidechains and Tracking Them (D1, M1, M2) + +#### D1 -- "Escrow_DB" + +The table below enumerates the new database fields, their size in bytes, and their purpose. In general, an escrow designer (for example, a sidechain-designer), is free to choose any value for these. + +Note: Fields 6 through 9 have been intentionally removed. Previously, this section allowed miners to set and commit to voting/waiting periods. However, I have since standardized the periods: withdrawals expire after 6 months (26298 blocks), and they succeed if they ever achieve an ACK score of 13140 or higher. I have removed the waiting period, because anyone who adopts a policy of ignoring all withdrawals with fewer than 400 ACKs will automatically gain all of the benefits of the waiting period. The justification for this change is that it strongly implies that an attack on any one sidechain is an attack on all of them (in a sense, this change makes the "victimhood" of each sidechain "fungible"). + +Field No. | Label | Bytes | Type | Description / Purpose +----------|-------|------|------|-------- +1 | Escrow Number | 1 | uInt | A number assigned to the entire escrow. Used to make it easy to refer to each escrow. +2 | Active\* | 2 | uInt | This counter starts at zero when a new entry is added, and increments as a new entry is supported by "M2". The entry either reaches the max value of 4032 (at which point the counter freezes), or else the entry is removed. This enforces the requirement "a soft fork is required to add a new escrow". +3 | Escrow Name/Description | 120 | string | A human-readable name and description of the sidechain. More than enough space to hold a 32 byte hash. Helps prevent destructive interference among sidechains (see below). +4 | Critical Private Key | 32 | hex | This is the Double-SHA256 of the binary release of the reference software. It ties the sidechain to a specific software release, and doubles as a way of signing withdrawals (see below). +5 | Critical Address\* | 32 | string | This is derived by taking f(x) = ImportPrivateKey(HexToBase58Check(x)) of field #3. It is intentionally in the human-readable format {{although it could be 25 bytes of binary}}. +10 | "CTIP" -- Part 1 "TxID"\* | 32 | hex | The CTIP, or "Critical (TxID, Index) Pair" is a variable for keeping track of where the escrow's money is (ie, which member of the UTXO set). +11 | "CTIP" -- Part 2 "Index"\* | 4 | hex | Of the CTIP, this is second element of the pair: the Index. See #10 above. + + +\* Denotes a "convenience field", the entry for this field is derived from other fields, or from the blockchain-state itself. The escrow-creator does not need to provide these values in M1 (or anywhere). + +Escrow_DB requires 223 bytes [1+120+32+32+2+32+4] for each escrow in the main blockchain. Of these, 70 bytes [2+32+32+4] are merely for convenience. Therefore, a sidechain is defined (see "M1") by 153 bytes of information. + +#### Notes on D1 + +1. D1 will always exist. +2. D1 will always have a unique sort (first by "Escrow Number" (field #1), and second by "Active" (field #2)). There is only ever one (escrow number, Active) pair. +3. D1 is updated according to M1 and M2 (below). +4. If a new entry is added to D1 with an "Escrow Number" that is already in use, then this entry will either eventually be removed (because it was not supported with an M2), or it will eventually overwrite the old entry (if it *was* supported via M2). + + +#### Notes on D1 + +##### Obligations Placed on Miners + +Miners have always upgraded their software according to criteria that are known only to them (in other words, "whenever they want"). + +However, this soft fork imposes two new criteria upon them. First: miners should only upgrade their software, if any modification to the portfolio of sidechains [that are added/removed in the upgrade] can be expected to increase miner wealth. Trivially, this implies that miners should make sure that the upgrade doesn't overwrite (and destroy) an existing sidechain that they like! But, more seriously, it implies that miners should take an interest in what the sidechain is doing to the mainchain and other sidechains (see below). + +##### Destructive Sidechain Interference + +People frequently emphasize that miners should have "as little control" as possible. It is a very safe claim to make, and a very easy sentence to write. Much harder is to determine exactly what this minimum value is, and how to achieve it. Harder still is to untie the knot of who is actually controlling what, in a decentralized, interacting system. + +Certainly, miners can not have "zero control" -- for that is the same as to just remove them from the system altogether. Some rules are enforced "on miners by nodes" (such as the infamous blocksize limit); other rules are enforced by nodes but are narrowly-controlled by miners (such as the proof-of-work itself, or the block's timestamp). Thirdly, some rules are enforced by both against each other (such as the rule against including invalid txns or double-spent txns), for mutual benefit. + +Some pause should be given, after one considers that the sidechain design goal is literally a piece of software that can do *anything*. Anything includes a great many things, many of which I demonstrate to be undesirable. Bitcoin itself does not allow "anything" -- it allows any person to transact, but, in contrast, it does not permit any person to double-spend. This is because "allowing anyone to do anything" is not viable in a world that contains undesirable interactions (what a libertarian might call "aggression") -- in the case of money, these are theft and counterfeiting. + +I have produced a comprehensive quantity of written material [1], presentations [2], etc [3] on exactly what the level of miner-control should be, and why. Specifically, I claim that **miners should be aware of the purpose of the sidechain, and they should reject sidechains which have an unclear purpose or which have a purpose that will lead to decrease in miner-wealth** (where wealth measured explicitly as: the estimated present value of the purchasing power of the blockchain's coinbase txns). I claim that this criterion is necessary because, just Original Bitcoin filters unwanted interactions among different BTC txns, so too much "Sidechain Bitcoin" filter out unwanted interactions among sidechain. + +* [1] http://www.truthcoin.info/blog/wise-contracts/ +* [2] https://www.youtube.com/watch?v=xGu0o8HH10U&index=1&list=PLw8-6ARlyVciMH79ZyLOpImsMug3LgNc4 +* [3] http://www.drivechain.info/literature/index.html + +Call it a "sidechain non-aggression principle", if you want. + +To the best of my knowledge, everyone who *has* reviewed this information as found the arguments to be acceptable. It has, also, changed a few minds (from "unacceptable" to "acceptable"). + + +##### ISSUE: "Signing" BTC Txns + +Currently, we use a process which may be suboptimal. It is that we *literally sign* a txn with a globally and publicly known private key. But this is for convenience purposes -- the signature that is produced is not doing anything, and is therefore wasteful. Instead we may use OP_TRUE, but this might interfere with how we detect the sidechain's balance. I'm not sure what the best way is. Someone needs to investigate how to do this -- removing OP_CheckSig, etc. This is a TODO for sure, and an opportunity for someone to help. + + + +(The following messages were modeled on SegWit -- https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki#commitment-structure ) + + + +#### M1 -- "Propose New Sidechain" + + 1-byte - OP_RETURN (0x6a) + 1-byte - Push the following 157 bytes (0x9d) + 4-byte - Commitment header (0x53707243) + 153-byte - the critical bytes mentioned above (fields #1, #3, and #4, to populate a new D1 entry) + + +#### New Block Validation Rules + +1. If the network detects a properly-formatted M1, it must add an entry to D1, into the very next block, with the following initial values: +* Field #5 will be calculated as per [version 1 Bitcoin addresses](https://en.bitcoin.it/wiki/Technical_background_of_version_1_Bitcoin_addresses), but with a prefix of "4" instead of "1". +* Field #9 will be derived from #7 and #8 using math. +* The initial values of Fields #10, #11, and #12 are set to zero. +2. Only one M1 (of any kind) can be added into a block at a time. + +#### Notes on M1 + +The escrow will "reuse" the same address over and over. But notice that there is no privacy loss due to address reuse, because the address in question does not "belong" to any specific individual. Furthermore, the transactions in question could already be linked by just watching the Hashrate Escrow for deposits ...this, in fact, is exactly what a sidechain must do in order to make main-to-side transfers. + + +#### M2 -- "ACK Sidechain Proposal" + + 1-byte - OP_RETURN (0x6a) + 1-byte - Push the following 29 bytes (0x1D) + 4-byte - Commitment header (0x53616343) + 25-byte - Commitment hash: RIPMD-160 of Sha256 of a given M1 (above) + +#### New Block Validation Rules + +1. An entry will be removed from D1, unless that block contains an M2 supporting it (this will increment field #2), or the entry has accumulated 4032 M2s. Therefore, each entry needs to have 4032 *consecutive* M2s supporting it, else it will be removed from D1 and must start over. This requires all miners to agree to "soft fork" the sidechain into existence. This makes it easier to curate the list of active sidechains, and prevents destructive interference (see above) among contradictory sidechains. +2. Once the entry has 4032 M2s, then a miner must make a 0 btc payment to the Critical Address, thus activating the chain. This payment becomes the first "critical (TxID, index) pair" (or "CTIP"). +3. It is possible to "overwrite" a D1 slot -- this destroys the sidechain that is there, and replaces it with a new one. This is defined as requiring 6 months (26298 blocks) of M2s, instead of 4 weeks (4032). This possibility does not change the security assumptions, because we already assume that users have access to "magically true" bandwidth at a rate of 1 bit per ACK-cycle. (This is why the voting cycles should be large, 1 month Waiting, 2+ months Voting). + + + +### Adding Withdrawal-Attempts and Tracking Their Success (D2, M3, M4) + +#### D2 -- "Withdrawal_DB" + +The table below enumerates the database fields, their size (in bytes), type and purpose. + + +Field No. | Label | Bytes | Type | Description / Purpose +----------|-------|------|------|----------------------- +1 | Escrow Number | 1 | uInt | Links the withdrawal-request to a specific escrow. +2 | WT^ | 32 | hex | This is a "blinded transaction id" (ie, the double-Sha256 of a txn that has had two fields zeroed out, see M6) of a withdrawal-attempt. +3 | ACKs\* | 2 | uInt | The current total number of "votes", this starts at 0 and remains there throughout the waiting period. +4 | Age\* | 3 | uInt | Total duration of time, in blocks, that this WT^ has been inside of D2. +5 | Waiting Period\* | 2 | uInt | Total duration of time, in blocks, that this entry must sit idle, before it can begin to accumulate ACKs/NACKs. Pulled from D1's field #6. +6 | Max Age\* | 3 | uInt | Determined by summing (D1's field #6) and (D1's field #7). +7 | Threshold\* | 2 | uInt | Total ACKs needed, this is pulled from D1's field #9. +8 | Approved\* | 1 | boolean | True while ACKs > Threshold, False otherwise. + +\* Denotes a "convenience field" (see above). + +Withdrawal_DB requires 46 bytes [1+32+2+3+2+3+2+1] per entry. Of these, 13 bytes ([2+3+2+3+2+1], all fields except #1 and #2) can be generated locally, leaving 33 critical bytes [1+32]. + +#### New Block Validation Rules for D2 + +1. In each block, a hash commitment to D2 must always exist (even if D2 is blank). +2. D2 must always be sorted first by field #1 (Escrow Number) and second by field #4 (Age). This imposes a unique sort. +3. From one block to the next, every entry's "Age" field must increase by exactly 1. +4. From one block to the next, entries are only removed from D2 (in the very next block) if: +* * "Age" = "MaxAge". +* * If the block contains a txn who's blinded txID matches WT^. {{ This might be unnecessary, and a lot of work. }} +5. In addition, there are special rules for the allowed values in the "ACKs" field (field #3). See M4 below. + +#### M3 -- "Propose Withdrawal" + + 1-byte - OP_RETURN (0x6a) + 1-byte - Push the following 37 bytes (0x25) + 4-byte - Commitment header (0xD45AA943) + 33-byte - the critical bytes mentioned above (fields #1 and #2, to populate a new D2 entry) + + +#### New Block Validation Rules for M3 + +1. If the network detects a properly-formatted M3, it must add an entry to D2 in the very next block. The starting values of fields #3 and #4 are zero, and #5 is pulled over by extracting the relevant value from D1. +2. Each block can only contain one M3 per sidechain. + + +#### M4 -- "ACK Withdrawal" + +#### Very Little Info, Probably Calculable in Advance + +M4 is exceptional (in comparison to the other M's) in a few ways. First, its content is not stored anywhere, only the *hash* of its *effect* is stored (in a leaf of a merkle tree who's root is inserted into a mainchain coinbase). M4 alters the contents of D2 -- the *contents* of D2 are consensus critical, but M4 (the process by which nodes reach a new valid D2) can be anything. + +In fact, M4 can also be *nothing*. In other words, it may be optional. This is precisely because, from one block to the next, we have constrained D2 such that it is only allowed to change in a few ways. Therefore, the exhaustive set of "candidate D2s" can be precomputed by full nodes in advance. + +#### A Recent Change: Two Withdrawals at Once + +The following sections assume a maximum of one sucessful withdrawal per sidechain at a time. In other words, as WT^s are proposed, only one can make progress toward the finish line. As a result, a given side-to-main transfer will always take between 3 and 6 months. If there were more simulataneous withdrawals, the worst-case transfer duration would improve. + +![dots-image](/images/two-groups.png?raw=true) + +The worst-case withdrawal time obeys f(n)=3+(3/n) months, where n is the number of simultaneous withdrawals. + +N=2 is the most desirable choice for several reasons. First, it delievers the greatest marginal benefit (of 1.5 months). Later choices only deliver 0.5 and 0.25 marginal months. + +Second, n=2 can be implemented in a clever way: by allowing a withdrawal to freely advance, if and only if has an ACK-score of 6575 or greater, and if it also has the largest ACK score. In other words, the withdrawal that is furthest along can advance (or retreat) for free, if it has already made it at least halfway to the finish line. With this change, our new M4, is either an "abstain" for the sidechain (in which case nothing happens to any ACK scores), or else it will be in one of two cases: old_M4 + "the largest advances", or new_M4 + "the largest retreats". As a result the number of M4 possibilities (of which the next section is concerned) only increases by a factor of two (instead of exponentially). + +It is possible to troll this rule, by getting two (or even three) withdrawals to have 6575+ ACK scores, and then getting them to *tie* for first place. So, if there are any ties, the ability to "bonus move" is disabled until all ties are broken. + +#### How Hard is it to Guess M4? + +If there are n Escrows and m Withdrawals-per-escrow1, then there are (m+2)^n total candidates for the next D2. This is because, [per block per escrow], one of three things can happen: (1) one of the m withdrawal-candidates can be "ACK"ed (or "upvoted" or "promoted"), which automatically downvotes the others; or (2) all withdrawal-candidates can be downvoted, or finally (3) the miners can abstain from voting on the escrow's withdrawals altogether, leaving the tallies the same. + +First, for nodes which validate all sidechains (assuming these escrows are sidechains), this simplifies to 2^n -- these nodes only have to choose between the single honest choice (on one hand) or an abstention (on the other). Second, even for nodes that don't validate any sidechains, the number of candidates might be reduced from m^n to 3^n, by making a simplifying assumption: whichever withdrawal was most recently added/upvoted, is likely to be the one which is upvoted next. + +Of course, that is still O(k^n) for n sidechains, which isn't great2. If the "D2 update" cannot be guessed, it must be transmitted in some way. + +#### Giving Up and Getting M4 the Old Fashioned Way + +Two examples for transmitting it are below: + +"Short Form" (Assumes there are no more than 254 active withdrawal-attempts per account) + + 4-byte - Message identifier (0x????????) + 1-byte - Version of this message + N-byte - N is the total number of active accounts ("sidechains"), each byte specifies the position of the single WT that was "upvoted". A value of 0 indicates "downvote everything", a value of 255 indicates abstention. + +"Long Form" (Makes no assumptions about anything) + + 4-byte - Message identifier (0x????????) + 1-byte - Version of this message + 1-byte - Length (in bytes) of this message; total number of withdrawal attempts; y = ceiling( sum_i(m_i +2)/8 ). Nodes should already know what length to expect, because they know the sequence of M3s and therefore the vector of WT^s. + Y-byte - stream of bits (not bytes), with a 1 indicating the position of the chosen action [downvote all, abstain, upvote1, upvote2, ...] + + +If the message is very very large, then nodes may not want to broadcast it. This opens up an "exhaustion attack"2, in which many miners create bad WT^s, vote on these randomly, and then refuse to broadcast their votes. Fortunately, even for a worst-case scenario of 200 sidechains and 1,000 withdrawal-attempts per sidechain, honest nodes can communicate a long form M4 with each other by using just 25,056 bytes per block [4+1+1+(200\*(1000+1+1)/8)]. + +Today's pre-drivechain miners can already carry out a similar attack, by creating and including txns and then not broadcasting that part of the block to anyone. This is often characterized as a ["block publication incentive"](https://petertodd.org/2016/block-publication-incentives-for-miners), because in that case the prospect of exhaustively computing all possible transactions (to uncover the missing ones) is completely out of the question. + +However, message M4 is different from a withheld-txn, because M4 operates outside of the block's mandated information-processing limits (ie, outside the infamous 1 MB nonwitness blocksize limit). So we should examine the conditions under which M4 grows and shrinks, to ensure that we are not smuggling in a tremendous burden on full nodes. + +Under adversarial conditions, to lengthen a long-form M4 by one bit per block, for C blocks, the attacker must pay 312 bits (39 bytes) one time (to embed a new M3 message). The value C is the length of the sidechain's voting period, which varies but which I expect to be approximately 8,064 (and which could theoretically be as high as 65,536). Thus the attacker can burden nodes disproportionately, if (s)he wishes. + +Fortunately, the attack in question has no motivation (as far as I can tell). If the miner's goal is to trick rivals into mining on top of invalid blocks, he can already do this much more effectively with the unpublished-txn method (above). If instead he is just trying to harass nodes, then nodes may freely "downgrade" to earlier versions of the protocol, and simply ignore all drivechain-related messages. It seems that the attack could best be used in order to: make a large D2, make D2 confusing, sneak in votes for evil WT^ lurking in D2. Thus, the attack disables the transparency of the drivechain system, to some extent. The cost of the attack is forgone transaction fees, due to block space wasted on useless M3s. + +In practice, n is already capped, and miners may impose [on each other] a "soft cap" on m for their mutual protection. Thus, n and m might never get above 10 and 30, respectfully. In this case, the [Short Form, this time] M4 can never require more than 15 bytes per block, no matter what the attacker tries. + +In practice, m should always be 1 or 2, else something fishy is going on; and m can only inch up by 1 unit per block. So the system as a whole is still quite transparent, in that users are warned appropriately and well in advance. Attackers must invest upfront and they face an uphill climb, in order to eventually make things more expensive for a few others; defenders can wait-and-see if the attack looks like it will ever amount to anything before lifting a finger. + + +##### New Block Validation Rules (for D2 and, by implication, M4) + +From one block to the next, D2 can only be edited in a few strict ways: + +* Entries can only be added/removed from D2 if they meet the criteria above (in M3, and implicitly M1 and M2). +* The ACK-counter of any individual entry can only change by (-1,0,+1) relative to its previous entry. +* Within a sidechain group, upvoting one withdrawal (ACK=ACK+1) requires you to downvote all other withdrawals in that group. However, the minimum ACK value is zero (and, therefore, downvotes cannot reduce it below zero). + +##### Footnotes for M4 + +1 This represents the worst-case scenario is one where all the Withdrawals are spread evenly over each Sidechain. Under normal operations, there is no reason to expect the all sidechains will have the same number of withdrawals at any given time. In fact, under normal operations, the very *concept* of counting the withdrawals-per-sidechain should be a purposeless one, because there should only be *one* withdrawal at a time. Nonetheless we consider the worst case scenario here. + +2 Guessing becomes more computationally intensive in a highly adversarial situation where the "limited range" is intentionally expanded. In such a scenario, [a] there are many sidechains, and [b] miners voluntarily sacrifice their scarce block-space by creating a high number of (mutually-exclusive, and hence ultimately invalid) withdrawal attempts and putting these into coinbase transactions; and then agree to all [c] vote on these randomly (guaranteeing that all withdrawals fail, including any true withdrawals) and [d] successfully withhold their random voting strategies from nodes (even including spy-miner-nodes). Under this bizarre scenario, nodes may require computing resources which increase near-exponentially with the number of withdrawals, and it may take a long time for an ignorant node to exhaustively work out the underlying state of Withdrawal_DB. In this case, nodes may decide to temporarily stop validating such transactions (as if they had not yet upgraded to support this soft fork). + + + +### Depositing and Withdrawing (M5, M6) + + +Both M5 and M6 are regular Bitcoin txns. They are identified by meeting an important criteria: they select a one of the Critical TxID-index Pairs (a "CTIP") as one of their inputs. Deposits ("M5") are distinguished from withdrawals ("M6") by simply checking to see if money is "going in", or "out". In other words, we compare the BTC value of the original CTIP to that of new CTIP. If original <= new it is a deposit, if original > new then it is a withdrawal. + +The code that identifies sidechain withdrawal / deposit txns (by calculating how much value is being put into or taken out of a sidechain) can be seen here: https://github.com/drivechain-project/bitcoin/blob/mainchainBMM/src/validation.cpp#L351-L386 + +Such txns are forced (by consensus) to obey two additional criteria: + +1. They must contain an output paying "to" the Critical Address [probably in TxOut0]. +2. They must be accompanied by an update to this sidechain's Critical TxID-index Pair (CTIP). The new CTIP must be "this" txn itself. + +These criteria are enforced here https://github.com/drivechain-project/bitcoin/blob/mainchainBMM/src/validation.cpp#L440-L473 by checking that a deposit is paying back to the sidechain more than it is taking out, and completely rejecting any withdrawal from the mempool. And here https://github.com/drivechain-project/bitcoin/blob/mainchainBMM/src/validation.cpp#L1747-L1757 we allow for a withdrawal only once it has attained sufficient work score (ACKs). + +The purpose of this is to have all of the escrow's money (ie all of the sidechain's money) in one TxID, so that depositors immediately undo any UTXO bloat they may cause. This simplifies the withdrawal process, as there is no need to worry about cleaning up "dust deposits" (...and such cleaning can often result in headaches, for example where a withdrawal-txn is larger than 1MB in size, or else may only withdraw an arbitrarily limited amount of BTC). Notice that, unless we assume that an account will last forever, all utxos which are deposited must eventually be withdrawn by someone. Therefore, the relevant design criterion is not "efficiency" (total network cost) but rather "who should pay" (allocation of costs). + +#### M5. "Make a Deposit" -- a transfer of BTC from-main-to-side + +As far as mainchain consensus is concerned, there are no additional requirements. + +However, in practice there *are* additional mainchain requirements...specified by the escrow account, (ie specified by the "sidechain" or "virtual chain"). These requirements are not part of mainchain consensus and are allowed to be anything. In other words, the sidechain is free to invent any way to credit depositor's money -- M5 is fully customizable. + +One method, is for mainchain depositors to append a zero-value OP Return to a Deposit txn, so that the sidechain knows how to credit funds. Mainchain users must upgrade their wallet software, of course, (on an individual basis) in order to become aware of and take advantage of new deposit-methods. + +##### Inconvenient Race Condition + +The requirement that each hashrate escrow be linked to a single TxID does create an interesting inconvenience for depositors. If a user is slow to sign a txn after constructing it (perhaps because the user employs an air-gapped computer, etc), then the signed txn may no longer be valid. This is because the input it selects, may no longer be the Critical TxID (as "the" Critical TxID changes with each deposit). **Only one user can deposit at a time** (although many can deposit per block). As a result, the transaction must fail, and the user would need to be prompted to remake and resign the txn. If this is problem is too frustrating, users can always make main-to-side transfers using atomic cross chain swaps (or, the LN, if they already have a channel open on both chains). + +Fortunately, it is already a part of mainchain consensus that no two txns can spend the same TxID. The only new issue here is the confusion it might create for the user (hence the need for error messages and alternative deposit-methods). + + +#### M6. "Execute Withdrawal" -- a transfer of BTC from-side-to-main + +We come, finally, to the critical matter: where users can take their money *out* of the escrow account, and return it to the "regular" UTXO set. As previously mentioned, this txn is one which (a) spends from a CTIP and (b) reduces the quantity of BTC in an account's CTIP. Most of the work has already been done by D1, M3, M4, and D2. Furthermore, existing Bitcoin tx-rules prevent the sidechain from ever withdrawing more money than has been placed into it. + +From there, we merely introduce two final concepts: + +1. In each block, an entry in D2 is considered an "approved candidate" if the "ACKs" value is above 13140. +2. A "blinded TxID" is way of hashing the txn, in which we first overwrite some parts of the txn with zeros. Specifically, the first 36 bytes of "TxIn0" (the first input, including TxOutHash and TxOutIndex), as well as the first 8 bytes of "TxOut0" (the first output). + +Blinding is necessary because we allow each sidechain only one UTXO at a time. + +of our restriction of the account to a single UTXO-member. Because of this, during the ACKing process the withdrawal-txn (which is currently being ACKed) may change in two ways: the CTIP (which changes with each deposit), and the total quantity of BTC stored in the account (which arbitrarily increases with each new deposit). In other words, a withdrawal-attempt is created via M3, but this takes place many blocks before the withdrawal is actually included via M6. During this time, a single new deposit to the account would change its CTIP and its value. So, what do we ACK? Well, we ACK a "blinded" version of the withdrawal. This blinded version is stable because the dynamic parts are always overwritten with zeros. + +While we ACK a blinded WT^, what is actually included in the blockchain ("M6") is an unblinded WT^. Since each blinded WT^ could correspond to many different unblinded WT^s, we need to impose further restrictions on those unblinded WT^s that are finally included. First, we will force the final unblinded WT^ to spend the entire sidechain balance (by forcing sum(input_values) to equal sum(output_values)). To avoid withdrawing the entire sidechain balance with every withdrawal, we will, secondly, force the unblinded WT^ to create a new output which is itself a deposit to the sidechain it withdrew from (which nodes can check using D1's CTIP field). Unfortunately, these requirements eliminate the possibility of including a transaction fee, as traditionally calculated. So, finally, to compensate for *that*, txn fees are encoded explicitly as a withdrawal to OP_TRUE (which the main:block's miner can immediately claim). + +With all of this in place, the only requirements for inclusion in a block are these: + +1. "Be ACKed" -- The "blinded TxID" of this txn must be member of the "approved candidate" set in the D2 of this block. +2. "Return Change to Account" -- TxOut0 must pay to the "critical account" (see D1) that corresponds to the CTIP that was selected as a TxIn. +3. "Return *all* Change to Account" -- Sum of inputs must equal the sum of outputs. No traditional tx fee is possible. + +Finally, don't forget that M6 inherits the requirement (common to both M5 and M6) that the CTIP be selected as an input, and that the CTIP then be updated. In this case, we know that the critical index will be zero, so the new CTIP will be ("this TxID" (NOT blinded), 0). The TxID is NOT blinded because blinding is only for accumulating ACKs. + +As a result of these requirements, every single withdrawal-attempt will fail, unless an entry has been added to D2 and "ACKed" a sufficient number of times. + + + +Backward compatibility +======================== + +As a soft fork, older software will continue to operate without modification. Non-upgraded nodes will see a number of phenomena that they don't understand -- coinbase txns with non-txn data, value accumulating in anyone-can-spend UTXOs for months at a time, and then random amounts leaving the UTXO in single, infrequent bursts. However, this phenomena doesn't affect them or the validity of the money that they receive. + +( As a nice bonus, note that the sidechains themselves inherit a resistance to hard forks. The only way to guarantee that the WT^s reported by different clients will continue to match identically, is to upgrade sidechains via soft forks of themselves. ) + + +Deployment +=========== + +This BIP will be deployed by "version bits" BIP9 with the name "hrescrow" and using bit 4. + +``` +// Deployment of Drivechains (BIPX, BIPY) +consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].bit = 4; +consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].nStartTime = 1515974401; // January 15th, 2018. +consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].nTimeout = 1547510401; // January 15th, 2019. +``` + +Reference Implementation +========================== + +See: https://github.com/drivechain-project/bitcoin/tree/mainchainBMM + +Also, for interest, see an example sidechain here: https://github.com/drivechain-project/bitcoin/tree/sidechainBMM + + +References +============ + +See http://www.drivechain.info/literature/index.html + + +Credits +========= + +Thanks to everyone who contributed to the discussion, especially: ZmnSCPxj, Adam Back, Peter Todd, Dan Anderson, Sergio Demian Lerner, Chris Stewart, Matt Corallo, Sjors Provoost, Tier Nolan, Erik Aronesty, Jason Dreyzehner, Joe Miyamoto, Ben Goldhaber. + + + +Copyright +========== + +This BIP is licensed under the BSD 2-clause license. -- cgit v1.2.3 From 99f553025fdf3bb3fd3f5ef9bd88846bfad1a8fa Mon Sep 17 00:00:00 2001 From: Paul Sztorc Date: Sat, 10 Feb 2018 16:30:07 -0500 Subject: mediawiki format --- bip-hashrate-escrows.mediawiki | 400 ++++++++++++++++++++++++++++++++++++++++ hashrate-escrows.md | 408 ----------------------------------------- 2 files changed, 400 insertions(+), 408 deletions(-) create mode 100644 bip-hashrate-escrows.mediawiki delete mode 100644 hashrate-escrows.md diff --git a/bip-hashrate-escrows.mediawiki b/bip-hashrate-escrows.mediawiki new file mode 100644 index 0000000..0f2210c --- /dev/null +++ b/bip-hashrate-escrows.mediawiki @@ -0,0 +1,400 @@ + + +
+    BIP: ????
+    Layer: Consensus (soft fork)
+    Title: Hashrate Escrows (Consensus layer)
+    Author: Paul Sztorc 
+            CryptAxe 
+    Comments-Summary: No comments yet.
+    Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-???????
+    Status: Draft
+    Type: Standards Track
+    Created: 2017-08-14
+    License: BSD-2-Clause
+    Post-History: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-May/014364.html
+
+ +==Abstract== + +A "Hashrate Escrow" is a clearer term for the concept of "locked to an SPV Proof", which is itself a restatement of the phrase "within a sidechain" as described in [a famous Oct 2014 paper](https://blockstream.com/sidechains.pdf) written partially by some Blockstream co-founders. + +A Hashrate Escrow resembles a 2-of-3 multisig escrow, where the 3rd party (who will arbitrate any disputes) is a decentralized group of people: the dynamic-membership set of Bitcoin Miners. However, the 3rd party does not sign escrow-withdrawal transactions with a private key. Instead, these are "signed" by directing hashpower over them for a period of time. + +This project has [a website](http://www.drivechain.info/) which includes [a FAQ](http://www.drivechain.info/faq/index.html). + + +==Motivation== + +In practice these escrows are likely to be "asymmetric sidechains" of Bitcoin (such as [Rootstock](http://www.rsk.co/)) or "virtual chains" within Bitcoin (such as [proposed by Blockstack](https://github.com/blockstack/virtualchain) in mid-2016). + +Sidechains have many potential benefits, including: + +1. Protect Bitcoin from competition from altcoins and spinoffs. Safely allow competing implementations (of *sidechains*). +2. Protect Bitcoin from hard fork campaigns. (Such campaigns represent an existential threat to Bitcoin, as well as an avenue for developer corruption.) +3. Help with review, by making it much easier for reviewers to ignore bad ideas. +4. Provide an avenue for good-but-confusing ideas to prove their value safely. + + + +==Specification== + +==== Components ==== + +Hashrate Escrows are built of two types of component: [1] new databases, and [2] new message-interpretations. + +===== 1. New Databases ===== + +* D1. "Escrow_DB" -- a database of "accounts" and their attributes. +* D2. "Withdrawal_DB" -- a database of pending withdrawals from these accounts, and their statuses. + +Please note that these structures (D1 and D2) will not literally exist anywhere in the blockchain. Instead they are constructed from messages...these messages, in contrast, *will* exist in the blockchain (with the exception of M4). + +===== 2. New Messages ===== + +* M1. "Propose New Escrow" +* M2. "ACK Escrow Proposal" +* M3. "Propose Withdrawal" +* M4. (implied) "ACK Withdrawal" +* M5. "Execute Deposit" -- a transfer of BTC from-main-to-side +* M6. "Execute Withdrawal" -- a transfer of BTC from-side-to-main + + +==== On the Resource Requirements of New Databases ==== + +The "new" databases are simply reinterpretations of data that are already contained elsewhere in the blockchain. Specifically, M1 M2 and M3 are all located in the block's coinbase txn, and M5 and M6 might be found in any regular txn. M4 is a special case and does not actually need to be included anywhere, so it is not. If you like, you can imagine that the M4s reside in an optional extension block. + +In other words, we just rearrange what is already there. Because of this, even though "new databases" are created and stored in memory, the existing bandwidth and storage limits are respected (although, see "M4" below). + + + + +=== Adding Sidechains and Tracking Them (D1, M1, M2) === + +==== D1 -- "Escrow_DB" ==== + +The table below enumerates the new database fields, their size in bytes, and their purpose. In general, an escrow designer (for example, a sidechain-designer), is free to choose any value for these. + +Note: Fields 6 through 9 have been intentionally removed. Previously, this section allowed miners to set and commit to voting/waiting periods. However, I have since standardized the periods: withdrawals expire after 6 months (26298 blocks), and they succeed if they ever achieve an ACK score of 13140 or higher. I have removed the waiting period, because anyone who adopts a policy of ignoring all withdrawals with fewer than 400 ACKs will automatically gain all of the benefits of the waiting period. The justification for this change is that it strongly implies that an attack on any one sidechain is an attack on all of them (in a sense, this change makes the "victimhood" of each sidechain "fungible"). + +Field No. | Label | Bytes | Type | Description / Purpose +----------|-------|------|------|-------- +1 | Escrow Number | 1 | uInt | A number assigned to the entire escrow. Used to make it easy to refer to each escrow. +2 | Active\* | 2 | uInt | This counter starts at zero when a new entry is added, and increments as a new entry is supported by "M2". The entry either reaches the max value of 4032 (at which point the counter freezes), or else the entry is removed. This enforces the requirement "a soft fork is required to add a new escrow". +3 | Escrow Name/Description | 120 | string | A human-readable name and description of the sidechain. More than enough space to hold a 32 byte hash. Helps prevent destructive interference among sidechains (see below). +4 | Critical Private Key | 32 | hex | This is the Double-SHA256 of the binary release of the reference software. It ties the sidechain to a specific software release, and doubles as a way of signing withdrawals (see below). +5 | Critical Address\* | 32 | string | This is derived by taking f(x) = ImportPrivateKey(HexToBase58Check(x)) of field #3. It is intentionally in the human-readable format {{although it could be 25 bytes of binary}}. +10 | "CTIP" -- Part 1 "TxID"\* | 32 | hex | The CTIP, or "Critical (TxID, Index) Pair" is a variable for keeping track of where the escrow's money is (ie, which member of the UTXO set). +11 | "CTIP" -- Part 2 "Index"\* | 4 | hex | Of the CTIP, this is second element of the pair: the Index. See #10 above. + + +\* Denotes a "convenience field", the entry for this field is derived from other fields, or from the blockchain-state itself. The escrow-creator does not need to provide these values in M1 (or anywhere). + +Escrow_DB requires 223 bytes [1+120+32+32+2+32+4] for each escrow in the main blockchain. Of these, 70 bytes [2+32+32+4] are merely for convenience. Therefore, a sidechain is defined (see "M1") by 153 bytes of information. + +====Notes on D1==== + +1. D1 will always exist. +2. D1 will always have a unique sort (first by "Escrow Number" (field #1), and second by "Active" (field #2)). There is only ever one (escrow number, Active) pair. +3. D1 is updated according to M1 and M2 (below). +4. If a new entry is added to D1 with an "Escrow Number" that is already in use, then this entry will either eventually be removed (because it was not supported with an M2), or it will eventually overwrite the old entry (if it *was* supported via M2). + + +====Notes on D1==== + +=====Obligations Placed on Miners===== + +Miners have always upgraded their software according to criteria that are known only to them (in other words, "whenever they want"). + +However, this soft fork imposes two new criteria upon them. First: miners should only upgrade their software, if any modification to the portfolio of sidechains [that are added/removed in the upgrade] can be expected to increase miner wealth. Trivially, this implies that miners should make sure that the upgrade doesn't overwrite (and destroy) an existing sidechain that they like! But, more seriously, it implies that miners should take an interest in what the sidechain is doing to the mainchain and other sidechains (see below). + +##### Destructive Sidechain Interference + +People frequently emphasize that miners should have "as little control" as possible. It is a very safe claim to make, and a very easy sentence to write. Much harder is to determine exactly what this minimum value is, and how to achieve it. Harder still is to untie the knot of who is actually controlling what, in a decentralized, interacting system. + +Certainly, miners can not have "zero control" -- for that is the same as to just remove them from the system altogether. Some rules are enforced "on miners by nodes" (such as the infamous blocksize limit); other rules are enforced by nodes but are narrowly-controlled by miners (such as the proof-of-work itself, or the block's timestamp). Thirdly, some rules are enforced by both against each other (such as the rule against including invalid txns or double-spent txns), for mutual benefit. + +Some pause should be given, after one considers that the sidechain design goal is literally a piece of software that can do *anything*. Anything includes a great many things, many of which I demonstrate to be undesirable. Bitcoin itself does not allow "anything" -- it allows any person to transact, but, in contrast, it does not permit any person to double-spend. This is because "allowing anyone to do anything" is not viable in a world that contains undesirable interactions (what a libertarian might call "aggression") -- in the case of money, these are theft and counterfeiting. + +I have produced a comprehensive quantity of written material [1], presentations [2], etc [3] on exactly what the level of miner-control should be, and why. Specifically, I claim that **miners should be aware of the purpose of the sidechain, and they should reject sidechains which have an unclear purpose or which have a purpose that will lead to decrease in miner-wealth** (where wealth measured explicitly as: the estimated present value of the purchasing power of the blockchain's coinbase txns). I claim that this criterion is necessary because, just Original Bitcoin filters unwanted interactions among different BTC txns, so too much "Sidechain Bitcoin" filter out unwanted interactions among sidechain. + +* [1] http://www.truthcoin.info/blog/wise-contracts/ +* [2] https://www.youtube.com/watch?v=xGu0o8HH10U&index=1&list=PLw8-6ARlyVciMH79ZyLOpImsMug3LgNc4 +* [3] http://www.drivechain.info/literature/index.html + +Call it a "sidechain non-aggression principle", if you want. + +To the best of my knowledge, everyone who *has* reviewed this information as found the arguments to be acceptable. It has, also, changed a few minds (from "unacceptable" to "acceptable"). + + +===== ISSUE: "Signing" BTC Txns ===== + +Currently, we use a process which may be suboptimal. It is that we *literally sign* a txn with a globally and publicly known private key. But this is for convenience purposes -- the signature that is produced is not doing anything, and is therefore wasteful. Instead we may use OP_TRUE, but this might interfere with how we detect the sidechain's balance. I'm not sure what the best way is. Someone needs to investigate how to do this -- removing OP_CheckSig, etc. This is a TODO for sure, and an opportunity for someone to help. + + + +(The following messages were modeled on SegWit -- https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki#commitment-structure ) + + + +==== M1 -- "Propose New Sidechain" ==== + + 1-byte - OP_RETURN (0x6a) + 1-byte - Push the following 157 bytes (0x9d) + 4-byte - Commitment header (0x53707243) + 153-byte - the critical bytes mentioned above (fields #1, #3, and #4, to populate a new D1 entry) + + +==== New Block Validation Rules ==== + +1. If the network detects a properly-formatted M1, it must add an entry to D1, into the very next block, with the following initial values: +* Field #5 will be calculated as per [version 1 Bitcoin addresses](https://en.bitcoin.it/wiki/Technical_background_of_version_1_Bitcoin_addresses), but with a prefix of "4" instead of "1". +* Field #9 will be derived from #7 and #8 using math. +* The initial values of Fields #10, #11, and #12 are set to zero. +2. Only one M1 (of any kind) can be added into a block at a time. + +==== Notes on M1 ==== + +The escrow will "reuse" the same address over and over. But notice that there is no privacy loss due to address reuse, because the address in question does not "belong" to any specific individual. Furthermore, the transactions in question could already be linked by just watching the Hashrate Escrow for deposits ...this, in fact, is exactly what a sidechain must do in order to make main-to-side transfers. + + +==== M2 -- "ACK Sidechain Proposal" ==== + + 1-byte - OP_RETURN (0x6a) + 1-byte - Push the following 29 bytes (0x1D) + 4-byte - Commitment header (0x53616343) + 25-byte - Commitment hash: RIPMD-160 of Sha256 of a given M1 (above) + +==== New Block Validation Rules ==== + +1. An entry will be removed from D1, unless that block contains an M2 supporting it (this will increment field #2), or the entry has accumulated 4032 M2s. Therefore, each entry needs to have 4032 *consecutive* M2s supporting it, else it will be removed from D1 and must start over. This requires all miners to agree to "soft fork" the sidechain into existence. This makes it easier to curate the list of active sidechains, and prevents destructive interference (see above) among contradictory sidechains. +2. Once the entry has 4032 M2s, then a miner must make a 0 btc payment to the Critical Address, thus activating the chain. This payment becomes the first "critical (TxID, index) pair" (or "CTIP"). +3. It is possible to "overwrite" a D1 slot -- this destroys the sidechain that is there, and replaces it with a new one. This is defined as requiring 6 months (26298 blocks) of M2s, instead of 4 weeks (4032). This possibility does not change the security assumptions, because we already assume that users have access to "magically true" bandwidth at a rate of 1 bit per ACK-cycle. (This is why the voting cycles should be large, 1 month Waiting, 2+ months Voting). + + + +=== Adding Withdrawal-Attempts and Tracking Their Success (D2, M3, M4) === + +==== D2 -- "Withdrawal_DB" ==== + +The table below enumerates the database fields, their size (in bytes), type and purpose. + + +Field No. | Label | Bytes | Type | Description / Purpose +----------|-------|------|------|----------------------- +1 | Escrow Number | 1 | uInt | Links the withdrawal-request to a specific escrow. +2 | WT^ | 32 | hex | This is a "blinded transaction id" (ie, the double-Sha256 of a txn that has had two fields zeroed out, see M6) of a withdrawal-attempt. +3 | ACKs\* | 2 | uInt | The current total number of "votes", this starts at 0 and remains there throughout the waiting period. +4 | Age\* | 3 | uInt | Total duration of time, in blocks, that this WT^ has been inside of D2. +5 | Waiting Period\* | 2 | uInt | Total duration of time, in blocks, that this entry must sit idle, before it can begin to accumulate ACKs/NACKs. Pulled from D1's field #6. +6 | Max Age\* | 3 | uInt | Determined by summing (D1's field #6) and (D1's field #7). +7 | Threshold\* | 2 | uInt | Total ACKs needed, this is pulled from D1's field #9. +8 | Approved\* | 1 | boolean | True while ACKs > Threshold, False otherwise. + +\* Denotes a "convenience field" (see above). + +Withdrawal_DB requires 46 bytes [1+32+2+3+2+3+2+1] per entry. Of these, 13 bytes ([2+3+2+3+2+1], all fields except #1 and #2) can be generated locally, leaving 33 critical bytes [1+32]. + +==== New Block Validation Rules for D2 ==== + +1. In each block, a hash commitment to D2 must always exist (even if D2 is blank). +2. D2 must always be sorted first by field #1 (Escrow Number) and second by field #4 (Age). This imposes a unique sort. +3. From one block to the next, every entry's "Age" field must increase by exactly 1. +4. From one block to the next, entries are only removed from D2 (in the very next block) if: +* * "Age" = "MaxAge". +* * If the block contains a txn who's blinded txID matches WT^. {{ This might be unnecessary, and a lot of work. }} +5. In addition, there are special rules for the allowed values in the "ACKs" field (field #3). See M4 below. + +==== M3 -- "Propose Withdrawal" ==== + + 1-byte - OP_RETURN (0x6a) + 1-byte - Push the following 37 bytes (0x25) + 4-byte - Commitment header (0xD45AA943) + 33-byte - the critical bytes mentioned above (fields #1 and #2, to populate a new D2 entry) + + +==== New Block Validation Rules for M3 ==== + +1. If the network detects a properly-formatted M3, it must add an entry to D2 in the very next block. The starting values of fields #3 and #4 are zero, and #5 is pulled over by extracting the relevant value from D1. +2. Each block can only contain one M3 per sidechain. + + +==== M4 -- "ACK Withdrawal" ==== + +==== Very Little Info, Probably Calculable in Advance ==== + +M4 is exceptional (in comparison to the other M's) in a few ways. First, its content is not stored anywhere, only the *hash* of its *effect* is stored (in a leaf of a merkle tree who's root is inserted into a mainchain coinbase). M4 alters the contents of D2 -- the *contents* of D2 are consensus critical, but M4 (the process by which nodes reach a new valid D2) can be anything. + +In fact, M4 can also be *nothing*. In other words, it may be optional. This is precisely because, from one block to the next, we have constrained D2 such that it is only allowed to change in a few ways. Therefore, the exhaustive set of "candidate D2s" can be precomputed by full nodes in advance. + +==== A Recent Change: Two Withdrawals at Once ==== + +The following sections assume a maximum of one sucessful withdrawal per sidechain at a time. In other words, as WT^s are proposed, only one can make progress toward the finish line. As a result, a given side-to-main transfer will always take between 3 and 6 months. If there were more simulataneous withdrawals, the worst-case transfer duration would improve. + +![dots-image](/images/two-groups.png?raw=true) + +The worst-case withdrawal time obeys f(n)=3+(3/n) months, where n is the number of simultaneous withdrawals. + +N=2 is the most desirable choice for several reasons. First, it delievers the greatest marginal benefit (of 1.5 months). Later choices only deliver 0.5 and 0.25 marginal months. + +Second, n=2 can be implemented in a clever way: by allowing a withdrawal to freely advance, if and only if has an ACK-score of 6575 or greater, and if it also has the largest ACK score. In other words, the withdrawal that is furthest along can advance (or retreat) for free, if it has already made it at least halfway to the finish line. With this change, our new M4, is either an "abstain" for the sidechain (in which case nothing happens to any ACK scores), or else it will be in one of two cases: old_M4 + "the largest advances", or new_M4 + "the largest retreats". As a result the number of M4 possibilities (of which the next section is concerned) only increases by a factor of two (instead of exponentially). + +It is possible to troll this rule, by getting two (or even three) withdrawals to have 6575+ ACK scores, and then getting them to *tie* for first place. So, if there are any ties, the ability to "bonus move" is disabled until all ties are broken. + +==== How Hard is it to Guess M4? ==== + +If there are n Escrows and m Withdrawals-per-escrow1, then there are (m+2)^n total candidates for the next D2. This is because, [per block per escrow], one of three things can happen: (1) one of the m withdrawal-candidates can be "ACK"ed (or "upvoted" or "promoted"), which automatically downvotes the others; or (2) all withdrawal-candidates can be downvoted, or finally (3) the miners can abstain from voting on the escrow's withdrawals altogether, leaving the tallies the same. + +First, for nodes which validate all sidechains (assuming these escrows are sidechains), this simplifies to 2^n -- these nodes only have to choose between the single honest choice (on one hand) or an abstention (on the other). Second, even for nodes that don't validate any sidechains, the number of candidates might be reduced from m^n to 3^n, by making a simplifying assumption: whichever withdrawal was most recently added/upvoted, is likely to be the one which is upvoted next. + +Of course, that is still O(k^n) for n sidechains, which isn't great2. If the "D2 update" cannot be guessed, it must be transmitted in some way. + +==== Giving Up and Getting M4 the Old Fashioned Way ==== + +Two examples for transmitting it are below: + +"Short Form" (Assumes there are no more than 254 active withdrawal-attempts per account) + + 4-byte - Message identifier (0x????????) + 1-byte - Version of this message + N-byte - N is the total number of active accounts ("sidechains"), each byte specifies the position of the single WT that was "upvoted". A value of 0 indicates "downvote everything", a value of 255 indicates abstention. + +"Long Form" (Makes no assumptions about anything) + + 4-byte - Message identifier (0x????????) + 1-byte - Version of this message + 1-byte - Length (in bytes) of this message; total number of withdrawal attempts; y = ceiling( sum_i(m_i +2)/8 ). Nodes should already know what length to expect, because they know the sequence of M3s and therefore the vector of WT^s. + Y-byte - stream of bits (not bytes), with a 1 indicating the position of the chosen action [downvote all, abstain, upvote1, upvote2, ...] + + +If the message is very very large, then nodes may not want to broadcast it. This opens up an "exhaustion attack"2, in which many miners create bad WT^s, vote on these randomly, and then refuse to broadcast their votes. Fortunately, even for a worst-case scenario of 200 sidechains and 1,000 withdrawal-attempts per sidechain, honest nodes can communicate a long form M4 with each other by using just 25,056 bytes per block [4+1+1+(200\*(1000+1+1)/8)]. + +Today's pre-drivechain miners can already carry out a similar attack, by creating and including txns and then not broadcasting that part of the block to anyone. This is often characterized as a ["block publication incentive"](https://petertodd.org/2016/block-publication-incentives-for-miners), because in that case the prospect of exhaustively computing all possible transactions (to uncover the missing ones) is completely out of the question. + +However, message M4 is different from a withheld-txn, because M4 operates outside of the block's mandated information-processing limits (ie, outside the infamous 1 MB nonwitness blocksize limit). So we should examine the conditions under which M4 grows and shrinks, to ensure that we are not smuggling in a tremendous burden on full nodes. + +Under adversarial conditions, to lengthen a long-form M4 by one bit per block, for C blocks, the attacker must pay 312 bits (39 bytes) one time (to embed a new M3 message). The value C is the length of the sidechain's voting period, which varies but which I expect to be approximately 8,064 (and which could theoretically be as high as 65,536). Thus the attacker can burden nodes disproportionately, if (s)he wishes. + +Fortunately, the attack in question has no motivation (as far as I can tell). If the miner's goal is to trick rivals into mining on top of invalid blocks, he can already do this much more effectively with the unpublished-txn method (above). If instead he is just trying to harass nodes, then nodes may freely "downgrade" to earlier versions of the protocol, and simply ignore all drivechain-related messages. It seems that the attack could best be used in order to: make a large D2, make D2 confusing, sneak in votes for evil WT^ lurking in D2. Thus, the attack disables the transparency of the drivechain system, to some extent. The cost of the attack is forgone transaction fees, due to block space wasted on useless M3s. + +In practice, n is already capped, and miners may impose [on each other] a "soft cap" on m for their mutual protection. Thus, n and m might never get above 10 and 30, respectfully. In this case, the [Short Form, this time] M4 can never require more than 15 bytes per block, no matter what the attacker tries. + +In practice, m should always be 1 or 2, else something fishy is going on; and m can only inch up by 1 unit per block. So the system as a whole is still quite transparent, in that users are warned appropriately and well in advance. Attackers must invest upfront and they face an uphill climb, in order to eventually make things more expensive for a few others; defenders can wait-and-see if the attack looks like it will ever amount to anything before lifting a finger. + + +===== New Block Validation Rules (for D2 and, by implication, M4) ===== + +From one block to the next, D2 can only be edited in a few strict ways: + +* Entries can only be added/removed from D2 if they meet the criteria above (in M3, and implicitly M1 and M2). +* The ACK-counter of any individual entry can only change by (-1,0,+1) relative to its previous entry. +* Within a sidechain group, upvoting one withdrawal (ACK=ACK+1) requires you to downvote all other withdrawals in that group. However, the minimum ACK value is zero (and, therefore, downvotes cannot reduce it below zero). + +===== Footnotes for M4 ===== + +1 This represents the worst-case scenario is one where all the Withdrawals are spread evenly over each Sidechain. Under normal operations, there is no reason to expect the all sidechains will have the same number of withdrawals at any given time. In fact, under normal operations, the very *concept* of counting the withdrawals-per-sidechain should be a purposeless one, because there should only be *one* withdrawal at a time. Nonetheless we consider the worst case scenario here. + +2 Guessing becomes more computationally intensive in a highly adversarial situation where the "limited range" is intentionally expanded. In such a scenario, [a] there are many sidechains, and [b] miners voluntarily sacrifice their scarce block-space by creating a high number of (mutually-exclusive, and hence ultimately invalid) withdrawal attempts and putting these into coinbase transactions; and then agree to all [c] vote on these randomly (guaranteeing that all withdrawals fail, including any true withdrawals) and [d] successfully withhold their random voting strategies from nodes (even including spy-miner-nodes). Under this bizarre scenario, nodes may require computing resources which increase near-exponentially with the number of withdrawals, and it may take a long time for an ignorant node to exhaustively work out the underlying state of Withdrawal_DB. In this case, nodes may decide to temporarily stop validating such transactions (as if they had not yet upgraded to support this soft fork). + + + +=== Depositing and Withdrawing (M5, M6) === + + +Both M5 and M6 are regular Bitcoin txns. They are identified by meeting an important criteria: they select a one of the Critical TxID-index Pairs (a "CTIP") as one of their inputs. Deposits ("M5") are distinguished from withdrawals ("M6") by simply checking to see if money is "going in", or "out". In other words, we compare the BTC value of the original CTIP to that of new CTIP. If original <= new it is a deposit, if original > new then it is a withdrawal. + +The code that identifies sidechain withdrawal / deposit txns (by calculating how much value is being put into or taken out of a sidechain) can be seen here: https://github.com/drivechain-project/bitcoin/blob/mainchainBMM/src/validation.cpp#L351-L386 + +Such txns are forced (by consensus) to obey two additional criteria: + +1. They must contain an output paying "to" the Critical Address [probably in TxOut0]. +2. They must be accompanied by an update to this sidechain's Critical TxID-index Pair (CTIP). The new CTIP must be "this" txn itself. + +These criteria are enforced here https://github.com/drivechain-project/bitcoin/blob/mainchainBMM/src/validation.cpp#L440-L473 by checking that a deposit is paying back to the sidechain more than it is taking out, and completely rejecting any withdrawal from the mempool. And here https://github.com/drivechain-project/bitcoin/blob/mainchainBMM/src/validation.cpp#L1747-L1757 we allow for a withdrawal only once it has attained sufficient work score (ACKs). + +The purpose of this is to have all of the escrow's money (ie all of the sidechain's money) in one TxID, so that depositors immediately undo any UTXO bloat they may cause. This simplifies the withdrawal process, as there is no need to worry about cleaning up "dust deposits" (...and such cleaning can often result in headaches, for example where a withdrawal-txn is larger than 1MB in size, or else may only withdraw an arbitrarily limited amount of BTC). Notice that, unless we assume that an account will last forever, all utxos which are deposited must eventually be withdrawn by someone. Therefore, the relevant design criterion is not "efficiency" (total network cost) but rather "who should pay" (allocation of costs). + +==== M5. "Make a Deposit" -- a transfer of BTC from-main-to-side ==== + +As far as mainchain consensus is concerned, there are no additional requirements. + +However, in practice there *are* additional mainchain requirements...specified by the escrow account, (ie specified by the "sidechain" or "virtual chain"). These requirements are not part of mainchain consensus and are allowed to be anything. In other words, the sidechain is free to invent any way to credit depositor's money -- M5 is fully customizable. + +One method, is for mainchain depositors to append a zero-value OP Return to a Deposit txn, so that the sidechain knows how to credit funds. Mainchain users must upgrade their wallet software, of course, (on an individual basis) in order to become aware of and take advantage of new deposit-methods. + +===== Inconvenient Race Condition ===== + +The requirement that each hashrate escrow be linked to a single TxID does create an interesting inconvenience for depositors. If a user is slow to sign a txn after constructing it (perhaps because the user employs an air-gapped computer, etc), then the signed txn may no longer be valid. This is because the input it selects, may no longer be the Critical TxID (as "the" Critical TxID changes with each deposit). **Only one user can deposit at a time** (although many can deposit per block). As a result, the transaction must fail, and the user would need to be prompted to remake and resign the txn. If this is problem is too frustrating, users can always make main-to-side transfers using atomic cross chain swaps (or, the LN, if they already have a channel open on both chains). + +Fortunately, it is already a part of mainchain consensus that no two txns can spend the same TxID. The only new issue here is the confusion it might create for the user (hence the need for error messages and alternative deposit-methods). + + +==== M6. "Execute Withdrawal" -- a transfer of BTC from-side-to-main ==== + +We come, finally, to the critical matter: where users can take their money *out* of the escrow account, and return it to the "regular" UTXO set. As previously mentioned, this txn is one which (a) spends from a CTIP and (b) reduces the quantity of BTC in an account's CTIP. Most of the work has already been done by D1, M3, M4, and D2. Furthermore, existing Bitcoin tx-rules prevent the sidechain from ever withdrawing more money than has been placed into it. + +From there, we merely introduce two final concepts: + +1. In each block, an entry in D2 is considered an "approved candidate" if the "ACKs" value is above 13140. +2. A "blinded TxID" is way of hashing the txn, in which we first overwrite some parts of the txn with zeros. Specifically, the first 36 bytes of "TxIn0" (the first input, including TxOutHash and TxOutIndex), as well as the first 8 bytes of "TxOut0" (the first output). + +Blinding is necessary because we allow each sidechain only one UTXO at a time. + +of our restriction of the account to a single UTXO-member. Because of this, during the ACKing process the withdrawal-txn (which is currently being ACKed) may change in two ways: the CTIP (which changes with each deposit), and the total quantity of BTC stored in the account (which arbitrarily increases with each new deposit). In other words, a withdrawal-attempt is created via M3, but this takes place many blocks before the withdrawal is actually included via M6. During this time, a single new deposit to the account would change its CTIP and its value. So, what do we ACK? Well, we ACK a "blinded" version of the withdrawal. This blinded version is stable because the dynamic parts are always overwritten with zeros. + +While we ACK a blinded WT^, what is actually included in the blockchain ("M6") is an unblinded WT^. Since each blinded WT^ could correspond to many different unblinded WT^s, we need to impose further restrictions on those unblinded WT^s that are finally included. First, we will force the final unblinded WT^ to spend the entire sidechain balance (by forcing sum(input_values) to equal sum(output_values)). To avoid withdrawing the entire sidechain balance with every withdrawal, we will, secondly, force the unblinded WT^ to create a new output which is itself a deposit to the sidechain it withdrew from (which nodes can check using D1's CTIP field). Unfortunately, these requirements eliminate the possibility of including a transaction fee, as traditionally calculated. So, finally, to compensate for *that*, txn fees are encoded explicitly as a withdrawal to OP_TRUE (which the main:block's miner can immediately claim). + +With all of this in place, the only requirements for inclusion in a block are these: + +1. "Be ACKed" -- The "blinded TxID" of this txn must be member of the "approved candidate" set in the D2 of this block. +2. "Return Change to Account" -- TxOut0 must pay to the "critical account" (see D1) that corresponds to the CTIP that was selected as a TxIn. +3. "Return *all* Change to Account" -- Sum of inputs must equal the sum of outputs. No traditional tx fee is possible. + +Finally, don't forget that M6 inherits the requirement (common to both M5 and M6) that the CTIP be selected as an input, and that the CTIP then be updated. In this case, we know that the critical index will be zero, so the new CTIP will be ("this TxID" (NOT blinded), 0). The TxID is NOT blinded because blinding is only for accumulating ACKs. + +As a result of these requirements, every single withdrawal-attempt will fail, unless an entry has been added to D2 and "ACKed" a sufficient number of times. + + + +==Backward compatibility== + + +As a soft fork, older software will continue to operate without modification. Non-upgraded nodes will see a number of phenomena that they don't understand -- coinbase txns with non-txn data, value accumulating in anyone-can-spend UTXOs for months at a time, and then random amounts leaving the UTXO in single, infrequent bursts. However, this phenomena doesn't affect them or the validity of the money that they receive. + +( As a nice bonus, note that the sidechains themselves inherit a resistance to hard forks. The only way to guarantee that the WT^s reported by different clients will continue to match identically, is to upgrade sidechains via soft forks of themselves. ) + + +==Deployment== + + +This BIP will be deployed by "version bits" BIP9 with the name "hrescrow" and using bit 4. + +``` +// Deployment of Drivechains (BIPX, BIPY) +consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].bit = 4; +consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].nStartTime = 1515974401; // January 15th, 2018. +consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].nTimeout = 1547510401; // January 15th, 2019. +``` + +==Reference Implementation== + + +See: https://github.com/drivechain-project/bitcoin/tree/mainchainBMM + +Also, for interest, see an example sidechain here: https://github.com/drivechain-project/bitcoin/tree/sidechainBMM + + +==References== + +See http://www.drivechain.info/literature/index.html + + +==Credits== + +Thanks to everyone who contributed to the discussion, especially: ZmnSCPxj, Adam Back, Peter Todd, Dan Anderson, Sergio Demian Lerner, Chris Stewart, Matt Corallo, Sjors Provoost, Tier Nolan, Erik Aronesty, Jason Dreyzehner, Joe Miyamoto, Ben Goldhaber. + + + +==Copyright== + +This BIP is licensed under the BSD 2-clause license. diff --git a/hashrate-escrows.md b/hashrate-escrows.md deleted file mode 100644 index a594b26..0000000 --- a/hashrate-escrows.md +++ /dev/null @@ -1,408 +0,0 @@ - -Header -======= - - BIP: ???? - Layer: Consensus (soft fork) - Title: Hashrate Escrows (Consensus layer) - Author: Paul Sztorc - CryptAxe - Comments-Summary: No comments yet. - Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-??????? - Status: Draft - Type: Standards Track - Created: 2017-08-14 - License: BSD-2-Clause - Post-History: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-May/014364.html - - -Abstract -========== - -A "Hashrate Escrow" is a clearer term for the concept of "locked to an SPV Proof", which is itself a restatement of the phrase "within a sidechain" as described in [a famous Oct 2014 paper](https://blockstream.com/sidechains.pdf) written partially by some Blockstream co-founders. - -A Hashrate Escrow resembles a 2-of-3 multisig escrow, where the 3rd party (who will arbitrate any disputes) is a decentralized group of people: the dynamic-membership set of Bitcoin Miners. However, the 3rd party does not sign escrow-withdrawal transactions with a private key. Instead, these are "signed" by directing hashpower over them for a period of time. - -This project has [a website](http://www.drivechain.info/) which includes [a FAQ](http://www.drivechain.info/faq/index.html). - - -Motivation -============ - -In practice these escrows are likely to be "asymmetric sidechains" of Bitcoin (such as [Rootstock](http://www.rsk.co/)) or "virtual chains" within Bitcoin (such as [proposed by Blockstack](https://github.com/blockstack/virtualchain) in mid-2016). - -Sidechains have many potential benefits, including: - -1. Protect Bitcoin from competition from altcoins and spinoffs. Safely allow competing implementations (of *sidechains*). -2. Protect Bitcoin from hard fork campaigns. (Such campaigns represent an existential threat to Bitcoin, as well as an avenue for developer corruption.) -3. Help with review, by making it much easier for reviewers to ignore bad ideas. -4. Provide an avenue for good-but-confusing ideas to prove their value safely. - - - -Specification -============== - - -#### Components - -Hashrate Escrows are built of two types of component: [1] new databases, and [2] new message-interpretations. - -##### 1. New Databases - -* D1. "Escrow_DB" -- a database of "accounts" and their attributes. -* D2. "Withdrawal_DB" -- a database of pending withdrawals from these accounts, and their statuses. - -Please note that these structures (D1 and D2) will not literally exist anywhere in the blockchain. Instead they are constructed from messages...these messages, in contrast, *will* exist in the blockchain (with the exception of M4). - -##### 2. New Messages - -* M1. "Propose New Escrow" -* M2. "ACK Escrow Proposal" -* M3. "Propose Withdrawal" -* M4. (implied) "ACK Withdrawal" -* M5. "Execute Deposit" -- a transfer of BTC from-main-to-side -* M6. "Execute Withdrawal" -- a transfer of BTC from-side-to-main - - -#### On the Resource Requirements of New Databases - -The "new" databases are simply reinterpretations of data that are already contained elsewhere in the blockchain. Specifically, M1 M2 and M3 are all located in the block's coinbase txn, and M5 and M6 might be found in any regular txn. M4 is a special case and does not actually need to be included anywhere, so it is not. If you like, you can imagine that the M4s reside in an optional extension block. - -In other words, we just rearrange what is already there. Because of this, even though "new databases" are created and stored in memory, the existing bandwidth and storage limits are respected (although, see "M4" below). - - - - -### Adding Sidechains and Tracking Them (D1, M1, M2) - -#### D1 -- "Escrow_DB" - -The table below enumerates the new database fields, their size in bytes, and their purpose. In general, an escrow designer (for example, a sidechain-designer), is free to choose any value for these. - -Note: Fields 6 through 9 have been intentionally removed. Previously, this section allowed miners to set and commit to voting/waiting periods. However, I have since standardized the periods: withdrawals expire after 6 months (26298 blocks), and they succeed if they ever achieve an ACK score of 13140 or higher. I have removed the waiting period, because anyone who adopts a policy of ignoring all withdrawals with fewer than 400 ACKs will automatically gain all of the benefits of the waiting period. The justification for this change is that it strongly implies that an attack on any one sidechain is an attack on all of them (in a sense, this change makes the "victimhood" of each sidechain "fungible"). - -Field No. | Label | Bytes | Type | Description / Purpose -----------|-------|------|------|-------- -1 | Escrow Number | 1 | uInt | A number assigned to the entire escrow. Used to make it easy to refer to each escrow. -2 | Active\* | 2 | uInt | This counter starts at zero when a new entry is added, and increments as a new entry is supported by "M2". The entry either reaches the max value of 4032 (at which point the counter freezes), or else the entry is removed. This enforces the requirement "a soft fork is required to add a new escrow". -3 | Escrow Name/Description | 120 | string | A human-readable name and description of the sidechain. More than enough space to hold a 32 byte hash. Helps prevent destructive interference among sidechains (see below). -4 | Critical Private Key | 32 | hex | This is the Double-SHA256 of the binary release of the reference software. It ties the sidechain to a specific software release, and doubles as a way of signing withdrawals (see below). -5 | Critical Address\* | 32 | string | This is derived by taking f(x) = ImportPrivateKey(HexToBase58Check(x)) of field #3. It is intentionally in the human-readable format {{although it could be 25 bytes of binary}}. -10 | "CTIP" -- Part 1 "TxID"\* | 32 | hex | The CTIP, or "Critical (TxID, Index) Pair" is a variable for keeping track of where the escrow's money is (ie, which member of the UTXO set). -11 | "CTIP" -- Part 2 "Index"\* | 4 | hex | Of the CTIP, this is second element of the pair: the Index. See #10 above. - - -\* Denotes a "convenience field", the entry for this field is derived from other fields, or from the blockchain-state itself. The escrow-creator does not need to provide these values in M1 (or anywhere). - -Escrow_DB requires 223 bytes [1+120+32+32+2+32+4] for each escrow in the main blockchain. Of these, 70 bytes [2+32+32+4] are merely for convenience. Therefore, a sidechain is defined (see "M1") by 153 bytes of information. - -#### Notes on D1 - -1. D1 will always exist. -2. D1 will always have a unique sort (first by "Escrow Number" (field #1), and second by "Active" (field #2)). There is only ever one (escrow number, Active) pair. -3. D1 is updated according to M1 and M2 (below). -4. If a new entry is added to D1 with an "Escrow Number" that is already in use, then this entry will either eventually be removed (because it was not supported with an M2), or it will eventually overwrite the old entry (if it *was* supported via M2). - - -#### Notes on D1 - -##### Obligations Placed on Miners - -Miners have always upgraded their software according to criteria that are known only to them (in other words, "whenever they want"). - -However, this soft fork imposes two new criteria upon them. First: miners should only upgrade their software, if any modification to the portfolio of sidechains [that are added/removed in the upgrade] can be expected to increase miner wealth. Trivially, this implies that miners should make sure that the upgrade doesn't overwrite (and destroy) an existing sidechain that they like! But, more seriously, it implies that miners should take an interest in what the sidechain is doing to the mainchain and other sidechains (see below). - -##### Destructive Sidechain Interference - -People frequently emphasize that miners should have "as little control" as possible. It is a very safe claim to make, and a very easy sentence to write. Much harder is to determine exactly what this minimum value is, and how to achieve it. Harder still is to untie the knot of who is actually controlling what, in a decentralized, interacting system. - -Certainly, miners can not have "zero control" -- for that is the same as to just remove them from the system altogether. Some rules are enforced "on miners by nodes" (such as the infamous blocksize limit); other rules are enforced by nodes but are narrowly-controlled by miners (such as the proof-of-work itself, or the block's timestamp). Thirdly, some rules are enforced by both against each other (such as the rule against including invalid txns or double-spent txns), for mutual benefit. - -Some pause should be given, after one considers that the sidechain design goal is literally a piece of software that can do *anything*. Anything includes a great many things, many of which I demonstrate to be undesirable. Bitcoin itself does not allow "anything" -- it allows any person to transact, but, in contrast, it does not permit any person to double-spend. This is because "allowing anyone to do anything" is not viable in a world that contains undesirable interactions (what a libertarian might call "aggression") -- in the case of money, these are theft and counterfeiting. - -I have produced a comprehensive quantity of written material [1], presentations [2], etc [3] on exactly what the level of miner-control should be, and why. Specifically, I claim that **miners should be aware of the purpose of the sidechain, and they should reject sidechains which have an unclear purpose or which have a purpose that will lead to decrease in miner-wealth** (where wealth measured explicitly as: the estimated present value of the purchasing power of the blockchain's coinbase txns). I claim that this criterion is necessary because, just Original Bitcoin filters unwanted interactions among different BTC txns, so too much "Sidechain Bitcoin" filter out unwanted interactions among sidechain. - -* [1] http://www.truthcoin.info/blog/wise-contracts/ -* [2] https://www.youtube.com/watch?v=xGu0o8HH10U&index=1&list=PLw8-6ARlyVciMH79ZyLOpImsMug3LgNc4 -* [3] http://www.drivechain.info/literature/index.html - -Call it a "sidechain non-aggression principle", if you want. - -To the best of my knowledge, everyone who *has* reviewed this information as found the arguments to be acceptable. It has, also, changed a few minds (from "unacceptable" to "acceptable"). - - -##### ISSUE: "Signing" BTC Txns - -Currently, we use a process which may be suboptimal. It is that we *literally sign* a txn with a globally and publicly known private key. But this is for convenience purposes -- the signature that is produced is not doing anything, and is therefore wasteful. Instead we may use OP_TRUE, but this might interfere with how we detect the sidechain's balance. I'm not sure what the best way is. Someone needs to investigate how to do this -- removing OP_CheckSig, etc. This is a TODO for sure, and an opportunity for someone to help. - - - -(The following messages were modeled on SegWit -- https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki#commitment-structure ) - - - -#### M1 -- "Propose New Sidechain" - - 1-byte - OP_RETURN (0x6a) - 1-byte - Push the following 157 bytes (0x9d) - 4-byte - Commitment header (0x53707243) - 153-byte - the critical bytes mentioned above (fields #1, #3, and #4, to populate a new D1 entry) - - -#### New Block Validation Rules - -1. If the network detects a properly-formatted M1, it must add an entry to D1, into the very next block, with the following initial values: -* Field #5 will be calculated as per [version 1 Bitcoin addresses](https://en.bitcoin.it/wiki/Technical_background_of_version_1_Bitcoin_addresses), but with a prefix of "4" instead of "1". -* Field #9 will be derived from #7 and #8 using math. -* The initial values of Fields #10, #11, and #12 are set to zero. -2. Only one M1 (of any kind) can be added into a block at a time. - -#### Notes on M1 - -The escrow will "reuse" the same address over and over. But notice that there is no privacy loss due to address reuse, because the address in question does not "belong" to any specific individual. Furthermore, the transactions in question could already be linked by just watching the Hashrate Escrow for deposits ...this, in fact, is exactly what a sidechain must do in order to make main-to-side transfers. - - -#### M2 -- "ACK Sidechain Proposal" - - 1-byte - OP_RETURN (0x6a) - 1-byte - Push the following 29 bytes (0x1D) - 4-byte - Commitment header (0x53616343) - 25-byte - Commitment hash: RIPMD-160 of Sha256 of a given M1 (above) - -#### New Block Validation Rules - -1. An entry will be removed from D1, unless that block contains an M2 supporting it (this will increment field #2), or the entry has accumulated 4032 M2s. Therefore, each entry needs to have 4032 *consecutive* M2s supporting it, else it will be removed from D1 and must start over. This requires all miners to agree to "soft fork" the sidechain into existence. This makes it easier to curate the list of active sidechains, and prevents destructive interference (see above) among contradictory sidechains. -2. Once the entry has 4032 M2s, then a miner must make a 0 btc payment to the Critical Address, thus activating the chain. This payment becomes the first "critical (TxID, index) pair" (or "CTIP"). -3. It is possible to "overwrite" a D1 slot -- this destroys the sidechain that is there, and replaces it with a new one. This is defined as requiring 6 months (26298 blocks) of M2s, instead of 4 weeks (4032). This possibility does not change the security assumptions, because we already assume that users have access to "magically true" bandwidth at a rate of 1 bit per ACK-cycle. (This is why the voting cycles should be large, 1 month Waiting, 2+ months Voting). - - - -### Adding Withdrawal-Attempts and Tracking Their Success (D2, M3, M4) - -#### D2 -- "Withdrawal_DB" - -The table below enumerates the database fields, their size (in bytes), type and purpose. - - -Field No. | Label | Bytes | Type | Description / Purpose -----------|-------|------|------|----------------------- -1 | Escrow Number | 1 | uInt | Links the withdrawal-request to a specific escrow. -2 | WT^ | 32 | hex | This is a "blinded transaction id" (ie, the double-Sha256 of a txn that has had two fields zeroed out, see M6) of a withdrawal-attempt. -3 | ACKs\* | 2 | uInt | The current total number of "votes", this starts at 0 and remains there throughout the waiting period. -4 | Age\* | 3 | uInt | Total duration of time, in blocks, that this WT^ has been inside of D2. -5 | Waiting Period\* | 2 | uInt | Total duration of time, in blocks, that this entry must sit idle, before it can begin to accumulate ACKs/NACKs. Pulled from D1's field #6. -6 | Max Age\* | 3 | uInt | Determined by summing (D1's field #6) and (D1's field #7). -7 | Threshold\* | 2 | uInt | Total ACKs needed, this is pulled from D1's field #9. -8 | Approved\* | 1 | boolean | True while ACKs > Threshold, False otherwise. - -\* Denotes a "convenience field" (see above). - -Withdrawal_DB requires 46 bytes [1+32+2+3+2+3+2+1] per entry. Of these, 13 bytes ([2+3+2+3+2+1], all fields except #1 and #2) can be generated locally, leaving 33 critical bytes [1+32]. - -#### New Block Validation Rules for D2 - -1. In each block, a hash commitment to D2 must always exist (even if D2 is blank). -2. D2 must always be sorted first by field #1 (Escrow Number) and second by field #4 (Age). This imposes a unique sort. -3. From one block to the next, every entry's "Age" field must increase by exactly 1. -4. From one block to the next, entries are only removed from D2 (in the very next block) if: -* * "Age" = "MaxAge". -* * If the block contains a txn who's blinded txID matches WT^. {{ This might be unnecessary, and a lot of work. }} -5. In addition, there are special rules for the allowed values in the "ACKs" field (field #3). See M4 below. - -#### M3 -- "Propose Withdrawal" - - 1-byte - OP_RETURN (0x6a) - 1-byte - Push the following 37 bytes (0x25) - 4-byte - Commitment header (0xD45AA943) - 33-byte - the critical bytes mentioned above (fields #1 and #2, to populate a new D2 entry) - - -#### New Block Validation Rules for M3 - -1. If the network detects a properly-formatted M3, it must add an entry to D2 in the very next block. The starting values of fields #3 and #4 are zero, and #5 is pulled over by extracting the relevant value from D1. -2. Each block can only contain one M3 per sidechain. - - -#### M4 -- "ACK Withdrawal" - -#### Very Little Info, Probably Calculable in Advance - -M4 is exceptional (in comparison to the other M's) in a few ways. First, its content is not stored anywhere, only the *hash* of its *effect* is stored (in a leaf of a merkle tree who's root is inserted into a mainchain coinbase). M4 alters the contents of D2 -- the *contents* of D2 are consensus critical, but M4 (the process by which nodes reach a new valid D2) can be anything. - -In fact, M4 can also be *nothing*. In other words, it may be optional. This is precisely because, from one block to the next, we have constrained D2 such that it is only allowed to change in a few ways. Therefore, the exhaustive set of "candidate D2s" can be precomputed by full nodes in advance. - -#### A Recent Change: Two Withdrawals at Once - -The following sections assume a maximum of one sucessful withdrawal per sidechain at a time. In other words, as WT^s are proposed, only one can make progress toward the finish line. As a result, a given side-to-main transfer will always take between 3 and 6 months. If there were more simulataneous withdrawals, the worst-case transfer duration would improve. - -![dots-image](/images/two-groups.png?raw=true) - -The worst-case withdrawal time obeys f(n)=3+(3/n) months, where n is the number of simultaneous withdrawals. - -N=2 is the most desirable choice for several reasons. First, it delievers the greatest marginal benefit (of 1.5 months). Later choices only deliver 0.5 and 0.25 marginal months. - -Second, n=2 can be implemented in a clever way: by allowing a withdrawal to freely advance, if and only if has an ACK-score of 6575 or greater, and if it also has the largest ACK score. In other words, the withdrawal that is furthest along can advance (or retreat) for free, if it has already made it at least halfway to the finish line. With this change, our new M4, is either an "abstain" for the sidechain (in which case nothing happens to any ACK scores), or else it will be in one of two cases: old_M4 + "the largest advances", or new_M4 + "the largest retreats". As a result the number of M4 possibilities (of which the next section is concerned) only increases by a factor of two (instead of exponentially). - -It is possible to troll this rule, by getting two (or even three) withdrawals to have 6575+ ACK scores, and then getting them to *tie* for first place. So, if there are any ties, the ability to "bonus move" is disabled until all ties are broken. - -#### How Hard is it to Guess M4? - -If there are n Escrows and m Withdrawals-per-escrow1, then there are (m+2)^n total candidates for the next D2. This is because, [per block per escrow], one of three things can happen: (1) one of the m withdrawal-candidates can be "ACK"ed (or "upvoted" or "promoted"), which automatically downvotes the others; or (2) all withdrawal-candidates can be downvoted, or finally (3) the miners can abstain from voting on the escrow's withdrawals altogether, leaving the tallies the same. - -First, for nodes which validate all sidechains (assuming these escrows are sidechains), this simplifies to 2^n -- these nodes only have to choose between the single honest choice (on one hand) or an abstention (on the other). Second, even for nodes that don't validate any sidechains, the number of candidates might be reduced from m^n to 3^n, by making a simplifying assumption: whichever withdrawal was most recently added/upvoted, is likely to be the one which is upvoted next. - -Of course, that is still O(k^n) for n sidechains, which isn't great2. If the "D2 update" cannot be guessed, it must be transmitted in some way. - -#### Giving Up and Getting M4 the Old Fashioned Way - -Two examples for transmitting it are below: - -"Short Form" (Assumes there are no more than 254 active withdrawal-attempts per account) - - 4-byte - Message identifier (0x????????) - 1-byte - Version of this message - N-byte - N is the total number of active accounts ("sidechains"), each byte specifies the position of the single WT that was "upvoted". A value of 0 indicates "downvote everything", a value of 255 indicates abstention. - -"Long Form" (Makes no assumptions about anything) - - 4-byte - Message identifier (0x????????) - 1-byte - Version of this message - 1-byte - Length (in bytes) of this message; total number of withdrawal attempts; y = ceiling( sum_i(m_i +2)/8 ). Nodes should already know what length to expect, because they know the sequence of M3s and therefore the vector of WT^s. - Y-byte - stream of bits (not bytes), with a 1 indicating the position of the chosen action [downvote all, abstain, upvote1, upvote2, ...] - - -If the message is very very large, then nodes may not want to broadcast it. This opens up an "exhaustion attack"2, in which many miners create bad WT^s, vote on these randomly, and then refuse to broadcast their votes. Fortunately, even for a worst-case scenario of 200 sidechains and 1,000 withdrawal-attempts per sidechain, honest nodes can communicate a long form M4 with each other by using just 25,056 bytes per block [4+1+1+(200\*(1000+1+1)/8)]. - -Today's pre-drivechain miners can already carry out a similar attack, by creating and including txns and then not broadcasting that part of the block to anyone. This is often characterized as a ["block publication incentive"](https://petertodd.org/2016/block-publication-incentives-for-miners), because in that case the prospect of exhaustively computing all possible transactions (to uncover the missing ones) is completely out of the question. - -However, message M4 is different from a withheld-txn, because M4 operates outside of the block's mandated information-processing limits (ie, outside the infamous 1 MB nonwitness blocksize limit). So we should examine the conditions under which M4 grows and shrinks, to ensure that we are not smuggling in a tremendous burden on full nodes. - -Under adversarial conditions, to lengthen a long-form M4 by one bit per block, for C blocks, the attacker must pay 312 bits (39 bytes) one time (to embed a new M3 message). The value C is the length of the sidechain's voting period, which varies but which I expect to be approximately 8,064 (and which could theoretically be as high as 65,536). Thus the attacker can burden nodes disproportionately, if (s)he wishes. - -Fortunately, the attack in question has no motivation (as far as I can tell). If the miner's goal is to trick rivals into mining on top of invalid blocks, he can already do this much more effectively with the unpublished-txn method (above). If instead he is just trying to harass nodes, then nodes may freely "downgrade" to earlier versions of the protocol, and simply ignore all drivechain-related messages. It seems that the attack could best be used in order to: make a large D2, make D2 confusing, sneak in votes for evil WT^ lurking in D2. Thus, the attack disables the transparency of the drivechain system, to some extent. The cost of the attack is forgone transaction fees, due to block space wasted on useless M3s. - -In practice, n is already capped, and miners may impose [on each other] a "soft cap" on m for their mutual protection. Thus, n and m might never get above 10 and 30, respectfully. In this case, the [Short Form, this time] M4 can never require more than 15 bytes per block, no matter what the attacker tries. - -In practice, m should always be 1 or 2, else something fishy is going on; and m can only inch up by 1 unit per block. So the system as a whole is still quite transparent, in that users are warned appropriately and well in advance. Attackers must invest upfront and they face an uphill climb, in order to eventually make things more expensive for a few others; defenders can wait-and-see if the attack looks like it will ever amount to anything before lifting a finger. - - -##### New Block Validation Rules (for D2 and, by implication, M4) - -From one block to the next, D2 can only be edited in a few strict ways: - -* Entries can only be added/removed from D2 if they meet the criteria above (in M3, and implicitly M1 and M2). -* The ACK-counter of any individual entry can only change by (-1,0,+1) relative to its previous entry. -* Within a sidechain group, upvoting one withdrawal (ACK=ACK+1) requires you to downvote all other withdrawals in that group. However, the minimum ACK value is zero (and, therefore, downvotes cannot reduce it below zero). - -##### Footnotes for M4 - -1 This represents the worst-case scenario is one where all the Withdrawals are spread evenly over each Sidechain. Under normal operations, there is no reason to expect the all sidechains will have the same number of withdrawals at any given time. In fact, under normal operations, the very *concept* of counting the withdrawals-per-sidechain should be a purposeless one, because there should only be *one* withdrawal at a time. Nonetheless we consider the worst case scenario here. - -2 Guessing becomes more computationally intensive in a highly adversarial situation where the "limited range" is intentionally expanded. In such a scenario, [a] there are many sidechains, and [b] miners voluntarily sacrifice their scarce block-space by creating a high number of (mutually-exclusive, and hence ultimately invalid) withdrawal attempts and putting these into coinbase transactions; and then agree to all [c] vote on these randomly (guaranteeing that all withdrawals fail, including any true withdrawals) and [d] successfully withhold their random voting strategies from nodes (even including spy-miner-nodes). Under this bizarre scenario, nodes may require computing resources which increase near-exponentially with the number of withdrawals, and it may take a long time for an ignorant node to exhaustively work out the underlying state of Withdrawal_DB. In this case, nodes may decide to temporarily stop validating such transactions (as if they had not yet upgraded to support this soft fork). - - - -### Depositing and Withdrawing (M5, M6) - - -Both M5 and M6 are regular Bitcoin txns. They are identified by meeting an important criteria: they select a one of the Critical TxID-index Pairs (a "CTIP") as one of their inputs. Deposits ("M5") are distinguished from withdrawals ("M6") by simply checking to see if money is "going in", or "out". In other words, we compare the BTC value of the original CTIP to that of new CTIP. If original <= new it is a deposit, if original > new then it is a withdrawal. - -The code that identifies sidechain withdrawal / deposit txns (by calculating how much value is being put into or taken out of a sidechain) can be seen here: https://github.com/drivechain-project/bitcoin/blob/mainchainBMM/src/validation.cpp#L351-L386 - -Such txns are forced (by consensus) to obey two additional criteria: - -1. They must contain an output paying "to" the Critical Address [probably in TxOut0]. -2. They must be accompanied by an update to this sidechain's Critical TxID-index Pair (CTIP). The new CTIP must be "this" txn itself. - -These criteria are enforced here https://github.com/drivechain-project/bitcoin/blob/mainchainBMM/src/validation.cpp#L440-L473 by checking that a deposit is paying back to the sidechain more than it is taking out, and completely rejecting any withdrawal from the mempool. And here https://github.com/drivechain-project/bitcoin/blob/mainchainBMM/src/validation.cpp#L1747-L1757 we allow for a withdrawal only once it has attained sufficient work score (ACKs). - -The purpose of this is to have all of the escrow's money (ie all of the sidechain's money) in one TxID, so that depositors immediately undo any UTXO bloat they may cause. This simplifies the withdrawal process, as there is no need to worry about cleaning up "dust deposits" (...and such cleaning can often result in headaches, for example where a withdrawal-txn is larger than 1MB in size, or else may only withdraw an arbitrarily limited amount of BTC). Notice that, unless we assume that an account will last forever, all utxos which are deposited must eventually be withdrawn by someone. Therefore, the relevant design criterion is not "efficiency" (total network cost) but rather "who should pay" (allocation of costs). - -#### M5. "Make a Deposit" -- a transfer of BTC from-main-to-side - -As far as mainchain consensus is concerned, there are no additional requirements. - -However, in practice there *are* additional mainchain requirements...specified by the escrow account, (ie specified by the "sidechain" or "virtual chain"). These requirements are not part of mainchain consensus and are allowed to be anything. In other words, the sidechain is free to invent any way to credit depositor's money -- M5 is fully customizable. - -One method, is for mainchain depositors to append a zero-value OP Return to a Deposit txn, so that the sidechain knows how to credit funds. Mainchain users must upgrade their wallet software, of course, (on an individual basis) in order to become aware of and take advantage of new deposit-methods. - -##### Inconvenient Race Condition - -The requirement that each hashrate escrow be linked to a single TxID does create an interesting inconvenience for depositors. If a user is slow to sign a txn after constructing it (perhaps because the user employs an air-gapped computer, etc), then the signed txn may no longer be valid. This is because the input it selects, may no longer be the Critical TxID (as "the" Critical TxID changes with each deposit). **Only one user can deposit at a time** (although many can deposit per block). As a result, the transaction must fail, and the user would need to be prompted to remake and resign the txn. If this is problem is too frustrating, users can always make main-to-side transfers using atomic cross chain swaps (or, the LN, if they already have a channel open on both chains). - -Fortunately, it is already a part of mainchain consensus that no two txns can spend the same TxID. The only new issue here is the confusion it might create for the user (hence the need for error messages and alternative deposit-methods). - - -#### M6. "Execute Withdrawal" -- a transfer of BTC from-side-to-main - -We come, finally, to the critical matter: where users can take their money *out* of the escrow account, and return it to the "regular" UTXO set. As previously mentioned, this txn is one which (a) spends from a CTIP and (b) reduces the quantity of BTC in an account's CTIP. Most of the work has already been done by D1, M3, M4, and D2. Furthermore, existing Bitcoin tx-rules prevent the sidechain from ever withdrawing more money than has been placed into it. - -From there, we merely introduce two final concepts: - -1. In each block, an entry in D2 is considered an "approved candidate" if the "ACKs" value is above 13140. -2. A "blinded TxID" is way of hashing the txn, in which we first overwrite some parts of the txn with zeros. Specifically, the first 36 bytes of "TxIn0" (the first input, including TxOutHash and TxOutIndex), as well as the first 8 bytes of "TxOut0" (the first output). - -Blinding is necessary because we allow each sidechain only one UTXO at a time. - -of our restriction of the account to a single UTXO-member. Because of this, during the ACKing process the withdrawal-txn (which is currently being ACKed) may change in two ways: the CTIP (which changes with each deposit), and the total quantity of BTC stored in the account (which arbitrarily increases with each new deposit). In other words, a withdrawal-attempt is created via M3, but this takes place many blocks before the withdrawal is actually included via M6. During this time, a single new deposit to the account would change its CTIP and its value. So, what do we ACK? Well, we ACK a "blinded" version of the withdrawal. This blinded version is stable because the dynamic parts are always overwritten with zeros. - -While we ACK a blinded WT^, what is actually included in the blockchain ("M6") is an unblinded WT^. Since each blinded WT^ could correspond to many different unblinded WT^s, we need to impose further restrictions on those unblinded WT^s that are finally included. First, we will force the final unblinded WT^ to spend the entire sidechain balance (by forcing sum(input_values) to equal sum(output_values)). To avoid withdrawing the entire sidechain balance with every withdrawal, we will, secondly, force the unblinded WT^ to create a new output which is itself a deposit to the sidechain it withdrew from (which nodes can check using D1's CTIP field). Unfortunately, these requirements eliminate the possibility of including a transaction fee, as traditionally calculated. So, finally, to compensate for *that*, txn fees are encoded explicitly as a withdrawal to OP_TRUE (which the main:block's miner can immediately claim). - -With all of this in place, the only requirements for inclusion in a block are these: - -1. "Be ACKed" -- The "blinded TxID" of this txn must be member of the "approved candidate" set in the D2 of this block. -2. "Return Change to Account" -- TxOut0 must pay to the "critical account" (see D1) that corresponds to the CTIP that was selected as a TxIn. -3. "Return *all* Change to Account" -- Sum of inputs must equal the sum of outputs. No traditional tx fee is possible. - -Finally, don't forget that M6 inherits the requirement (common to both M5 and M6) that the CTIP be selected as an input, and that the CTIP then be updated. In this case, we know that the critical index will be zero, so the new CTIP will be ("this TxID" (NOT blinded), 0). The TxID is NOT blinded because blinding is only for accumulating ACKs. - -As a result of these requirements, every single withdrawal-attempt will fail, unless an entry has been added to D2 and "ACKed" a sufficient number of times. - - - -Backward compatibility -======================== - -As a soft fork, older software will continue to operate without modification. Non-upgraded nodes will see a number of phenomena that they don't understand -- coinbase txns with non-txn data, value accumulating in anyone-can-spend UTXOs for months at a time, and then random amounts leaving the UTXO in single, infrequent bursts. However, this phenomena doesn't affect them or the validity of the money that they receive. - -( As a nice bonus, note that the sidechains themselves inherit a resistance to hard forks. The only way to guarantee that the WT^s reported by different clients will continue to match identically, is to upgrade sidechains via soft forks of themselves. ) - - -Deployment -=========== - -This BIP will be deployed by "version bits" BIP9 with the name "hrescrow" and using bit 4. - -``` -// Deployment of Drivechains (BIPX, BIPY) -consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].bit = 4; -consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].nStartTime = 1515974401; // January 15th, 2018. -consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].nTimeout = 1547510401; // January 15th, 2019. -``` - -Reference Implementation -========================== - -See: https://github.com/drivechain-project/bitcoin/tree/mainchainBMM - -Also, for interest, see an example sidechain here: https://github.com/drivechain-project/bitcoin/tree/sidechainBMM - - -References -============ - -See http://www.drivechain.info/literature/index.html - - -Credits -========= - -Thanks to everyone who contributed to the discussion, especially: ZmnSCPxj, Adam Back, Peter Todd, Dan Anderson, Sergio Demian Lerner, Chris Stewart, Matt Corallo, Sjors Provoost, Tier Nolan, Erik Aronesty, Jason Dreyzehner, Joe Miyamoto, Ben Goldhaber. - - - -Copyright -========== - -This BIP is licensed under the BSD 2-clause license. -- cgit v1.2.3 From 9207df7d054c21cb276b6873f009be0fda992d93 Mon Sep 17 00:00:00 2001 From: Paul Sztorc Date: Sat, 10 Feb 2018 17:00:23 -0500 Subject: fix mediawiki formatting --- bip-hashrate-escrows.mediawiki | 189 +++++++++++++++++++++++++++++------------ 1 file changed, 136 insertions(+), 53 deletions(-) diff --git a/bip-hashrate-escrows.mediawiki b/bip-hashrate-escrows.mediawiki index 0f2210c..82c436c 100644 --- a/bip-hashrate-escrows.mediawiki +++ b/bip-hashrate-escrows.mediawiki @@ -17,16 +17,16 @@ ==Abstract== -A "Hashrate Escrow" is a clearer term for the concept of "locked to an SPV Proof", which is itself a restatement of the phrase "within a sidechain" as described in [a famous Oct 2014 paper](https://blockstream.com/sidechains.pdf) written partially by some Blockstream co-founders. +A "Hashrate Escrow" is a clearer term for the concept of "locked to an SPV Proof", which is itself a restatement of the phrase "within a sidechain" as described in [https://blockstream.com/sidechains.pdf a famous Oct 2014 paper] written partially by some Blockstream co-founders. A Hashrate Escrow resembles a 2-of-3 multisig escrow, where the 3rd party (who will arbitrate any disputes) is a decentralized group of people: the dynamic-membership set of Bitcoin Miners. However, the 3rd party does not sign escrow-withdrawal transactions with a private key. Instead, these are "signed" by directing hashpower over them for a period of time. -This project has [a website](http://www.drivechain.info/) which includes [a FAQ](http://www.drivechain.info/faq/index.html). +This project has [http://www.drivechain.info/ a website] which includes [http://www.drivechain.info/faq/index.html a FAQ]. ==Motivation== -In practice these escrows are likely to be "asymmetric sidechains" of Bitcoin (such as [Rootstock](http://www.rsk.co/)) or "virtual chains" within Bitcoin (such as [proposed by Blockstack](https://github.com/blockstack/virtualchain) in mid-2016). +In practice these escrows are likely to be "asymmetric sidechains" of Bitcoin (such as [http://www.rsk.co/ Rootstock]) or "virtual chains" within Bitcoin (such as [https://github.com/blockstack/virtualchain proposed by Blockstack] in mid-2016). Sidechains have many potential benefits, including: @@ -77,16 +77,55 @@ The table below enumerates the new database fields, their size in bytes, and the Note: Fields 6 through 9 have been intentionally removed. Previously, this section allowed miners to set and commit to voting/waiting periods. However, I have since standardized the periods: withdrawals expire after 6 months (26298 blocks), and they succeed if they ever achieve an ACK score of 13140 or higher. I have removed the waiting period, because anyone who adopts a policy of ignoring all withdrawals with fewer than 400 ACKs will automatically gain all of the benefits of the waiting period. The justification for this change is that it strongly implies that an attack on any one sidechain is an attack on all of them (in a sense, this change makes the "victimhood" of each sidechain "fungible"). -Field No. | Label | Bytes | Type | Description / Purpose -----------|-------|------|------|-------- -1 | Escrow Number | 1 | uInt | A number assigned to the entire escrow. Used to make it easy to refer to each escrow. -2 | Active\* | 2 | uInt | This counter starts at zero when a new entry is added, and increments as a new entry is supported by "M2". The entry either reaches the max value of 4032 (at which point the counter freezes), or else the entry is removed. This enforces the requirement "a soft fork is required to add a new escrow". -3 | Escrow Name/Description | 120 | string | A human-readable name and description of the sidechain. More than enough space to hold a 32 byte hash. Helps prevent destructive interference among sidechains (see below). -4 | Critical Private Key | 32 | hex | This is the Double-SHA256 of the binary release of the reference software. It ties the sidechain to a specific software release, and doubles as a way of signing withdrawals (see below). -5 | Critical Address\* | 32 | string | This is derived by taking f(x) = ImportPrivateKey(HexToBase58Check(x)) of field #3. It is intentionally in the human-readable format {{although it could be 25 bytes of binary}}. -10 | "CTIP" -- Part 1 "TxID"\* | 32 | hex | The CTIP, or "Critical (TxID, Index) Pair" is a variable for keeping track of where the escrow's money is (ie, which member of the UTXO set). -11 | "CTIP" -- Part 2 "Index"\* | 4 | hex | Of the CTIP, this is second element of the pair: the Index. See #10 above. - +{| class="wikitable" +! Field No. +! Label +! Bytes +! Type +! Description / Purpose +|- +| 1 +| Escrow Number +| 1 +| uInt +| A number assigned to the entire escrow. Used to make it easy to refer to each escrow. +|- +| 2 +| Active* +| 2 +| uInt +| This counter starts at zero when a new entry is added, and increments as a new entry is supported by "M2". The entry either reaches the max value of 4032 (at which point the counter freezes), or else the entry is removed. This enforces the requirement "a soft fork is required to add a new escrow". +|- +| 3 +| Escrow Name/Description +| 120 +| string +| A human-readable name and description of the sidechain. More than enough space to hold a 32 byte hash. Helps prevent destructive interference among sidechains (see below). +|- +| 4 +| Critical Private Key +| 32 +| hex +| This is the Double-SHA256 of the binary release of the reference software. It ties the sidechain to a specific software release, and doubles as a way of signing withdrawals (see below). +|- +| 5 +| Critical Address* +| 32 +| string +| This is derived by taking f(x) = ImportPrivateKey(HexToBase58Check(x)) of field #3. It is intentionally in the human-readable format {{although it could be 25 bytes of binary}}. +|- +| 10 +| "CTIP" -- Part 1 "TxID"* +| 32 +| hex +| The CTIP, or "Critical (TxID, Index) Pair" is a variable for keeping track of where the escrow's money is (ie, which member of the UTXO set). +|- +| 11 +| "CTIP" -- Part 2 "Index"* +| 4 +| hex +| Of the CTIP, this is second element of the pair: the Index. See #10 above. +|} \* Denotes a "convenience field", the entry for this field is derived from other fields, or from the blockchain-state itself. The escrow-creator does not need to provide these values in M1 (or anywhere). @@ -94,10 +133,10 @@ Escrow_DB requires 223 bytes [1+120+32+32+2+32+4] for each escrow in the main bl ====Notes on D1==== -1. D1 will always exist. -2. D1 will always have a unique sort (first by "Escrow Number" (field #1), and second by "Active" (field #2)). There is only ever one (escrow number, Active) pair. -3. D1 is updated according to M1 and M2 (below). -4. If a new entry is added to D1 with an "Escrow Number" that is already in use, then this entry will either eventually be removed (because it was not supported with an M2), or it will eventually overwrite the old entry (if it *was* supported via M2). +# D1 will always exist. +# D1 will always have a unique sort (first by "Escrow Number" (field #1), and second by "Active" (field #2)). There is only ever one (escrow number, Active) pair. +# D1 is updated according to M1 and M2 (below). +# If a new entry is added to D1 with an "Escrow Number" that is already in use, then this entry will either eventually be removed (because it was not supported with an M2), or it will eventually overwrite the old entry (if it *was* supported via M2). ====Notes on D1==== @@ -108,7 +147,7 @@ Miners have always upgraded their software according to criteria that are known However, this soft fork imposes two new criteria upon them. First: miners should only upgrade their software, if any modification to the portfolio of sidechains [that are added/removed in the upgrade] can be expected to increase miner wealth. Trivially, this implies that miners should make sure that the upgrade doesn't overwrite (and destroy) an existing sidechain that they like! But, more seriously, it implies that miners should take an interest in what the sidechain is doing to the mainchain and other sidechains (see below). -##### Destructive Sidechain Interference +===== Destructive Sidechain Interference ===== People frequently emphasize that miners should have "as little control" as possible. It is a very safe claim to make, and a very easy sentence to write. Much harder is to determine exactly what this minimum value is, and how to achieve it. Harder still is to untie the knot of who is actually controlling what, in a decentralized, interacting system. @@ -147,11 +186,11 @@ Currently, we use a process which may be suboptimal. It is that we *literally si ==== New Block Validation Rules ==== -1. If the network detects a properly-formatted M1, it must add an entry to D1, into the very next block, with the following initial values: -* Field #5 will be calculated as per [version 1 Bitcoin addresses](https://en.bitcoin.it/wiki/Technical_background_of_version_1_Bitcoin_addresses), but with a prefix of "4" instead of "1". -* Field #9 will be derived from #7 and #8 using math. -* The initial values of Fields #10, #11, and #12 are set to zero. -2. Only one M1 (of any kind) can be added into a block at a time. +# If the network detects a properly-formatted M1, it must add an entry to D1, into the very next block, with the following initial values: +## Field #5 will be calculated as per [https://en.bitcoin.it/wiki/Technical_background_of_version_1_Bitcoin_addresses version 1 Bitcoin addresses], but with a prefix of "4" instead of "1". +## Field #9 will be derived from #7 and #8 using math. +## The initial values of Fields #10, #11, and #12 are set to zero. +# Only one M1 (of any kind) can be added into a block at a time. ==== Notes on M1 ==== @@ -180,16 +219,61 @@ The escrow will "reuse" the same address over and over. But notice that there is The table below enumerates the database fields, their size (in bytes), type and purpose. -Field No. | Label | Bytes | Type | Description / Purpose -----------|-------|------|------|----------------------- -1 | Escrow Number | 1 | uInt | Links the withdrawal-request to a specific escrow. -2 | WT^ | 32 | hex | This is a "blinded transaction id" (ie, the double-Sha256 of a txn that has had two fields zeroed out, see M6) of a withdrawal-attempt. -3 | ACKs\* | 2 | uInt | The current total number of "votes", this starts at 0 and remains there throughout the waiting period. -4 | Age\* | 3 | uInt | Total duration of time, in blocks, that this WT^ has been inside of D2. -5 | Waiting Period\* | 2 | uInt | Total duration of time, in blocks, that this entry must sit idle, before it can begin to accumulate ACKs/NACKs. Pulled from D1's field #6. -6 | Max Age\* | 3 | uInt | Determined by summing (D1's field #6) and (D1's field #7). -7 | Threshold\* | 2 | uInt | Total ACKs needed, this is pulled from D1's field #9. -8 | Approved\* | 1 | boolean | True while ACKs > Threshold, False otherwise. +{| class="wikitable" +! Field No. +! Label +! Bytes +! Type +! Description / Purpose +|- +| 1 +| Escrow Number +| 1 +| uInt +| Links the withdrawal-request to a specific escrow. +|- +| 2 +| WT^ +| 32 +| hex +| This is a "blinded transaction id" (ie, the double-Sha256 of a txn that has had two fields zeroed out, see M6) of a withdrawal-attempt. +|- +| 3 +| ACKs* +| 2 +| uInt +| The current total number of "votes", this starts at 0 and remains there throughout the waiting period. +|- +| 4 +| Age* +| 3 +| uInt +| Total duration of time, in blocks, that this WT^ has been inside of D2. +|- +| 5 +| Waiting Period* +| 2 +| uInt +| Total duration of time, in blocks, that this entry must sit idle, before it can begin to accumulate ACKs/NACKs. Pulled from D1's field #6. +|- +| 6 +| Max Age* +| 3 +| uInt +| Determined by summing (D1's field #6) and (D1's field #7). +|- +| 7 +| Threshold* +| 2 +| uInt +| Total ACKs needed, this is pulled from D1's field #9. +|- +| 8 +| Approved* +| 1 +| boolean +| True while ACKs > Threshold, False otherwise. +|} \* Denotes a "convenience field" (see above). @@ -197,13 +281,13 @@ Withdrawal_DB requires 46 bytes [1+32+2+3+2+3+2+1] per entry. Of these, 13 bytes ==== New Block Validation Rules for D2 ==== -1. In each block, a hash commitment to D2 must always exist (even if D2 is blank). -2. D2 must always be sorted first by field #1 (Escrow Number) and second by field #4 (Age). This imposes a unique sort. -3. From one block to the next, every entry's "Age" field must increase by exactly 1. -4. From one block to the next, entries are only removed from D2 (in the very next block) if: -* * "Age" = "MaxAge". -* * If the block contains a txn who's blinded txID matches WT^. {{ This might be unnecessary, and a lot of work. }} -5. In addition, there are special rules for the allowed values in the "ACKs" field (field #3). See M4 below. +# In each block, a hash commitment to D2 must always exist (even if D2 is blank). +# D2 must always be sorted first by field #1 (Escrow Number) and second by field #4 (Age). This imposes a unique sort. +# From one block to the next, every entry's "Age" field must increase by exactly 1. +# From one block to the next, entries are only removed from D2 (in the very next block) if: +## "Age" = "MaxAge". +## If the block contains a txn who's blinded txID matches WT^. {{ This might be unnecessary, and a lot of work. }} +# In addition, there are special rules for the allowed values in the "ACKs" field (field #3). See M4 below. ==== M3 -- "Propose Withdrawal" ==== @@ -215,8 +299,8 @@ Withdrawal_DB requires 46 bytes [1+32+2+3+2+3+2+1] per entry. Of these, 13 bytes ==== New Block Validation Rules for M3 ==== -1. If the network detects a properly-formatted M3, it must add an entry to D2 in the very next block. The starting values of fields #3 and #4 are zero, and #5 is pulled over by extracting the relevant value from D1. -2. Each block can only contain one M3 per sidechain. +# If the network detects a properly-formatted M3, it must add an entry to D2 in the very next block. The starting values of fields #3 and #4 are zero, and #5 is pulled over by extracting the relevant value from D1. +# Each block can only contain one M3 per sidechain. ==== M4 -- "ACK Withdrawal" ==== @@ -231,7 +315,7 @@ In fact, M4 can also be *nothing*. In other words, it may be optional. This is p The following sections assume a maximum of one sucessful withdrawal per sidechain at a time. In other words, as WT^s are proposed, only one can make progress toward the finish line. As a result, a given side-to-main transfer will always take between 3 and 6 months. If there were more simulataneous withdrawals, the worst-case transfer duration would improve. -![dots-image](/images/two-groups.png?raw=true) + The worst-case withdrawal time obeys f(n)=3+(3/n) months, where n is the number of simultaneous withdrawals. @@ -269,7 +353,7 @@ Two examples for transmitting it are below: If the message is very very large, then nodes may not want to broadcast it. This opens up an "exhaustion attack"2, in which many miners create bad WT^s, vote on these randomly, and then refuse to broadcast their votes. Fortunately, even for a worst-case scenario of 200 sidechains and 1,000 withdrawal-attempts per sidechain, honest nodes can communicate a long form M4 with each other by using just 25,056 bytes per block [4+1+1+(200\*(1000+1+1)/8)]. -Today's pre-drivechain miners can already carry out a similar attack, by creating and including txns and then not broadcasting that part of the block to anyone. This is often characterized as a ["block publication incentive"](https://petertodd.org/2016/block-publication-incentives-for-miners), because in that case the prospect of exhaustively computing all possible transactions (to uncover the missing ones) is completely out of the question. +Today's pre-drivechain miners can already carry out a similar attack, by creating and including txns and then not broadcasting that part of the block to anyone. This is often characterized as a [https://petertodd.org/2016/block-publication-incentives-for-miners "block publication incentive"], because in that case the prospect of exhaustively computing all possible transactions (to uncover the missing ones) is completely out of the question. However, message M4 is different from a withheld-txn, because M4 operates outside of the block's mandated information-processing limits (ie, outside the infamous 1 MB nonwitness blocksize limit). So we should examine the conditions under which M4 grows and shrinks, to ensure that we are not smuggling in a tremendous burden on full nodes. @@ -307,10 +391,10 @@ The code that identifies sidechain withdrawal / deposit txns (by calculating how Such txns are forced (by consensus) to obey two additional criteria: -1. They must contain an output paying "to" the Critical Address [probably in TxOut0]. -2. They must be accompanied by an update to this sidechain's Critical TxID-index Pair (CTIP). The new CTIP must be "this" txn itself. +# They must contain an output paying "to" the Critical Address [probably in TxOut0]. +# They must be accompanied by an update to this sidechain's Critical TxID-index Pair (CTIP). The new CTIP must be "this" txn itself. -These criteria are enforced here https://github.com/drivechain-project/bitcoin/blob/mainchainBMM/src/validation.cpp#L440-L473 by checking that a deposit is paying back to the sidechain more than it is taking out, and completely rejecting any withdrawal from the mempool. And here https://github.com/drivechain-project/bitcoin/blob/mainchainBMM/src/validation.cpp#L1747-L1757 we allow for a withdrawal only once it has attained sufficient work score (ACKs). +These criteria are enforced [https://github.com/drivechain-project/bitcoin/blob/mainchainBMM/src/validation.cpp#L440-L473 here] by checking that a deposit is paying back to the sidechain more than it is taking out, and completely rejecting any withdrawal from the mempool. And [https://github.com/drivechain-project/bitcoin/blob/mainchainBMM/src/validation.cpp#L1747-L1757 here] we allow for a withdrawal only once it has attained sufficient work score (ACKs). The purpose of this is to have all of the escrow's money (ie all of the sidechain's money) in one TxID, so that depositors immediately undo any UTXO bloat they may cause. This simplifies the withdrawal process, as there is no need to worry about cleaning up "dust deposits" (...and such cleaning can often result in headaches, for example where a withdrawal-txn is larger than 1MB in size, or else may only withdraw an arbitrarily limited amount of BTC). Notice that, unless we assume that an account will last forever, all utxos which are deposited must eventually be withdrawn by someone. Therefore, the relevant design criterion is not "efficiency" (total network cost) but rather "who should pay" (allocation of costs). @@ -335,8 +419,8 @@ We come, finally, to the critical matter: where users can take their money *out* From there, we merely introduce two final concepts: -1. In each block, an entry in D2 is considered an "approved candidate" if the "ACKs" value is above 13140. -2. A "blinded TxID" is way of hashing the txn, in which we first overwrite some parts of the txn with zeros. Specifically, the first 36 bytes of "TxIn0" (the first input, including TxOutHash and TxOutIndex), as well as the first 8 bytes of "TxOut0" (the first output). +# In each block, an entry in D2 is considered an "approved candidate" if the "ACKs" value is above 13140. +# A "blinded TxID" is way of hashing the txn, in which we first overwrite some parts of the txn with zeros. Specifically, the first 36 bytes of "TxIn0" (the first input, including TxOutHash and TxOutIndex), as well as the first 8 bytes of "TxOut0" (the first output). Blinding is necessary because we allow each sidechain only one UTXO at a time. @@ -346,9 +430,9 @@ While we ACK a blinded WT^, what is actually included in the blockchain ("M6") i With all of this in place, the only requirements for inclusion in a block are these: -1. "Be ACKed" -- The "blinded TxID" of this txn must be member of the "approved candidate" set in the D2 of this block. -2. "Return Change to Account" -- TxOut0 must pay to the "critical account" (see D1) that corresponds to the CTIP that was selected as a TxIn. -3. "Return *all* Change to Account" -- Sum of inputs must equal the sum of outputs. No traditional tx fee is possible. +# "Be ACKed" -- The "blinded TxID" of this txn must be member of the "approved candidate" set in the D2 of this block. +# "Return Change to Account" -- TxOut0 must pay to the "critical account" (see D1) that corresponds to the CTIP that was selected as a TxIn. +# "Return *all* Change to Account" -- Sum of inputs must equal the sum of outputs. No traditional tx fee is possible. Finally, don't forget that M6 inherits the requirement (common to both M5 and M6) that the CTIP be selected as an input, and that the CTIP then be updated. In this case, we know that the critical index will be zero, so the new CTIP will be ("this TxID" (NOT blinded), 0). The TxID is NOT blinded because blinding is only for accumulating ACKs. @@ -394,7 +478,6 @@ See http://www.drivechain.info/literature/index.html Thanks to everyone who contributed to the discussion, especially: ZmnSCPxj, Adam Back, Peter Todd, Dan Anderson, Sergio Demian Lerner, Chris Stewart, Matt Corallo, Sjors Provoost, Tier Nolan, Erik Aronesty, Jason Dreyzehner, Joe Miyamoto, Ben Goldhaber. - ==Copyright== This BIP is licensed under the BSD 2-clause license. -- cgit v1.2.3 From 486505b8fad62cb3c2f01b1a0eb79b1912196fda Mon Sep 17 00:00:00 2001 From: Paul Sztorc Date: Sat, 10 Feb 2018 17:02:33 -0500 Subject: typos --- bip-hashrate-escrows.mediawiki | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/bip-hashrate-escrows.mediawiki b/bip-hashrate-escrows.mediawiki index 82c436c..0987bf5 100644 --- a/bip-hashrate-escrows.mediawiki +++ b/bip-hashrate-escrows.mediawiki @@ -315,7 +315,7 @@ In fact, M4 can also be *nothing*. In other words, it may be optional. This is p The following sections assume a maximum of one sucessful withdrawal per sidechain at a time. In other words, as WT^s are proposed, only one can make progress toward the finish line. As a result, a given side-to-main transfer will always take between 3 and 6 months. If there were more simulataneous withdrawals, the worst-case transfer duration would improve. - + The worst-case withdrawal time obeys f(n)=3+(3/n) months, where n is the number of simultaneous withdrawals. @@ -453,12 +453,12 @@ As a soft fork, older software will continue to operate without modification. No This BIP will be deployed by "version bits" BIP9 with the name "hrescrow" and using bit 4. -``` +
 // Deployment of Drivechains (BIPX, BIPY)
 consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].bit = 4;
 consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].nStartTime = 1515974401; // January 15th, 2018.
 consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].nTimeout = 1547510401; // January 15th, 2019.
-```
+
==Reference Implementation== -- cgit v1.2.3 From 68918ad24a568a302bf6e54932a38d3b73ec3d65 Mon Sep 17 00:00:00 2001 From: Paul Sztorc Date: Sat, 10 Feb 2018 17:15:21 -0500 Subject: resync --- bip-blind-merged-mining.md | 329 ++++++++++++++++ bip-blind-merged-mining/bmm-dots-examples.png | Bin 0 -> 41116 bytes bip-blind-merged-mining/images.txt | 1 + bip-blind-merged-mining/witness-vs-critical.png | Bin 0 -> 67570 bytes bip-hashrate-escrows.mediawiki | 483 ------------------------ bip-hashrate-escrows/images.txt | 1 - bip-hashrate-escrows/two-groups.png | Bin 39695 -> 0 bytes 7 files changed, 330 insertions(+), 484 deletions(-) create mode 100644 bip-blind-merged-mining.md create mode 100644 bip-blind-merged-mining/bmm-dots-examples.png create mode 100644 bip-blind-merged-mining/images.txt create mode 100644 bip-blind-merged-mining/witness-vs-critical.png delete mode 100644 bip-hashrate-escrows.mediawiki delete mode 100644 bip-hashrate-escrows/images.txt delete mode 100644 bip-hashrate-escrows/two-groups.png diff --git a/bip-blind-merged-mining.md b/bip-blind-merged-mining.md new file mode 100644 index 0000000..f203e40 --- /dev/null +++ b/bip-blind-merged-mining.md @@ -0,0 +1,329 @@ + Drivechain Documentation -- Blind Merged Mining BIP + Paul Sztorc + November 17, 2017 + Document 3 of 3 + v4.1 + + +Header +======= + + BIP: ???? + Layer: Consensus (soft fork) + Title: Blind Merged Mining (Consensus layer) + Author: Paul Sztorc + CryptAxe + Chris Stewart + Comments-Summary: No comments yet. + Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-??????? + Status: Draft + Type: Standards Track + Created: 2017-10-24 + License: BSD-2-Clause + + +Abstract +========== + +Blind Merged Mining (BMM) is a way of mining special extension blocks, ie "sidechains". It produces strong guarantees that the block is valid, for *any* arbitrary set of rules; and yet it does so without requiring miners to actually do any validation on the block whatsoever. + +BMM actually is a process that spans two or more chains. For an explanation of the "whole picture", please see [this post](http://www.truthcoin.info/blog/blind-merged-mining/). Here we focus on the modifications to mainchain Bitcoin. + +To support BMM, the mainchain is asked to accomplish two goals: +1. Track a set of ordered hashes (the merged-mining). +2. Allow miners to "sell" the act of finding a sidechain block (through the use of a new extended serialization transaction type). + +These goals are accomplished by forcing nodes to validate two new messages (M7, M8), and track data in one new database (D3). + + +Motivation +============ + +Regular "Merged-Mining" (MM) allows miners to reuse their hashing work to secure other chains (for example, as in Namecoin). However, traditional MM has two drawbacks: + +1. Miners must run a full node of the other chain. (This is because [while miners can effortlessly create the block] miners will not create a valid payment to themselves, unless the block that they MM is a valid one. Therefore, miners must assemble a *valid* block first, then MM it.) +2. Miners are paid on the other chain, not on the regular BTC mainchain. For example, miners who MM Namecoin will earn NMC (and they will need to sell the NMC for BTC, before selling the BTC in order to pay for electricity). + +Blind Merged-Mining (BMM) attempts to address those shortcomings. + + +Specification +============ + +Note: This document uses the notation side:\* and main:\* in front of otherwise-ambiguous words (such as "block", "node", or "chain"), to distinguish the mainchain version from its sidechain counterpart. + +As stated above, we have two goals: [1] create and monitor an alt-chain (defined only by a deterministic list of hashes), and [2] allow miners to "sell" the act of finding a sidechain block (through the use of a new extended serialization transaction type). + +### Sidechain Critical Data ("Sidechain Mini-Header") + +Specifically, per side:block per side:chain, we track the following 35 bytes of information: + + 1-byte - ChainIndex (known as "Account Number" in hashrate-escrows.md , or as "Sidechain Number") + 32-bytes - sideHeaderHash (also known as "h*" / hashCritical, the hash of the sidechain block) + 2-bytes - prevBlockRef (an index which points to this side:block's parent side:block) + +The **ChainIndex** indicates which sidechain this critical data is relevant to. As we may eventually have more than one sidechain, this serves as an identifier similar to the Bitcoin network's magic bytes (0xF9BEB4D9). Drivechains however only need to use 1 byte for the identifier (there is a hard limit of 256 sidechains identified as 0-255). The **sideHeaderHash** is the hash of a side:block which will receive PoW via BMM. It serves a similar function to Bitcoin's "hashMerkleRoot", in that it contains the data for its blockchain. The **prevBlockRef** forms the set of headers into a blockchain structure by making each headers refer to one parent header. It is most similar to Bitcoin's hashPrevBlock. + +Where does this data come from, and how does it get around? + +#### Creating / Broadcasting This Data + +##### Creation + +By the time Blind Merged Mining can take place, the ChainIndex is globally known (it is the "Account Number" in D1 [see previous BIP], and "nSidechain" in the code). Each sidechain, when activated by soft fork, will take one of the 0-255 available indexes. + +The other two items, sideHeaderHash and prevBlockRef, are created by sidechain nodes. sideHeaderHash is quite straightforward -- side:nodes build side:blocks, and take the hash of these. + +The final item, prevBlockRef, is a little more complicated. It is an integer that counts the number of "skips" one must take in the side:chain in order to find the current side:block's parent block. In practice, this value will usually be zero. It will only be a value other than zero, in cases where invalid sidechain blocks have been mined, or when a side:node intentionally wants to orphan some side:blocks (if a side:node wants to orphan the most-recent N blocks, the value of the current block will be equal to N ; in the block after that it will be back to zero). + +![dots-image](/bip-blind-merged-mining/bmm-dots-examples.png?raw=true) + +Since the hashes themselves are already ordered by the mainchain, tracing the blockchain's path by index (prevBlockRef) will be the same as tracing it by identifying a list of hashes. In other words, the ordering given via each side:block's "prevBlockRef" will be isomorphic to an ordering given by each side:block's "prevSideHeaderHash" ... if "prevSideHeaderHash is defined to be the sidechain's equivalent of the mainchain's "prevBlockHash". It will be possible to freely convert from one to the other. See M8 to learn more about how these hashes are requested by sidechain block creators to be included in the mainchain. + +Now that we know what our critical data is, and how it is made, how is this data broadcast and stored? + +##### Broadcast + +Mainchain nodes are going to need this data later, so it must be easy to find. We will put it into the coinbase via OP RETURN. + +#### M7 -- "Blind-Mine the Sidechain(s)" + +Thus, (for n sidechains) we have a coinbase output with: + + 1-byte - OP_RETURN (0x6a) + 1-byte - Push the following (4+(n*35)) bytes (0x??) + 4-byte - Message header (0xD3407053) + (n*(32+5))-byte - A sequence of bytes, of the three Mini-Header items (h*, prevBlockRef, ChainIndex). + +( We assume that 5 bytes are used for the Critical Data bytes (non h* parts of the Sidechain Mini-Header). For 256 sidechains, a total of 9,478 bytes [1+1+4+256\*(32+5)] are required, conveniently just below the 10 KB scriptPubKey size limit.) + +This data is parsed by laying it in sequential 37-byte chunks (any remaining data --ie, some final chunk that is less than 37 bytes in length-- has no consensus meaning). + +Each 37-byte chunk is then parsed to obtain the data outlined above (in "Description"). If two 35-byte chunks being with the same "Sidechain number" (ie, if the two chunks have the same first byte), then only the first chunk has consensus meaning. + +We are left with, at most, one (h*, prevBlockRef) pair per sidechain per block. This data is added directly to D3, a new database. + +#### D3 -- "RecentSidechains_DB" + +To suit our purposes, the mainchain full nodes will need to keep track of the most recent 8000 (h\*, prevBlockRef) pairs. + +( This 8,000 figure is a tradeoff between decentralization (costs of running the main:node) and sidechain security -- it requires attackers to merged-mine 8,000 invalid blocks consecutively, in order to cause the sidechain to fail. The mainchain burden is minimal, so this figure might be raised to 12,000 or higher. ) + +Therefore, D3 would look something like this: + + + BlockHeight CB_Index SC_1 Blks_Atop_1 SC 2 Blks_Atop_2 SC 3 Blks_Atop_3 + --------- ------ ------ --------- ------ --------- ------ --------- + 1. 401,005 2 (h*, 0) 7985 (h*, 0) 1 (h*, 0) 0 + 2. 401,006 4 (h*, 0) 7984 (h*, 0) 0 (h*, 1) 7801 + 3. 401,007 2 (h*, 0) 7983 (h*, 5) 2027 (h*, 0) 0 + 4. 401,008 2 (h*, 0) 7982 (h*, 0) 2028 (h*, 1) 7800 + ... ... ) + 7999. 409,003 3 (h*, 0) 1 (h*, 0) 0 (h*, 0) 1 + 8000. 409,004 2 (h*, 0) 0 (h*, 1) 0 (h*, 0) 0 + + +When new sidechains (or "hashrate escrows") are soft-forked into existence, a new column is added to D3 (to contain any BMMing that might be done on it). + +For each sidechain we also track the field "Blocks Atop". This is the number of side:blocks that are "on top" of the specified side:block. These might be regarded as "side:confirmations" (pseudo-confirmations that are specific to each sidechain). + +D3 also contains a column (not shown) for each sidechain containing "prevSideBlockHash". This value is is either derived from prevBlockRef; or else it is given explicitly (in which case it is the converse: prevBlockRef is derived from prevSideBlockHash). + + +#### Coinbase Cache + +As mentioned above, M7s cause data to be added to D3. Recent D3 data is tracked by all mainchain nodes for a period of time. + +To efficiently keep track of the above data, without having to constantly load and process entire blocks from disk, we temporarily cache enough coinbases in the chain index to maintain D3. + + +### M8 -- Paying miners to include BMM data in their coinbase outputs + +This section introduces a new type of transaction, which allows sidechain block creators to request, and pay for, a critical hash to be included in a specific block by mainchain miners. See [the Blind Merged Mining spec](http://www.truthcoin.info/blog/blind-merged-mining/). This txn allows miners to "sell" the act of mining a sidechain block. By taking advantage of this option, miners earn tx fees for mining sidechains...truly "for free". They do not even need to run sidechain nodes, and the tx-fees they earn are in mainchain BTC. As a result, sidechains affect all miners equally and do not affect the mining ecosystem. + +M8 will ultimately come in two versions. The second version will be specialized for use in the Lightning Network and must use the full 32-byte prevBlockHash (ironically, this larger transaction is cheaper for the Bitcoin network to process, as it is completely off-chain). The first version of M8, in contrast, cannot be used inside the Lightning Network, but is slightly more space-efficient (using the 2 prevBlockRef bytes to maintain chain order). It is important to make both options available to the user, because some side:nodes may be unwilling or unable to open a payment channels with each main:miner. However, in the long run we expect the lightning version to be preferred. + +#### Setup + +We define **"Mary"** as a mainchain miner, and **"Simon"** as a sidechain node. + +The goal is to construct a payment from Simon to Mary, such that: + +1. If the critical data conditions are met, **Mary** can claim the outputs of the transaction with finality. +2. If the critical data conditions are not met, the outputs become immediately available again to **Simon**. + + +#### Goals (this is rather philosophical, and skippable) + +##### Immediate Expiration ("Fill-or-Kill") + +We would like to make special guarantees to the counterparties of this transaction. Specifically, instead of Simon making a "payment" to Mary, we prefer that Simon give Mary an "offer" (which she can either accept or decline). + +Crucially, we want Simon to safely make many offers to several different Mary's, in realtime (ie, quickly and off-chain). However, we ultimately want only one offer to be accepted, at most. In other words, we want Simon's offers to *immediately expire*. If only one offer can become a bona fide transaction, then Simon will feel comfortable making offers all day long. Because all of the Simons are making many offers, the Marys collectively gain access to a large set of offers to choose from. + +##### Forward Progress (The Need for a "Ratchet") + +The "ratchet" concept is an attempt to harmonize incentives among the main and side chain(s). +We need to ensure that a sidechain is making "forward progress", without tracking too much about the sidechain such that we burden Bitcoin (see [1] and [2]) all while still allowing the sidechain to reorganize [3]. + +* [1] https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-July/014789.html +* [2] http://www.drivechain.info/faq/index.html#what-is-the-difference-between-drivechain-and-extension-blocks +* [3] http://www.truthcoin.info/blog/blind-merged-mining/#handling-reorganizations + +The ratchet system must keep track of sidechain "mini-headers" (see Sidechain Critical Data ("Sidechain Mini-Header")) and count the "blocks atop" maturity of the related side:blocks. + +Simon's offer to Mary to include a critical hash in exchange for payment must be *atomic*. The "ratchet" concept helps to construct a very tight connection between two things: + +1. The sidechain-block-generator "Simon" paying himself the side:block's side:tx-fees (which he receives in 100 sidechain blocks (blocks atop) hence). +2. "Simon" making a mainchain main:btc payment to a mainchain miner "Mary". + +Either both of the two should succeed, or else both should jointly fail. + +However, absent our intervention, there are cases in which [2, the payment to Mary] succeeds but [1, side:tx-fees] fails. One such case is when a side:block contains unusually high side:tx-fees. Here, there will be many requests to include a critical hash in exchange for payment submitted to Mary, but only one can be included in each main:block per sidechain. Without an incentive to make "forward progress", Mary is likely to include one of the highest paying requests in the next main:block (and the main:block after that, and so on). Mary will "blindly" include high-paying requests for *older* blocks, unless something prevents her from doing so. + +To address these potential issues, we utilize the concept of "Blocks_Atop" (the "side:confirmations") that we mentioned earlier. As previously mentioned, Mary will not be able to spend Simon's M8 payment until satisfying the critical data requirements as well as the blocks atop (side:confirmations) requirement. + + +#### M8 -- The two forms of M8 transactions + +As previously mentioned, M8 can take two forms. The first does not require the Lightning Network, but it does have new requirements for Immediate Expiration (see above). The second inherits Immediate Expiration from the Lightning Network itself, but requires extra preparation and a different/larger message. + +Both forms require that certain Critical Data will be committed to within the coinbase of the block that the transaction is included in. For the non Lightning version, we have created a new extended serialization transaction type (very similar to how segwit handles witness data (the witness stack)). + +##### M8_V1 - No Lightning Network + +M8_V1 does not require the Lightning network but does have new requirements for validation. + +A M8_V1 TxOut is expected to contain: + + 1-byte - OP_RETURN (0x6a) + 1-byte - Push the following 36 bytes (0x24) + 4-byte - Message header (0xD1617368) + 32-bytes - h* side:block hash + 5~7-bytes - BMM request identifying bytes (0x00bf00) + prevBlockRef & ChainIndex (sidechain mini-header) + + +In the first version of M8, we need to introduce the concept of Immediate Expiration (see above). In other words, we need a way for Simon to construct many payments to multiple Marys, such that only one of these is ever included; and only then if Simon's txn is expected to coincide with the finding of Simon's side:block. + +We do this by imposing validity-rules on the txn itself: + +1. The txn's content, when examined, must match part of the main:block's content. Specifically, the (ChainIndex, h\*) pair of the txn, must match one of the (ChainIndex, h\*) pairs in the M7 of this main:block. +2. Only one payment per sidechain per main:block is valid. In other words, if 400 people all try to bm-mine the sidechain with ChainIndex==4, then not only is it the case that only one side_4:block can be found, but it is also the case that only the corresponding M8 txn can be included (out of all of the 400 M8s which are for ChainIndex==4). +3. Simon's txns must only be valid for the current block; afterward, they immediately expire. This is because Simon's intended prevBlockRef & side:block contents will most likely change from one main:block to the next. + +To impose new requirements on the transaction level (not the block level nor the TxOutput level), we borrow the "flag" trick from SegWit style transactions. If the flag is present, the transaction is examined for extra data, and if this data does not pass certain requirements, the transaction is invalid. With SegWit, this extra data is the signatures, and the extra requirements are the signatures' locations and validity. In the BMM-transactions, the extra data is the (ChainIndex, h\*) pair, which must meet the first two requirements (above) as well as the main:blocknumber, which must meet the third requirement (above). + +To impose new requirements at the transaction level, we borrow the dummy vin & "flag" trick from SegWit style transactions. If the flag is set to 2 (0010), the transaction contains Critical Data and requires that our new validation rules be met in order for the txn to be valid in a block. Unless all of the requirements for sidechain critical data transactions are met by the block it is included in, the transaction is invalid. With SegWit, this extra data is the segwit signature stack, and the extra requirements are the signatures' locations and validity. In the sidechain BMM critical data transactions, the extra data is the (ChainIndex, h\*) pair, which must meet the first two requirements (above) as well as the main:blocknumber, which must meet the third requirement (above). Note The main:blocknumber does not take up any additional space compared to a normal txn, as we reuse the locktime field for our purposes. + + + + +![extra-data-image](/bip-blind-merged-mining/witness-vs-critical.png?raw=true) + +This txn structure conserves main:blockspace, because it is the easiest way to refer to a previous sidechain block in 4 bytes, (prevBlockRef + FoK_nLockTime). Instead, we would need to use at least 32 bytes (prevSideBlockHash). + +These types of transactions have slightly different mempool behavior, and should probably be kept in a second mempool. These txns are received, checked immediately, and if valid they are evaluated for inclusion in a block. If they are not able to be included in the specific requested block (if the block height requested has been surpassed by the chain tip), they are discarded. In fact, after any main:block is found, everything in this "second mempool" can be discarded as new payments will be created immediately for the next block height. (This includes cases where the blockchain reorganizes.) There is no re-evaluation of the txns in this mempool ever -- they are evaluated once and then either included or discarded. To be clear, when the transaction is received we are able to evaluate its validity, and do not need to rescan these transactions again. + +Interestingly, these payments (M8) will *always* be directed to miners from non-miners. Therefore, non-mining nodes do not need to keep them in any mempool at all. Non-miner nodes can just wait for a block to be found, and check the txn then. These transactions more resemble a stock market's pit trades (in contrast, regular Bitcoin txns remind me more of paper checks). + +##### M8_V2 With Lightning + +M8_V2 requires having a LN-channel open with a miner. This may not always be practical (or even possible), especially today. + +A M8_V1 TxOut is expected to contain: + + 1-byte - OP_RETURN (0x6a) + 1-byte - Push the following 68 bytes (0x44) + 4-byte - Message header (0xD0520C6E) + 32-bytes - h* side:block hash + 32-bytes - prevSideBlockHash + 5~7-bytes - BMM request identifying bytes (0x00bf00) + prevBlockRef & ChainIndex (sidechain mini-header) + + +Notice that, in M8_V1, Simon could reuse the same h\* all he wanted, because only one M8_V1 could be included per main:block per sidechain. However, on the LN no such rule can be enforced, as the goal is to push everything off-chain and include *zero* M8s. So, we will never know what the M8s were or how many had an effect on anything. + +Therefore, Simon will need to ensure that he **gives each Mary a different h\***. Simon can easily do this, as he controls the side:block's contents and can simply increment a nonce -- this changes the side:block, and changes its hash (ie, changes h\*). + +With a unique h\* per Mary, and at most 1 h\* making it into a block (per sidechain), we can guarantee that only one of the M8_V2's critical data can be committed to in a single main:block. By giving each miner (who Simon has a payment channel open with) a different h*, Simon can figure out which miner followed through with the commit, and know that only one such commit went through. Furthermore, if this Simon's requested critical data is not found in a block, none of the M8_V2 payments will be spendable by the Mary(s), because none of the h\* in question have ever made it into D3 (which is always on-chain) and no blocks atop will be accumulated. + +That's probably confusing, so here is an example, in which: Simon starts with 13 BTC, Mary starts with 40 BTC, the side:block's tx-fees currently total 7.1 BTC, and Simon is keeping 0.1 BTC for himself and paying 7 BTC to Mary. + +We start with (I): + + Simon 13 in, Mary 40 in ; 53 in total + Simon's version [signed by Mary] + 13 ; to Simon if TimeLock=over; OR to Mary if SimonSig + 40 ; to Mary + Mary's version [signed by Simon] + 40 ; to me if TimeLock=over; OR to Simon if MarySig + 13 ; to Simon + + +And both parties move, from there to "M8_V2" (II): + + Simon 13 in, Mary 40 in ; 53 in total + Simon's version [signed by Mary] + 6 ; to Simon if TimeLock=over; OR to Mary if SimonSig + 40 ; to Mary + 7 ; to Mary if critical data requirements met; OR to Simon if LongTimeLock=over + Mary's version [signed by Simon] + 40 ; to Mary if TimeLock=over; OR to Simon if MarySig + 6 ; to Simon + 7 ; to Mary if critical data requirements met; OR to Simon if LongTimeLock=over + +From here, if the h\* side:block in question is BMMed, they can proceed to (III): + + Simon 13 in, Mary 40 in ; 53 in total + Simon's version [signed by Mary] + 6 ; to Simon if TimeLock=over; OR to Mary if SimonSig + 47 ; to Mary + Mary's version [signed by Simon] + 47 ; to me if TimeLock=over; OR to Simon if MarySig + 6 ; to Simon + +Although, if Simon proceeds immediately, he removes the protection of the 'ratchet'. Ie, Simon removes Mary's incentive to care about blocks being built on this side:block. If Simon's side:block is orphaned, he loses his 7 BTC. Simon can either play it safe, and wait the full 100 side:blocks before moving on (ie, moving on to the third LN txn, above); or else Simon can take the risk if he feels comfortable with it. + +If the h\* side:block is not found, then (II) and (III) are basically equivalent to each other. Simon and Mary could jointly reconstruct (I) and go back there, or they could proceed to a new version of II (with a different h\*, trying again with new side:block in the next main:block). + + + + +Deployment +=========== + +This BIP will be deployed by "version bits" BIP9 with the name "blindmm" and using bit 4. + +``` +// Deployment of Drivechains (BIPX, BIPY) +consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].bit = 4; +consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].nStartTime = 1515974401; // January 15th, 2018. +consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].nTimeout = 1547510401; // January 15th, 2019. +``` + +Reference Implementation +========================== + +See: https://github.com/drivechain-project/bitcoin/tree/mainchainBMM + +Also, for interest, see an example sidechain here: https://github.com/drivechain-project/bitcoin/tree/sidechainBMM + + +References +============ + +* http://www.drivechain.info/literature/index.html +* http://www.truthcoin.info/blog/blind-merged-mining/ +* https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-July/014789.html +* http://www.truthcoin.info/images/bmm-outline.txt + + +Thanks +========= + +Thanks to everyone who contributed to the discussion, especially: ZmnSCPxj, Adam Back, Peter Todd, Dan Anderson, Sergio Demian Lerner, Matt Corallo, Sjors Provoost, Tier Nolan, Erik Aronesty, Jason Dreyzehner, Joe Miyamoto, Ben Goldhaber. + + +Copyright +========== + +This BIP is licensed under the BSD 2-clause license. diff --git a/bip-blind-merged-mining/bmm-dots-examples.png b/bip-blind-merged-mining/bmm-dots-examples.png new file mode 100644 index 0000000..70f11f6 Binary files /dev/null and b/bip-blind-merged-mining/bmm-dots-examples.png differ diff --git a/bip-blind-merged-mining/images.txt b/bip-blind-merged-mining/images.txt new file mode 100644 index 0000000..2fbbf63 --- /dev/null +++ b/bip-blind-merged-mining/images.txt @@ -0,0 +1 @@ +Images used as reference in the documentation. diff --git a/bip-blind-merged-mining/witness-vs-critical.png b/bip-blind-merged-mining/witness-vs-critical.png new file mode 100644 index 0000000..1a2458d Binary files /dev/null and b/bip-blind-merged-mining/witness-vs-critical.png differ diff --git a/bip-hashrate-escrows.mediawiki b/bip-hashrate-escrows.mediawiki deleted file mode 100644 index 0987bf5..0000000 --- a/bip-hashrate-escrows.mediawiki +++ /dev/null @@ -1,483 +0,0 @@ - - -
-    BIP: ????
-    Layer: Consensus (soft fork)
-    Title: Hashrate Escrows (Consensus layer)
-    Author: Paul Sztorc 
-            CryptAxe 
-    Comments-Summary: No comments yet.
-    Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-???????
-    Status: Draft
-    Type: Standards Track
-    Created: 2017-08-14
-    License: BSD-2-Clause
-    Post-History: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-May/014364.html
-
- -==Abstract== - -A "Hashrate Escrow" is a clearer term for the concept of "locked to an SPV Proof", which is itself a restatement of the phrase "within a sidechain" as described in [https://blockstream.com/sidechains.pdf a famous Oct 2014 paper] written partially by some Blockstream co-founders. - -A Hashrate Escrow resembles a 2-of-3 multisig escrow, where the 3rd party (who will arbitrate any disputes) is a decentralized group of people: the dynamic-membership set of Bitcoin Miners. However, the 3rd party does not sign escrow-withdrawal transactions with a private key. Instead, these are "signed" by directing hashpower over them for a period of time. - -This project has [http://www.drivechain.info/ a website] which includes [http://www.drivechain.info/faq/index.html a FAQ]. - - -==Motivation== - -In practice these escrows are likely to be "asymmetric sidechains" of Bitcoin (such as [http://www.rsk.co/ Rootstock]) or "virtual chains" within Bitcoin (such as [https://github.com/blockstack/virtualchain proposed by Blockstack] in mid-2016). - -Sidechains have many potential benefits, including: - -1. Protect Bitcoin from competition from altcoins and spinoffs. Safely allow competing implementations (of *sidechains*). -2. Protect Bitcoin from hard fork campaigns. (Such campaigns represent an existential threat to Bitcoin, as well as an avenue for developer corruption.) -3. Help with review, by making it much easier for reviewers to ignore bad ideas. -4. Provide an avenue for good-but-confusing ideas to prove their value safely. - - - -==Specification== - -==== Components ==== - -Hashrate Escrows are built of two types of component: [1] new databases, and [2] new message-interpretations. - -===== 1. New Databases ===== - -* D1. "Escrow_DB" -- a database of "accounts" and their attributes. -* D2. "Withdrawal_DB" -- a database of pending withdrawals from these accounts, and their statuses. - -Please note that these structures (D1 and D2) will not literally exist anywhere in the blockchain. Instead they are constructed from messages...these messages, in contrast, *will* exist in the blockchain (with the exception of M4). - -===== 2. New Messages ===== - -* M1. "Propose New Escrow" -* M2. "ACK Escrow Proposal" -* M3. "Propose Withdrawal" -* M4. (implied) "ACK Withdrawal" -* M5. "Execute Deposit" -- a transfer of BTC from-main-to-side -* M6. "Execute Withdrawal" -- a transfer of BTC from-side-to-main - - -==== On the Resource Requirements of New Databases ==== - -The "new" databases are simply reinterpretations of data that are already contained elsewhere in the blockchain. Specifically, M1 M2 and M3 are all located in the block's coinbase txn, and M5 and M6 might be found in any regular txn. M4 is a special case and does not actually need to be included anywhere, so it is not. If you like, you can imagine that the M4s reside in an optional extension block. - -In other words, we just rearrange what is already there. Because of this, even though "new databases" are created and stored in memory, the existing bandwidth and storage limits are respected (although, see "M4" below). - - - - -=== Adding Sidechains and Tracking Them (D1, M1, M2) === - -==== D1 -- "Escrow_DB" ==== - -The table below enumerates the new database fields, their size in bytes, and their purpose. In general, an escrow designer (for example, a sidechain-designer), is free to choose any value for these. - -Note: Fields 6 through 9 have been intentionally removed. Previously, this section allowed miners to set and commit to voting/waiting periods. However, I have since standardized the periods: withdrawals expire after 6 months (26298 blocks), and they succeed if they ever achieve an ACK score of 13140 or higher. I have removed the waiting period, because anyone who adopts a policy of ignoring all withdrawals with fewer than 400 ACKs will automatically gain all of the benefits of the waiting period. The justification for this change is that it strongly implies that an attack on any one sidechain is an attack on all of them (in a sense, this change makes the "victimhood" of each sidechain "fungible"). - -{| class="wikitable" -! Field No. -! Label -! Bytes -! Type -! Description / Purpose -|- -| 1 -| Escrow Number -| 1 -| uInt -| A number assigned to the entire escrow. Used to make it easy to refer to each escrow. -|- -| 2 -| Active* -| 2 -| uInt -| This counter starts at zero when a new entry is added, and increments as a new entry is supported by "M2". The entry either reaches the max value of 4032 (at which point the counter freezes), or else the entry is removed. This enforces the requirement "a soft fork is required to add a new escrow". -|- -| 3 -| Escrow Name/Description -| 120 -| string -| A human-readable name and description of the sidechain. More than enough space to hold a 32 byte hash. Helps prevent destructive interference among sidechains (see below). -|- -| 4 -| Critical Private Key -| 32 -| hex -| This is the Double-SHA256 of the binary release of the reference software. It ties the sidechain to a specific software release, and doubles as a way of signing withdrawals (see below). -|- -| 5 -| Critical Address* -| 32 -| string -| This is derived by taking f(x) = ImportPrivateKey(HexToBase58Check(x)) of field #3. It is intentionally in the human-readable format {{although it could be 25 bytes of binary}}. -|- -| 10 -| "CTIP" -- Part 1 "TxID"* -| 32 -| hex -| The CTIP, or "Critical (TxID, Index) Pair" is a variable for keeping track of where the escrow's money is (ie, which member of the UTXO set). -|- -| 11 -| "CTIP" -- Part 2 "Index"* -| 4 -| hex -| Of the CTIP, this is second element of the pair: the Index. See #10 above. -|} - -\* Denotes a "convenience field", the entry for this field is derived from other fields, or from the blockchain-state itself. The escrow-creator does not need to provide these values in M1 (or anywhere). - -Escrow_DB requires 223 bytes [1+120+32+32+2+32+4] for each escrow in the main blockchain. Of these, 70 bytes [2+32+32+4] are merely for convenience. Therefore, a sidechain is defined (see "M1") by 153 bytes of information. - -====Notes on D1==== - -# D1 will always exist. -# D1 will always have a unique sort (first by "Escrow Number" (field #1), and second by "Active" (field #2)). There is only ever one (escrow number, Active) pair. -# D1 is updated according to M1 and M2 (below). -# If a new entry is added to D1 with an "Escrow Number" that is already in use, then this entry will either eventually be removed (because it was not supported with an M2), or it will eventually overwrite the old entry (if it *was* supported via M2). - - -====Notes on D1==== - -=====Obligations Placed on Miners===== - -Miners have always upgraded their software according to criteria that are known only to them (in other words, "whenever they want"). - -However, this soft fork imposes two new criteria upon them. First: miners should only upgrade their software, if any modification to the portfolio of sidechains [that are added/removed in the upgrade] can be expected to increase miner wealth. Trivially, this implies that miners should make sure that the upgrade doesn't overwrite (and destroy) an existing sidechain that they like! But, more seriously, it implies that miners should take an interest in what the sidechain is doing to the mainchain and other sidechains (see below). - -===== Destructive Sidechain Interference ===== - -People frequently emphasize that miners should have "as little control" as possible. It is a very safe claim to make, and a very easy sentence to write. Much harder is to determine exactly what this minimum value is, and how to achieve it. Harder still is to untie the knot of who is actually controlling what, in a decentralized, interacting system. - -Certainly, miners can not have "zero control" -- for that is the same as to just remove them from the system altogether. Some rules are enforced "on miners by nodes" (such as the infamous blocksize limit); other rules are enforced by nodes but are narrowly-controlled by miners (such as the proof-of-work itself, or the block's timestamp). Thirdly, some rules are enforced by both against each other (such as the rule against including invalid txns or double-spent txns), for mutual benefit. - -Some pause should be given, after one considers that the sidechain design goal is literally a piece of software that can do *anything*. Anything includes a great many things, many of which I demonstrate to be undesirable. Bitcoin itself does not allow "anything" -- it allows any person to transact, but, in contrast, it does not permit any person to double-spend. This is because "allowing anyone to do anything" is not viable in a world that contains undesirable interactions (what a libertarian might call "aggression") -- in the case of money, these are theft and counterfeiting. - -I have produced a comprehensive quantity of written material [1], presentations [2], etc [3] on exactly what the level of miner-control should be, and why. Specifically, I claim that **miners should be aware of the purpose of the sidechain, and they should reject sidechains which have an unclear purpose or which have a purpose that will lead to decrease in miner-wealth** (where wealth measured explicitly as: the estimated present value of the purchasing power of the blockchain's coinbase txns). I claim that this criterion is necessary because, just Original Bitcoin filters unwanted interactions among different BTC txns, so too much "Sidechain Bitcoin" filter out unwanted interactions among sidechain. - -* [1] http://www.truthcoin.info/blog/wise-contracts/ -* [2] https://www.youtube.com/watch?v=xGu0o8HH10U&index=1&list=PLw8-6ARlyVciMH79ZyLOpImsMug3LgNc4 -* [3] http://www.drivechain.info/literature/index.html - -Call it a "sidechain non-aggression principle", if you want. - -To the best of my knowledge, everyone who *has* reviewed this information as found the arguments to be acceptable. It has, also, changed a few minds (from "unacceptable" to "acceptable"). - - -===== ISSUE: "Signing" BTC Txns ===== - -Currently, we use a process which may be suboptimal. It is that we *literally sign* a txn with a globally and publicly known private key. But this is for convenience purposes -- the signature that is produced is not doing anything, and is therefore wasteful. Instead we may use OP_TRUE, but this might interfere with how we detect the sidechain's balance. I'm not sure what the best way is. Someone needs to investigate how to do this -- removing OP_CheckSig, etc. This is a TODO for sure, and an opportunity for someone to help. - - - -(The following messages were modeled on SegWit -- https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki#commitment-structure ) - - - -==== M1 -- "Propose New Sidechain" ==== - - 1-byte - OP_RETURN (0x6a) - 1-byte - Push the following 157 bytes (0x9d) - 4-byte - Commitment header (0x53707243) - 153-byte - the critical bytes mentioned above (fields #1, #3, and #4, to populate a new D1 entry) - - -==== New Block Validation Rules ==== - -# If the network detects a properly-formatted M1, it must add an entry to D1, into the very next block, with the following initial values: -## Field #5 will be calculated as per [https://en.bitcoin.it/wiki/Technical_background_of_version_1_Bitcoin_addresses version 1 Bitcoin addresses], but with a prefix of "4" instead of "1". -## Field #9 will be derived from #7 and #8 using math. -## The initial values of Fields #10, #11, and #12 are set to zero. -# Only one M1 (of any kind) can be added into a block at a time. - -==== Notes on M1 ==== - -The escrow will "reuse" the same address over and over. But notice that there is no privacy loss due to address reuse, because the address in question does not "belong" to any specific individual. Furthermore, the transactions in question could already be linked by just watching the Hashrate Escrow for deposits ...this, in fact, is exactly what a sidechain must do in order to make main-to-side transfers. - - -==== M2 -- "ACK Sidechain Proposal" ==== - - 1-byte - OP_RETURN (0x6a) - 1-byte - Push the following 29 bytes (0x1D) - 4-byte - Commitment header (0x53616343) - 25-byte - Commitment hash: RIPMD-160 of Sha256 of a given M1 (above) - -==== New Block Validation Rules ==== - -1. An entry will be removed from D1, unless that block contains an M2 supporting it (this will increment field #2), or the entry has accumulated 4032 M2s. Therefore, each entry needs to have 4032 *consecutive* M2s supporting it, else it will be removed from D1 and must start over. This requires all miners to agree to "soft fork" the sidechain into existence. This makes it easier to curate the list of active sidechains, and prevents destructive interference (see above) among contradictory sidechains. -2. Once the entry has 4032 M2s, then a miner must make a 0 btc payment to the Critical Address, thus activating the chain. This payment becomes the first "critical (TxID, index) pair" (or "CTIP"). -3. It is possible to "overwrite" a D1 slot -- this destroys the sidechain that is there, and replaces it with a new one. This is defined as requiring 6 months (26298 blocks) of M2s, instead of 4 weeks (4032). This possibility does not change the security assumptions, because we already assume that users have access to "magically true" bandwidth at a rate of 1 bit per ACK-cycle. (This is why the voting cycles should be large, 1 month Waiting, 2+ months Voting). - - - -=== Adding Withdrawal-Attempts and Tracking Their Success (D2, M3, M4) === - -==== D2 -- "Withdrawal_DB" ==== - -The table below enumerates the database fields, their size (in bytes), type and purpose. - - -{| class="wikitable" -! Field No. -! Label -! Bytes -! Type -! Description / Purpose -|- -| 1 -| Escrow Number -| 1 -| uInt -| Links the withdrawal-request to a specific escrow. -|- -| 2 -| WT^ -| 32 -| hex -| This is a "blinded transaction id" (ie, the double-Sha256 of a txn that has had two fields zeroed out, see M6) of a withdrawal-attempt. -|- -| 3 -| ACKs* -| 2 -| uInt -| The current total number of "votes", this starts at 0 and remains there throughout the waiting period. -|- -| 4 -| Age* -| 3 -| uInt -| Total duration of time, in blocks, that this WT^ has been inside of D2. -|- -| 5 -| Waiting Period* -| 2 -| uInt -| Total duration of time, in blocks, that this entry must sit idle, before it can begin to accumulate ACKs/NACKs. Pulled from D1's field #6. -|- -| 6 -| Max Age* -| 3 -| uInt -| Determined by summing (D1's field #6) and (D1's field #7). -|- -| 7 -| Threshold* -| 2 -| uInt -| Total ACKs needed, this is pulled from D1's field #9. -|- -| 8 -| Approved* -| 1 -| boolean -| True while ACKs > Threshold, False otherwise. -|} - -\* Denotes a "convenience field" (see above). - -Withdrawal_DB requires 46 bytes [1+32+2+3+2+3+2+1] per entry. Of these, 13 bytes ([2+3+2+3+2+1], all fields except #1 and #2) can be generated locally, leaving 33 critical bytes [1+32]. - -==== New Block Validation Rules for D2 ==== - -# In each block, a hash commitment to D2 must always exist (even if D2 is blank). -# D2 must always be sorted first by field #1 (Escrow Number) and second by field #4 (Age). This imposes a unique sort. -# From one block to the next, every entry's "Age" field must increase by exactly 1. -# From one block to the next, entries are only removed from D2 (in the very next block) if: -## "Age" = "MaxAge". -## If the block contains a txn who's blinded txID matches WT^. {{ This might be unnecessary, and a lot of work. }} -# In addition, there are special rules for the allowed values in the "ACKs" field (field #3). See M4 below. - -==== M3 -- "Propose Withdrawal" ==== - - 1-byte - OP_RETURN (0x6a) - 1-byte - Push the following 37 bytes (0x25) - 4-byte - Commitment header (0xD45AA943) - 33-byte - the critical bytes mentioned above (fields #1 and #2, to populate a new D2 entry) - - -==== New Block Validation Rules for M3 ==== - -# If the network detects a properly-formatted M3, it must add an entry to D2 in the very next block. The starting values of fields #3 and #4 are zero, and #5 is pulled over by extracting the relevant value from D1. -# Each block can only contain one M3 per sidechain. - - -==== M4 -- "ACK Withdrawal" ==== - -==== Very Little Info, Probably Calculable in Advance ==== - -M4 is exceptional (in comparison to the other M's) in a few ways. First, its content is not stored anywhere, only the *hash* of its *effect* is stored (in a leaf of a merkle tree who's root is inserted into a mainchain coinbase). M4 alters the contents of D2 -- the *contents* of D2 are consensus critical, but M4 (the process by which nodes reach a new valid D2) can be anything. - -In fact, M4 can also be *nothing*. In other words, it may be optional. This is precisely because, from one block to the next, we have constrained D2 such that it is only allowed to change in a few ways. Therefore, the exhaustive set of "candidate D2s" can be precomputed by full nodes in advance. - -==== A Recent Change: Two Withdrawals at Once ==== - -The following sections assume a maximum of one sucessful withdrawal per sidechain at a time. In other words, as WT^s are proposed, only one can make progress toward the finish line. As a result, a given side-to-main transfer will always take between 3 and 6 months. If there were more simulataneous withdrawals, the worst-case transfer duration would improve. - - - -The worst-case withdrawal time obeys f(n)=3+(3/n) months, where n is the number of simultaneous withdrawals. - -N=2 is the most desirable choice for several reasons. First, it delievers the greatest marginal benefit (of 1.5 months). Later choices only deliver 0.5 and 0.25 marginal months. - -Second, n=2 can be implemented in a clever way: by allowing a withdrawal to freely advance, if and only if has an ACK-score of 6575 or greater, and if it also has the largest ACK score. In other words, the withdrawal that is furthest along can advance (or retreat) for free, if it has already made it at least halfway to the finish line. With this change, our new M4, is either an "abstain" for the sidechain (in which case nothing happens to any ACK scores), or else it will be in one of two cases: old_M4 + "the largest advances", or new_M4 + "the largest retreats". As a result the number of M4 possibilities (of which the next section is concerned) only increases by a factor of two (instead of exponentially). - -It is possible to troll this rule, by getting two (or even three) withdrawals to have 6575+ ACK scores, and then getting them to *tie* for first place. So, if there are any ties, the ability to "bonus move" is disabled until all ties are broken. - -==== How Hard is it to Guess M4? ==== - -If there are n Escrows and m Withdrawals-per-escrow1, then there are (m+2)^n total candidates for the next D2. This is because, [per block per escrow], one of three things can happen: (1) one of the m withdrawal-candidates can be "ACK"ed (or "upvoted" or "promoted"), which automatically downvotes the others; or (2) all withdrawal-candidates can be downvoted, or finally (3) the miners can abstain from voting on the escrow's withdrawals altogether, leaving the tallies the same. - -First, for nodes which validate all sidechains (assuming these escrows are sidechains), this simplifies to 2^n -- these nodes only have to choose between the single honest choice (on one hand) or an abstention (on the other). Second, even for nodes that don't validate any sidechains, the number of candidates might be reduced from m^n to 3^n, by making a simplifying assumption: whichever withdrawal was most recently added/upvoted, is likely to be the one which is upvoted next. - -Of course, that is still O(k^n) for n sidechains, which isn't great2. If the "D2 update" cannot be guessed, it must be transmitted in some way. - -==== Giving Up and Getting M4 the Old Fashioned Way ==== - -Two examples for transmitting it are below: - -"Short Form" (Assumes there are no more than 254 active withdrawal-attempts per account) - - 4-byte - Message identifier (0x????????) - 1-byte - Version of this message - N-byte - N is the total number of active accounts ("sidechains"), each byte specifies the position of the single WT that was "upvoted". A value of 0 indicates "downvote everything", a value of 255 indicates abstention. - -"Long Form" (Makes no assumptions about anything) - - 4-byte - Message identifier (0x????????) - 1-byte - Version of this message - 1-byte - Length (in bytes) of this message; total number of withdrawal attempts; y = ceiling( sum_i(m_i +2)/8 ). Nodes should already know what length to expect, because they know the sequence of M3s and therefore the vector of WT^s. - Y-byte - stream of bits (not bytes), with a 1 indicating the position of the chosen action [downvote all, abstain, upvote1, upvote2, ...] - - -If the message is very very large, then nodes may not want to broadcast it. This opens up an "exhaustion attack"2, in which many miners create bad WT^s, vote on these randomly, and then refuse to broadcast their votes. Fortunately, even for a worst-case scenario of 200 sidechains and 1,000 withdrawal-attempts per sidechain, honest nodes can communicate a long form M4 with each other by using just 25,056 bytes per block [4+1+1+(200\*(1000+1+1)/8)]. - -Today's pre-drivechain miners can already carry out a similar attack, by creating and including txns and then not broadcasting that part of the block to anyone. This is often characterized as a [https://petertodd.org/2016/block-publication-incentives-for-miners "block publication incentive"], because in that case the prospect of exhaustively computing all possible transactions (to uncover the missing ones) is completely out of the question. - -However, message M4 is different from a withheld-txn, because M4 operates outside of the block's mandated information-processing limits (ie, outside the infamous 1 MB nonwitness blocksize limit). So we should examine the conditions under which M4 grows and shrinks, to ensure that we are not smuggling in a tremendous burden on full nodes. - -Under adversarial conditions, to lengthen a long-form M4 by one bit per block, for C blocks, the attacker must pay 312 bits (39 bytes) one time (to embed a new M3 message). The value C is the length of the sidechain's voting period, which varies but which I expect to be approximately 8,064 (and which could theoretically be as high as 65,536). Thus the attacker can burden nodes disproportionately, if (s)he wishes. - -Fortunately, the attack in question has no motivation (as far as I can tell). If the miner's goal is to trick rivals into mining on top of invalid blocks, he can already do this much more effectively with the unpublished-txn method (above). If instead he is just trying to harass nodes, then nodes may freely "downgrade" to earlier versions of the protocol, and simply ignore all drivechain-related messages. It seems that the attack could best be used in order to: make a large D2, make D2 confusing, sneak in votes for evil WT^ lurking in D2. Thus, the attack disables the transparency of the drivechain system, to some extent. The cost of the attack is forgone transaction fees, due to block space wasted on useless M3s. - -In practice, n is already capped, and miners may impose [on each other] a "soft cap" on m for their mutual protection. Thus, n and m might never get above 10 and 30, respectfully. In this case, the [Short Form, this time] M4 can never require more than 15 bytes per block, no matter what the attacker tries. - -In practice, m should always be 1 or 2, else something fishy is going on; and m can only inch up by 1 unit per block. So the system as a whole is still quite transparent, in that users are warned appropriately and well in advance. Attackers must invest upfront and they face an uphill climb, in order to eventually make things more expensive for a few others; defenders can wait-and-see if the attack looks like it will ever amount to anything before lifting a finger. - - -===== New Block Validation Rules (for D2 and, by implication, M4) ===== - -From one block to the next, D2 can only be edited in a few strict ways: - -* Entries can only be added/removed from D2 if they meet the criteria above (in M3, and implicitly M1 and M2). -* The ACK-counter of any individual entry can only change by (-1,0,+1) relative to its previous entry. -* Within a sidechain group, upvoting one withdrawal (ACK=ACK+1) requires you to downvote all other withdrawals in that group. However, the minimum ACK value is zero (and, therefore, downvotes cannot reduce it below zero). - -===== Footnotes for M4 ===== - -1 This represents the worst-case scenario is one where all the Withdrawals are spread evenly over each Sidechain. Under normal operations, there is no reason to expect the all sidechains will have the same number of withdrawals at any given time. In fact, under normal operations, the very *concept* of counting the withdrawals-per-sidechain should be a purposeless one, because there should only be *one* withdrawal at a time. Nonetheless we consider the worst case scenario here. - -2 Guessing becomes more computationally intensive in a highly adversarial situation where the "limited range" is intentionally expanded. In such a scenario, [a] there are many sidechains, and [b] miners voluntarily sacrifice their scarce block-space by creating a high number of (mutually-exclusive, and hence ultimately invalid) withdrawal attempts and putting these into coinbase transactions; and then agree to all [c] vote on these randomly (guaranteeing that all withdrawals fail, including any true withdrawals) and [d] successfully withhold their random voting strategies from nodes (even including spy-miner-nodes). Under this bizarre scenario, nodes may require computing resources which increase near-exponentially with the number of withdrawals, and it may take a long time for an ignorant node to exhaustively work out the underlying state of Withdrawal_DB. In this case, nodes may decide to temporarily stop validating such transactions (as if they had not yet upgraded to support this soft fork). - - - -=== Depositing and Withdrawing (M5, M6) === - - -Both M5 and M6 are regular Bitcoin txns. They are identified by meeting an important criteria: they select a one of the Critical TxID-index Pairs (a "CTIP") as one of their inputs. Deposits ("M5") are distinguished from withdrawals ("M6") by simply checking to see if money is "going in", or "out". In other words, we compare the BTC value of the original CTIP to that of new CTIP. If original <= new it is a deposit, if original > new then it is a withdrawal. - -The code that identifies sidechain withdrawal / deposit txns (by calculating how much value is being put into or taken out of a sidechain) can be seen here: https://github.com/drivechain-project/bitcoin/blob/mainchainBMM/src/validation.cpp#L351-L386 - -Such txns are forced (by consensus) to obey two additional criteria: - -# They must contain an output paying "to" the Critical Address [probably in TxOut0]. -# They must be accompanied by an update to this sidechain's Critical TxID-index Pair (CTIP). The new CTIP must be "this" txn itself. - -These criteria are enforced [https://github.com/drivechain-project/bitcoin/blob/mainchainBMM/src/validation.cpp#L440-L473 here] by checking that a deposit is paying back to the sidechain more than it is taking out, and completely rejecting any withdrawal from the mempool. And [https://github.com/drivechain-project/bitcoin/blob/mainchainBMM/src/validation.cpp#L1747-L1757 here] we allow for a withdrawal only once it has attained sufficient work score (ACKs). - -The purpose of this is to have all of the escrow's money (ie all of the sidechain's money) in one TxID, so that depositors immediately undo any UTXO bloat they may cause. This simplifies the withdrawal process, as there is no need to worry about cleaning up "dust deposits" (...and such cleaning can often result in headaches, for example where a withdrawal-txn is larger than 1MB in size, or else may only withdraw an arbitrarily limited amount of BTC). Notice that, unless we assume that an account will last forever, all utxos which are deposited must eventually be withdrawn by someone. Therefore, the relevant design criterion is not "efficiency" (total network cost) but rather "who should pay" (allocation of costs). - -==== M5. "Make a Deposit" -- a transfer of BTC from-main-to-side ==== - -As far as mainchain consensus is concerned, there are no additional requirements. - -However, in practice there *are* additional mainchain requirements...specified by the escrow account, (ie specified by the "sidechain" or "virtual chain"). These requirements are not part of mainchain consensus and are allowed to be anything. In other words, the sidechain is free to invent any way to credit depositor's money -- M5 is fully customizable. - -One method, is for mainchain depositors to append a zero-value OP Return to a Deposit txn, so that the sidechain knows how to credit funds. Mainchain users must upgrade their wallet software, of course, (on an individual basis) in order to become aware of and take advantage of new deposit-methods. - -===== Inconvenient Race Condition ===== - -The requirement that each hashrate escrow be linked to a single TxID does create an interesting inconvenience for depositors. If a user is slow to sign a txn after constructing it (perhaps because the user employs an air-gapped computer, etc), then the signed txn may no longer be valid. This is because the input it selects, may no longer be the Critical TxID (as "the" Critical TxID changes with each deposit). **Only one user can deposit at a time** (although many can deposit per block). As a result, the transaction must fail, and the user would need to be prompted to remake and resign the txn. If this is problem is too frustrating, users can always make main-to-side transfers using atomic cross chain swaps (or, the LN, if they already have a channel open on both chains). - -Fortunately, it is already a part of mainchain consensus that no two txns can spend the same TxID. The only new issue here is the confusion it might create for the user (hence the need for error messages and alternative deposit-methods). - - -==== M6. "Execute Withdrawal" -- a transfer of BTC from-side-to-main ==== - -We come, finally, to the critical matter: where users can take their money *out* of the escrow account, and return it to the "regular" UTXO set. As previously mentioned, this txn is one which (a) spends from a CTIP and (b) reduces the quantity of BTC in an account's CTIP. Most of the work has already been done by D1, M3, M4, and D2. Furthermore, existing Bitcoin tx-rules prevent the sidechain from ever withdrawing more money than has been placed into it. - -From there, we merely introduce two final concepts: - -# In each block, an entry in D2 is considered an "approved candidate" if the "ACKs" value is above 13140. -# A "blinded TxID" is way of hashing the txn, in which we first overwrite some parts of the txn with zeros. Specifically, the first 36 bytes of "TxIn0" (the first input, including TxOutHash and TxOutIndex), as well as the first 8 bytes of "TxOut0" (the first output). - -Blinding is necessary because we allow each sidechain only one UTXO at a time. - -of our restriction of the account to a single UTXO-member. Because of this, during the ACKing process the withdrawal-txn (which is currently being ACKed) may change in two ways: the CTIP (which changes with each deposit), and the total quantity of BTC stored in the account (which arbitrarily increases with each new deposit). In other words, a withdrawal-attempt is created via M3, but this takes place many blocks before the withdrawal is actually included via M6. During this time, a single new deposit to the account would change its CTIP and its value. So, what do we ACK? Well, we ACK a "blinded" version of the withdrawal. This blinded version is stable because the dynamic parts are always overwritten with zeros. - -While we ACK a blinded WT^, what is actually included in the blockchain ("M6") is an unblinded WT^. Since each blinded WT^ could correspond to many different unblinded WT^s, we need to impose further restrictions on those unblinded WT^s that are finally included. First, we will force the final unblinded WT^ to spend the entire sidechain balance (by forcing sum(input_values) to equal sum(output_values)). To avoid withdrawing the entire sidechain balance with every withdrawal, we will, secondly, force the unblinded WT^ to create a new output which is itself a deposit to the sidechain it withdrew from (which nodes can check using D1's CTIP field). Unfortunately, these requirements eliminate the possibility of including a transaction fee, as traditionally calculated. So, finally, to compensate for *that*, txn fees are encoded explicitly as a withdrawal to OP_TRUE (which the main:block's miner can immediately claim). - -With all of this in place, the only requirements for inclusion in a block are these: - -# "Be ACKed" -- The "blinded TxID" of this txn must be member of the "approved candidate" set in the D2 of this block. -# "Return Change to Account" -- TxOut0 must pay to the "critical account" (see D1) that corresponds to the CTIP that was selected as a TxIn. -# "Return *all* Change to Account" -- Sum of inputs must equal the sum of outputs. No traditional tx fee is possible. - -Finally, don't forget that M6 inherits the requirement (common to both M5 and M6) that the CTIP be selected as an input, and that the CTIP then be updated. In this case, we know that the critical index will be zero, so the new CTIP will be ("this TxID" (NOT blinded), 0). The TxID is NOT blinded because blinding is only for accumulating ACKs. - -As a result of these requirements, every single withdrawal-attempt will fail, unless an entry has been added to D2 and "ACKed" a sufficient number of times. - - - -==Backward compatibility== - - -As a soft fork, older software will continue to operate without modification. Non-upgraded nodes will see a number of phenomena that they don't understand -- coinbase txns with non-txn data, value accumulating in anyone-can-spend UTXOs for months at a time, and then random amounts leaving the UTXO in single, infrequent bursts. However, this phenomena doesn't affect them or the validity of the money that they receive. - -( As a nice bonus, note that the sidechains themselves inherit a resistance to hard forks. The only way to guarantee that the WT^s reported by different clients will continue to match identically, is to upgrade sidechains via soft forks of themselves. ) - - -==Deployment== - - -This BIP will be deployed by "version bits" BIP9 with the name "hrescrow" and using bit 4. - -
-// Deployment of Drivechains (BIPX, BIPY)
-consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].bit = 4;
-consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].nStartTime = 1515974401; // January 15th, 2018.
-consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].nTimeout = 1547510401; // January 15th, 2019.
-
- -==Reference Implementation== - - -See: https://github.com/drivechain-project/bitcoin/tree/mainchainBMM - -Also, for interest, see an example sidechain here: https://github.com/drivechain-project/bitcoin/tree/sidechainBMM - - -==References== - -See http://www.drivechain.info/literature/index.html - - -==Credits== - -Thanks to everyone who contributed to the discussion, especially: ZmnSCPxj, Adam Back, Peter Todd, Dan Anderson, Sergio Demian Lerner, Chris Stewart, Matt Corallo, Sjors Provoost, Tier Nolan, Erik Aronesty, Jason Dreyzehner, Joe Miyamoto, Ben Goldhaber. - - -==Copyright== - -This BIP is licensed under the BSD 2-clause license. diff --git a/bip-hashrate-escrows/images.txt b/bip-hashrate-escrows/images.txt deleted file mode 100644 index 2fbbf63..0000000 --- a/bip-hashrate-escrows/images.txt +++ /dev/null @@ -1 +0,0 @@ -Images used as reference in the documentation. diff --git a/bip-hashrate-escrows/two-groups.png b/bip-hashrate-escrows/two-groups.png deleted file mode 100644 index c8a3ffa..0000000 Binary files a/bip-hashrate-escrows/two-groups.png and /dev/null differ -- cgit v1.2.3 From 70f0ed6c39bec8d50ae29a2b2b562b44d84df5b7 Mon Sep 17 00:00:00 2001 From: Paul Sztorc Date: Sat, 10 Feb 2018 17:23:31 -0500 Subject: switch to mediawiki format --- bip-blind-merged-mining.md | 329 ------------------------------------- bip-blind-merged-mining.mediawiki | 330 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 330 insertions(+), 329 deletions(-) delete mode 100644 bip-blind-merged-mining.md create mode 100644 bip-blind-merged-mining.mediawiki diff --git a/bip-blind-merged-mining.md b/bip-blind-merged-mining.md deleted file mode 100644 index f203e40..0000000 --- a/bip-blind-merged-mining.md +++ /dev/null @@ -1,329 +0,0 @@ - Drivechain Documentation -- Blind Merged Mining BIP - Paul Sztorc - November 17, 2017 - Document 3 of 3 - v4.1 - - -Header -======= - - BIP: ???? - Layer: Consensus (soft fork) - Title: Blind Merged Mining (Consensus layer) - Author: Paul Sztorc - CryptAxe - Chris Stewart - Comments-Summary: No comments yet. - Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-??????? - Status: Draft - Type: Standards Track - Created: 2017-10-24 - License: BSD-2-Clause - - -Abstract -========== - -Blind Merged Mining (BMM) is a way of mining special extension blocks, ie "sidechains". It produces strong guarantees that the block is valid, for *any* arbitrary set of rules; and yet it does so without requiring miners to actually do any validation on the block whatsoever. - -BMM actually is a process that spans two or more chains. For an explanation of the "whole picture", please see [this post](http://www.truthcoin.info/blog/blind-merged-mining/). Here we focus on the modifications to mainchain Bitcoin. - -To support BMM, the mainchain is asked to accomplish two goals: -1. Track a set of ordered hashes (the merged-mining). -2. Allow miners to "sell" the act of finding a sidechain block (through the use of a new extended serialization transaction type). - -These goals are accomplished by forcing nodes to validate two new messages (M7, M8), and track data in one new database (D3). - - -Motivation -============ - -Regular "Merged-Mining" (MM) allows miners to reuse their hashing work to secure other chains (for example, as in Namecoin). However, traditional MM has two drawbacks: - -1. Miners must run a full node of the other chain. (This is because [while miners can effortlessly create the block] miners will not create a valid payment to themselves, unless the block that they MM is a valid one. Therefore, miners must assemble a *valid* block first, then MM it.) -2. Miners are paid on the other chain, not on the regular BTC mainchain. For example, miners who MM Namecoin will earn NMC (and they will need to sell the NMC for BTC, before selling the BTC in order to pay for electricity). - -Blind Merged-Mining (BMM) attempts to address those shortcomings. - - -Specification -============ - -Note: This document uses the notation side:\* and main:\* in front of otherwise-ambiguous words (such as "block", "node", or "chain"), to distinguish the mainchain version from its sidechain counterpart. - -As stated above, we have two goals: [1] create and monitor an alt-chain (defined only by a deterministic list of hashes), and [2] allow miners to "sell" the act of finding a sidechain block (through the use of a new extended serialization transaction type). - -### Sidechain Critical Data ("Sidechain Mini-Header") - -Specifically, per side:block per side:chain, we track the following 35 bytes of information: - - 1-byte - ChainIndex (known as "Account Number" in hashrate-escrows.md , or as "Sidechain Number") - 32-bytes - sideHeaderHash (also known as "h*" / hashCritical, the hash of the sidechain block) - 2-bytes - prevBlockRef (an index which points to this side:block's parent side:block) - -The **ChainIndex** indicates which sidechain this critical data is relevant to. As we may eventually have more than one sidechain, this serves as an identifier similar to the Bitcoin network's magic bytes (0xF9BEB4D9). Drivechains however only need to use 1 byte for the identifier (there is a hard limit of 256 sidechains identified as 0-255). The **sideHeaderHash** is the hash of a side:block which will receive PoW via BMM. It serves a similar function to Bitcoin's "hashMerkleRoot", in that it contains the data for its blockchain. The **prevBlockRef** forms the set of headers into a blockchain structure by making each headers refer to one parent header. It is most similar to Bitcoin's hashPrevBlock. - -Where does this data come from, and how does it get around? - -#### Creating / Broadcasting This Data - -##### Creation - -By the time Blind Merged Mining can take place, the ChainIndex is globally known (it is the "Account Number" in D1 [see previous BIP], and "nSidechain" in the code). Each sidechain, when activated by soft fork, will take one of the 0-255 available indexes. - -The other two items, sideHeaderHash and prevBlockRef, are created by sidechain nodes. sideHeaderHash is quite straightforward -- side:nodes build side:blocks, and take the hash of these. - -The final item, prevBlockRef, is a little more complicated. It is an integer that counts the number of "skips" one must take in the side:chain in order to find the current side:block's parent block. In practice, this value will usually be zero. It will only be a value other than zero, in cases where invalid sidechain blocks have been mined, or when a side:node intentionally wants to orphan some side:blocks (if a side:node wants to orphan the most-recent N blocks, the value of the current block will be equal to N ; in the block after that it will be back to zero). - -![dots-image](/bip-blind-merged-mining/bmm-dots-examples.png?raw=true) - -Since the hashes themselves are already ordered by the mainchain, tracing the blockchain's path by index (prevBlockRef) will be the same as tracing it by identifying a list of hashes. In other words, the ordering given via each side:block's "prevBlockRef" will be isomorphic to an ordering given by each side:block's "prevSideHeaderHash" ... if "prevSideHeaderHash is defined to be the sidechain's equivalent of the mainchain's "prevBlockHash". It will be possible to freely convert from one to the other. See M8 to learn more about how these hashes are requested by sidechain block creators to be included in the mainchain. - -Now that we know what our critical data is, and how it is made, how is this data broadcast and stored? - -##### Broadcast - -Mainchain nodes are going to need this data later, so it must be easy to find. We will put it into the coinbase via OP RETURN. - -#### M7 -- "Blind-Mine the Sidechain(s)" - -Thus, (for n sidechains) we have a coinbase output with: - - 1-byte - OP_RETURN (0x6a) - 1-byte - Push the following (4+(n*35)) bytes (0x??) - 4-byte - Message header (0xD3407053) - (n*(32+5))-byte - A sequence of bytes, of the three Mini-Header items (h*, prevBlockRef, ChainIndex). - -( We assume that 5 bytes are used for the Critical Data bytes (non h* parts of the Sidechain Mini-Header). For 256 sidechains, a total of 9,478 bytes [1+1+4+256\*(32+5)] are required, conveniently just below the 10 KB scriptPubKey size limit.) - -This data is parsed by laying it in sequential 37-byte chunks (any remaining data --ie, some final chunk that is less than 37 bytes in length-- has no consensus meaning). - -Each 37-byte chunk is then parsed to obtain the data outlined above (in "Description"). If two 35-byte chunks being with the same "Sidechain number" (ie, if the two chunks have the same first byte), then only the first chunk has consensus meaning. - -We are left with, at most, one (h*, prevBlockRef) pair per sidechain per block. This data is added directly to D3, a new database. - -#### D3 -- "RecentSidechains_DB" - -To suit our purposes, the mainchain full nodes will need to keep track of the most recent 8000 (h\*, prevBlockRef) pairs. - -( This 8,000 figure is a tradeoff between decentralization (costs of running the main:node) and sidechain security -- it requires attackers to merged-mine 8,000 invalid blocks consecutively, in order to cause the sidechain to fail. The mainchain burden is minimal, so this figure might be raised to 12,000 or higher. ) - -Therefore, D3 would look something like this: - - - BlockHeight CB_Index SC_1 Blks_Atop_1 SC 2 Blks_Atop_2 SC 3 Blks_Atop_3 - --------- ------ ------ --------- ------ --------- ------ --------- - 1. 401,005 2 (h*, 0) 7985 (h*, 0) 1 (h*, 0) 0 - 2. 401,006 4 (h*, 0) 7984 (h*, 0) 0 (h*, 1) 7801 - 3. 401,007 2 (h*, 0) 7983 (h*, 5) 2027 (h*, 0) 0 - 4. 401,008 2 (h*, 0) 7982 (h*, 0) 2028 (h*, 1) 7800 - ... ... ) - 7999. 409,003 3 (h*, 0) 1 (h*, 0) 0 (h*, 0) 1 - 8000. 409,004 2 (h*, 0) 0 (h*, 1) 0 (h*, 0) 0 - - -When new sidechains (or "hashrate escrows") are soft-forked into existence, a new column is added to D3 (to contain any BMMing that might be done on it). - -For each sidechain we also track the field "Blocks Atop". This is the number of side:blocks that are "on top" of the specified side:block. These might be regarded as "side:confirmations" (pseudo-confirmations that are specific to each sidechain). - -D3 also contains a column (not shown) for each sidechain containing "prevSideBlockHash". This value is is either derived from prevBlockRef; or else it is given explicitly (in which case it is the converse: prevBlockRef is derived from prevSideBlockHash). - - -#### Coinbase Cache - -As mentioned above, M7s cause data to be added to D3. Recent D3 data is tracked by all mainchain nodes for a period of time. - -To efficiently keep track of the above data, without having to constantly load and process entire blocks from disk, we temporarily cache enough coinbases in the chain index to maintain D3. - - -### M8 -- Paying miners to include BMM data in their coinbase outputs - -This section introduces a new type of transaction, which allows sidechain block creators to request, and pay for, a critical hash to be included in a specific block by mainchain miners. See [the Blind Merged Mining spec](http://www.truthcoin.info/blog/blind-merged-mining/). This txn allows miners to "sell" the act of mining a sidechain block. By taking advantage of this option, miners earn tx fees for mining sidechains...truly "for free". They do not even need to run sidechain nodes, and the tx-fees they earn are in mainchain BTC. As a result, sidechains affect all miners equally and do not affect the mining ecosystem. - -M8 will ultimately come in two versions. The second version will be specialized for use in the Lightning Network and must use the full 32-byte prevBlockHash (ironically, this larger transaction is cheaper for the Bitcoin network to process, as it is completely off-chain). The first version of M8, in contrast, cannot be used inside the Lightning Network, but is slightly more space-efficient (using the 2 prevBlockRef bytes to maintain chain order). It is important to make both options available to the user, because some side:nodes may be unwilling or unable to open a payment channels with each main:miner. However, in the long run we expect the lightning version to be preferred. - -#### Setup - -We define **"Mary"** as a mainchain miner, and **"Simon"** as a sidechain node. - -The goal is to construct a payment from Simon to Mary, such that: - -1. If the critical data conditions are met, **Mary** can claim the outputs of the transaction with finality. -2. If the critical data conditions are not met, the outputs become immediately available again to **Simon**. - - -#### Goals (this is rather philosophical, and skippable) - -##### Immediate Expiration ("Fill-or-Kill") - -We would like to make special guarantees to the counterparties of this transaction. Specifically, instead of Simon making a "payment" to Mary, we prefer that Simon give Mary an "offer" (which she can either accept or decline). - -Crucially, we want Simon to safely make many offers to several different Mary's, in realtime (ie, quickly and off-chain). However, we ultimately want only one offer to be accepted, at most. In other words, we want Simon's offers to *immediately expire*. If only one offer can become a bona fide transaction, then Simon will feel comfortable making offers all day long. Because all of the Simons are making many offers, the Marys collectively gain access to a large set of offers to choose from. - -##### Forward Progress (The Need for a "Ratchet") - -The "ratchet" concept is an attempt to harmonize incentives among the main and side chain(s). -We need to ensure that a sidechain is making "forward progress", without tracking too much about the sidechain such that we burden Bitcoin (see [1] and [2]) all while still allowing the sidechain to reorganize [3]. - -* [1] https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-July/014789.html -* [2] http://www.drivechain.info/faq/index.html#what-is-the-difference-between-drivechain-and-extension-blocks -* [3] http://www.truthcoin.info/blog/blind-merged-mining/#handling-reorganizations - -The ratchet system must keep track of sidechain "mini-headers" (see Sidechain Critical Data ("Sidechain Mini-Header")) and count the "blocks atop" maturity of the related side:blocks. - -Simon's offer to Mary to include a critical hash in exchange for payment must be *atomic*. The "ratchet" concept helps to construct a very tight connection between two things: - -1. The sidechain-block-generator "Simon" paying himself the side:block's side:tx-fees (which he receives in 100 sidechain blocks (blocks atop) hence). -2. "Simon" making a mainchain main:btc payment to a mainchain miner "Mary". - -Either both of the two should succeed, or else both should jointly fail. - -However, absent our intervention, there are cases in which [2, the payment to Mary] succeeds but [1, side:tx-fees] fails. One such case is when a side:block contains unusually high side:tx-fees. Here, there will be many requests to include a critical hash in exchange for payment submitted to Mary, but only one can be included in each main:block per sidechain. Without an incentive to make "forward progress", Mary is likely to include one of the highest paying requests in the next main:block (and the main:block after that, and so on). Mary will "blindly" include high-paying requests for *older* blocks, unless something prevents her from doing so. - -To address these potential issues, we utilize the concept of "Blocks_Atop" (the "side:confirmations") that we mentioned earlier. As previously mentioned, Mary will not be able to spend Simon's M8 payment until satisfying the critical data requirements as well as the blocks atop (side:confirmations) requirement. - - -#### M8 -- The two forms of M8 transactions - -As previously mentioned, M8 can take two forms. The first does not require the Lightning Network, but it does have new requirements for Immediate Expiration (see above). The second inherits Immediate Expiration from the Lightning Network itself, but requires extra preparation and a different/larger message. - -Both forms require that certain Critical Data will be committed to within the coinbase of the block that the transaction is included in. For the non Lightning version, we have created a new extended serialization transaction type (very similar to how segwit handles witness data (the witness stack)). - -##### M8_V1 - No Lightning Network - -M8_V1 does not require the Lightning network but does have new requirements for validation. - -A M8_V1 TxOut is expected to contain: - - 1-byte - OP_RETURN (0x6a) - 1-byte - Push the following 36 bytes (0x24) - 4-byte - Message header (0xD1617368) - 32-bytes - h* side:block hash - 5~7-bytes - BMM request identifying bytes (0x00bf00) + prevBlockRef & ChainIndex (sidechain mini-header) - - -In the first version of M8, we need to introduce the concept of Immediate Expiration (see above). In other words, we need a way for Simon to construct many payments to multiple Marys, such that only one of these is ever included; and only then if Simon's txn is expected to coincide with the finding of Simon's side:block. - -We do this by imposing validity-rules on the txn itself: - -1. The txn's content, when examined, must match part of the main:block's content. Specifically, the (ChainIndex, h\*) pair of the txn, must match one of the (ChainIndex, h\*) pairs in the M7 of this main:block. -2. Only one payment per sidechain per main:block is valid. In other words, if 400 people all try to bm-mine the sidechain with ChainIndex==4, then not only is it the case that only one side_4:block can be found, but it is also the case that only the corresponding M8 txn can be included (out of all of the 400 M8s which are for ChainIndex==4). -3. Simon's txns must only be valid for the current block; afterward, they immediately expire. This is because Simon's intended prevBlockRef & side:block contents will most likely change from one main:block to the next. - -To impose new requirements on the transaction level (not the block level nor the TxOutput level), we borrow the "flag" trick from SegWit style transactions. If the flag is present, the transaction is examined for extra data, and if this data does not pass certain requirements, the transaction is invalid. With SegWit, this extra data is the signatures, and the extra requirements are the signatures' locations and validity. In the BMM-transactions, the extra data is the (ChainIndex, h\*) pair, which must meet the first two requirements (above) as well as the main:blocknumber, which must meet the third requirement (above). - -To impose new requirements at the transaction level, we borrow the dummy vin & "flag" trick from SegWit style transactions. If the flag is set to 2 (0010), the transaction contains Critical Data and requires that our new validation rules be met in order for the txn to be valid in a block. Unless all of the requirements for sidechain critical data transactions are met by the block it is included in, the transaction is invalid. With SegWit, this extra data is the segwit signature stack, and the extra requirements are the signatures' locations and validity. In the sidechain BMM critical data transactions, the extra data is the (ChainIndex, h\*) pair, which must meet the first two requirements (above) as well as the main:blocknumber, which must meet the third requirement (above). Note The main:blocknumber does not take up any additional space compared to a normal txn, as we reuse the locktime field for our purposes. - - - - -![extra-data-image](/bip-blind-merged-mining/witness-vs-critical.png?raw=true) - -This txn structure conserves main:blockspace, because it is the easiest way to refer to a previous sidechain block in 4 bytes, (prevBlockRef + FoK_nLockTime). Instead, we would need to use at least 32 bytes (prevSideBlockHash). - -These types of transactions have slightly different mempool behavior, and should probably be kept in a second mempool. These txns are received, checked immediately, and if valid they are evaluated for inclusion in a block. If they are not able to be included in the specific requested block (if the block height requested has been surpassed by the chain tip), they are discarded. In fact, after any main:block is found, everything in this "second mempool" can be discarded as new payments will be created immediately for the next block height. (This includes cases where the blockchain reorganizes.) There is no re-evaluation of the txns in this mempool ever -- they are evaluated once and then either included or discarded. To be clear, when the transaction is received we are able to evaluate its validity, and do not need to rescan these transactions again. - -Interestingly, these payments (M8) will *always* be directed to miners from non-miners. Therefore, non-mining nodes do not need to keep them in any mempool at all. Non-miner nodes can just wait for a block to be found, and check the txn then. These transactions more resemble a stock market's pit trades (in contrast, regular Bitcoin txns remind me more of paper checks). - -##### M8_V2 With Lightning - -M8_V2 requires having a LN-channel open with a miner. This may not always be practical (or even possible), especially today. - -A M8_V1 TxOut is expected to contain: - - 1-byte - OP_RETURN (0x6a) - 1-byte - Push the following 68 bytes (0x44) - 4-byte - Message header (0xD0520C6E) - 32-bytes - h* side:block hash - 32-bytes - prevSideBlockHash - 5~7-bytes - BMM request identifying bytes (0x00bf00) + prevBlockRef & ChainIndex (sidechain mini-header) - - -Notice that, in M8_V1, Simon could reuse the same h\* all he wanted, because only one M8_V1 could be included per main:block per sidechain. However, on the LN no such rule can be enforced, as the goal is to push everything off-chain and include *zero* M8s. So, we will never know what the M8s were or how many had an effect on anything. - -Therefore, Simon will need to ensure that he **gives each Mary a different h\***. Simon can easily do this, as he controls the side:block's contents and can simply increment a nonce -- this changes the side:block, and changes its hash (ie, changes h\*). - -With a unique h\* per Mary, and at most 1 h\* making it into a block (per sidechain), we can guarantee that only one of the M8_V2's critical data can be committed to in a single main:block. By giving each miner (who Simon has a payment channel open with) a different h*, Simon can figure out which miner followed through with the commit, and know that only one such commit went through. Furthermore, if this Simon's requested critical data is not found in a block, none of the M8_V2 payments will be spendable by the Mary(s), because none of the h\* in question have ever made it into D3 (which is always on-chain) and no blocks atop will be accumulated. - -That's probably confusing, so here is an example, in which: Simon starts with 13 BTC, Mary starts with 40 BTC, the side:block's tx-fees currently total 7.1 BTC, and Simon is keeping 0.1 BTC for himself and paying 7 BTC to Mary. - -We start with (I): - - Simon 13 in, Mary 40 in ; 53 in total - Simon's version [signed by Mary] - 13 ; to Simon if TimeLock=over; OR to Mary if SimonSig - 40 ; to Mary - Mary's version [signed by Simon] - 40 ; to me if TimeLock=over; OR to Simon if MarySig - 13 ; to Simon - - -And both parties move, from there to "M8_V2" (II): - - Simon 13 in, Mary 40 in ; 53 in total - Simon's version [signed by Mary] - 6 ; to Simon if TimeLock=over; OR to Mary if SimonSig - 40 ; to Mary - 7 ; to Mary if critical data requirements met; OR to Simon if LongTimeLock=over - Mary's version [signed by Simon] - 40 ; to Mary if TimeLock=over; OR to Simon if MarySig - 6 ; to Simon - 7 ; to Mary if critical data requirements met; OR to Simon if LongTimeLock=over - -From here, if the h\* side:block in question is BMMed, they can proceed to (III): - - Simon 13 in, Mary 40 in ; 53 in total - Simon's version [signed by Mary] - 6 ; to Simon if TimeLock=over; OR to Mary if SimonSig - 47 ; to Mary - Mary's version [signed by Simon] - 47 ; to me if TimeLock=over; OR to Simon if MarySig - 6 ; to Simon - -Although, if Simon proceeds immediately, he removes the protection of the 'ratchet'. Ie, Simon removes Mary's incentive to care about blocks being built on this side:block. If Simon's side:block is orphaned, he loses his 7 BTC. Simon can either play it safe, and wait the full 100 side:blocks before moving on (ie, moving on to the third LN txn, above); or else Simon can take the risk if he feels comfortable with it. - -If the h\* side:block is not found, then (II) and (III) are basically equivalent to each other. Simon and Mary could jointly reconstruct (I) and go back there, or they could proceed to a new version of II (with a different h\*, trying again with new side:block in the next main:block). - - - - -Deployment -=========== - -This BIP will be deployed by "version bits" BIP9 with the name "blindmm" and using bit 4. - -``` -// Deployment of Drivechains (BIPX, BIPY) -consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].bit = 4; -consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].nStartTime = 1515974401; // January 15th, 2018. -consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].nTimeout = 1547510401; // January 15th, 2019. -``` - -Reference Implementation -========================== - -See: https://github.com/drivechain-project/bitcoin/tree/mainchainBMM - -Also, for interest, see an example sidechain here: https://github.com/drivechain-project/bitcoin/tree/sidechainBMM - - -References -============ - -* http://www.drivechain.info/literature/index.html -* http://www.truthcoin.info/blog/blind-merged-mining/ -* https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-July/014789.html -* http://www.truthcoin.info/images/bmm-outline.txt - - -Thanks -========= - -Thanks to everyone who contributed to the discussion, especially: ZmnSCPxj, Adam Back, Peter Todd, Dan Anderson, Sergio Demian Lerner, Matt Corallo, Sjors Provoost, Tier Nolan, Erik Aronesty, Jason Dreyzehner, Joe Miyamoto, Ben Goldhaber. - - -Copyright -========== - -This BIP is licensed under the BSD 2-clause license. diff --git a/bip-blind-merged-mining.mediawiki b/bip-blind-merged-mining.mediawiki new file mode 100644 index 0000000..3aaf81b --- /dev/null +++ b/bip-blind-merged-mining.mediawiki @@ -0,0 +1,330 @@ + +
+
+    BIP: ????
+    Layer: Consensus (soft fork)
+    Title: Blind Merged Mining (Consensus layer)
+    Author: Paul Sztorc 
+            CryptAxe 
+            Chris Stewart 
+    Comments-Summary: No comments yet.
+    Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-???????
+    Status: Draft
+    Type: Standards Track
+    Created: 2017-10-24
+    License: BSD-2-Clause
+
+
+ + +==Abstract== + + +Blind Merged Mining (BMM) is a way of mining special extension blocks, ie "sidechains". It produces strong guarantees that the block is valid, for *any* arbitrary set of rules; and yet it does so without requiring miners to actually do any validation on the block whatsoever. + +BMM actually is a process that spans two or more chains. For an explanation of the "whole picture", please see [this post](http://www.truthcoin.info/blog/blind-merged-mining/). Here we focus on the modifications to mainchain Bitcoin. + +To support BMM, the mainchain is asked to accomplish two goals: + +# Track a set of ordered hashes (the merged-mining). +# Allow miners to "sell" the act of finding a sidechain block (through the use of a new extended serialization transaction type). + +These goals are accomplished by forcing nodes to validate two new messages (M7, M8), and track data in one new database (D3). + + +==Motivation== + + +Regular "Merged-Mining" (MM) allows miners to reuse their hashing work to secure other chains (for example, as in Namecoin). However, traditional MM has two drawbacks: + +# Miners must run a full node of the other chain. (This is because [while miners can effortlessly create the block] miners will not create a valid payment to themselves, unless the block that they MM is a valid one. Therefore, miners must assemble a *valid* block first, then MM it.) +# Miners are paid on the other chain, not on the regular BTC mainchain. For example, miners who MM Namecoin will earn NMC (and they will need to sell the NMC for BTC, before selling the BTC in order to pay for electricity). + +Blind Merged-Mining (BMM) attempts to address those shortcomings. + + +==Specification== + + +Note: This document uses the notation side:\* and main:\* in front of otherwise-ambiguous words (such as "block", "node", or "chain"), to distinguish the mainchain version from its sidechain counterpart. + +As stated above, we have two goals: [1] create and monitor an alt-chain (defined only by a deterministic list of hashes), and [2] allow miners to "sell" the act of finding a sidechain block (through the use of a new extended serialization transaction type). + +=== Sidechain Critical Data ("Sidechain Mini-Header") === + +Specifically, per side:block per side:chain, we track the following 35 bytes of information: + + 1-byte - ChainIndex (known as "Account Number" in hashrate-escrows.md , or as "Sidechain Number") + 32-bytes - sideHeaderHash (also known as "h*" / hashCritical, the hash of the sidechain block) + 2-bytes - prevBlockRef (an index which points to this side:block's parent side:block) + +The **ChainIndex** indicates which sidechain this critical data is relevant to. As we may eventually have more than one sidechain, this serves as an identifier similar to the Bitcoin network's magic bytes (0xF9BEB4D9). Drivechains however only need to use 1 byte for the identifier (there is a hard limit of 256 sidechains identified as 0-255). The **sideHeaderHash** is the hash of a side:block which will receive PoW via BMM. It serves a similar function to Bitcoin's "hashMerkleRoot", in that it contains the data for its blockchain. The **prevBlockRef** forms the set of headers into a blockchain structure by making each headers refer to one parent header. It is most similar to Bitcoin's hashPrevBlock. + +Where does this data come from, and how does it get around? + +==== Creating / Broadcasting This Data ==== + +===== Creation ===== + +By the time Blind Merged Mining can take place, the ChainIndex is globally known (it is the "Account Number" in D1 [see previous BIP], and "nSidechain" in the code). Each sidechain, when activated by soft fork, will take one of the 0-255 available indexes. + +The other two items, sideHeaderHash and prevBlockRef, are created by sidechain nodes. sideHeaderHash is quite straightforward -- side:nodes build side:blocks, and take the hash of these. + +The final item, prevBlockRef, is a little more complicated. It is an integer that counts the number of "skips" one must take in the side:chain in order to find the current side:block's parent block. In practice, this value will usually be zero. It will only be a value other than zero, in cases where invalid sidechain blocks have been mined, or when a side:node intentionally wants to orphan some side:blocks (if a side:node wants to orphan the most-recent N blocks, the value of the current block will be equal to N ; in the block after that it will be back to zero). + + + +Since the hashes themselves are already ordered by the mainchain, tracing the blockchain's path by index (prevBlockRef) will be the same as tracing it by identifying a list of hashes. In other words, the ordering given via each side:block's "prevBlockRef" will be isomorphic to an ordering given by each side:block's "prevSideHeaderHash" ... if "prevSideHeaderHash is defined to be the sidechain's equivalent of the mainchain's "prevBlockHash". It will be possible to freely convert from one to the other. See M8 to learn more about how these hashes are requested by sidechain block creators to be included in the mainchain. + +Now that we know what our critical data is, and how it is made, how is this data broadcast and stored? + +===== Broadcast ===== + +Mainchain nodes are going to need this data later, so it must be easy to find. We will put it into the coinbase via OP RETURN. + +==== M7 -- "Blind-Mine the Sidechain(s)" ==== + +Thus, (for n sidechains) we have a coinbase output with: + + 1-byte - OP_RETURN (0x6a) + 1-byte - Push the following (4+(n*35)) bytes (0x??) + 4-byte - Message header (0xD3407053) + (n*(32+5))-byte - A sequence of bytes, of the three Mini-Header items (h*, prevBlockRef, ChainIndex). + +( We assume that 5 bytes are used for the Critical Data bytes (non h* parts of the Sidechain Mini-Header). For 256 sidechains, a total of 9,478 bytes [1+1+4+256\*(32+5)] are required, conveniently just below the 10 KB scriptPubKey size limit.) + +This data is parsed by laying it in sequential 37-byte chunks (any remaining data --ie, some final chunk that is less than 37 bytes in length-- has no consensus meaning). + +Each 37-byte chunk is then parsed to obtain the data outlined above (in "Description"). If two 35-byte chunks being with the same "Sidechain number" (ie, if the two chunks have the same first byte), then only the first chunk has consensus meaning. + +We are left with, at most, one (h*, prevBlockRef) pair per sidechain per block. This data is added directly to D3, a new database. + +==== D3 -- "RecentSidechains_DB" ==== + +To suit our purposes, the mainchain full nodes will need to keep track of the most recent 8000 (h\*, prevBlockRef) pairs. + +( This 8,000 figure is a tradeoff between decentralization (costs of running the main:node) and sidechain security -- it requires attackers to merged-mine 8,000 invalid blocks consecutively, in order to cause the sidechain to fail. The mainchain burden is minimal, so this figure might be raised to 12,000 or higher. ) + +Therefore, D3 would look something like this: + + +
+           BlockHeight  CB_Index    SC_1   Blks_Atop_1   SC 2   Blks_Atop_2   SC 3   Blks_Atop_3
+            ---------    ------    ------   ---------   ------   ---------   ------   ---------
+       1.    401,005        2      (h*, 0)     7985     (h*, 0)        1     (h*, 0)        0
+       2.    401,006        4      (h*, 0)     7984     (h*, 0)        0     (h*, 1)     7801
+       3.    401,007        2      (h*, 0)     7983     (h*, 5)     2027     (h*, 0)        0
+       4.    401,008        2      (h*, 0)     7982     (h*, 0)     2028     (h*, 1)     7800
+      ...     ...                                                                  )
+    7999.    409,003        3      (h*, 0)        1     (h*, 0)        0     (h*, 0)        1
+    8000.    409,004        2      (h*, 0)        0     (h*, 1)        0     (h*, 0)        0
+
+ + +When new sidechains (or "hashrate escrows") are soft-forked into existence, a new column is added to D3 (to contain any BMMing that might be done on it). + +For each sidechain we also track the field "Blocks Atop". This is the number of side:blocks that are "on top" of the specified side:block. These might be regarded as "side:confirmations" (pseudo-confirmations that are specific to each sidechain). + +D3 also contains a column (not shown) for each sidechain containing "prevSideBlockHash". This value is is either derived from prevBlockRef; or else it is given explicitly (in which case it is the converse: prevBlockRef is derived from prevSideBlockHash). + + +==== Coinbase Cache ==== + +As mentioned above, M7s cause data to be added to D3. Recent D3 data is tracked by all mainchain nodes for a period of time. + +To efficiently keep track of the above data, without having to constantly load and process entire blocks from disk, we temporarily cache enough coinbases in the chain index to maintain D3. + + +=== M8 -- Paying miners to include BMM data in their coinbase outputs === + +This section introduces a new type of transaction, which allows sidechain block creators to request, and pay for, a critical hash to be included in a specific block by mainchain miners. See [the Blind Merged Mining spec](http://www.truthcoin.info/blog/blind-merged-mining/). This txn allows miners to "sell" the act of mining a sidechain block. By taking advantage of this option, miners earn tx fees for mining sidechains...truly "for free". They do not even need to run sidechain nodes, and the tx-fees they earn are in mainchain BTC. As a result, sidechains affect all miners equally and do not affect the mining ecosystem. + +M8 will ultimately come in two versions. The second version will be specialized for use in the Lightning Network and must use the full 32-byte prevBlockHash (ironically, this larger transaction is cheaper for the Bitcoin network to process, as it is completely off-chain). The first version of M8, in contrast, cannot be used inside the Lightning Network, but is slightly more space-efficient (using the 2 prevBlockRef bytes to maintain chain order). It is important to make both options available to the user, because some side:nodes may be unwilling or unable to open a payment channels with each main:miner. However, in the long run we expect the lightning version to be preferred. + +==== Setup ==== + +We define **"Mary"** as a mainchain miner, and **"Simon"** as a sidechain node. + +The goal is to construct a payment from Simon to Mary, such that: + +# If the critical data conditions are met, **Mary** can claim the outputs of the transaction with finality. +# If the critical data conditions are not met, the outputs become immediately available again to **Simon**. + + +==== Goals (this is rather philosophical, and skippable) ==== + +===== Immediate Expiration ("Fill-or-Kill") ===== + +We would like to make special guarantees to the counterparties of this transaction. Specifically, instead of Simon making a "payment" to Mary, we prefer that Simon give Mary an "offer" (which she can either accept or decline). + +Crucially, we want Simon to safely make many offers to several different Mary's, in realtime (ie, quickly and off-chain). However, we ultimately want only one offer to be accepted, at most. In other words, we want Simon's offers to *immediately expire*. If only one offer can become a bona fide transaction, then Simon will feel comfortable making offers all day long. Because all of the Simons are making many offers, the Marys collectively gain access to a large set of offers to choose from. + +===== Forward Progress (The Need for a "Ratchet") ===== + +The "ratchet" concept is an attempt to harmonize incentives among the main and side chain(s). +We need to ensure that a sidechain is making "forward progress", without tracking too much about the sidechain such that we burden Bitcoin (see [1] and [2]) all while still allowing the sidechain to reorganize [3]. + +* [1] https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-July/014789.html +* [2] http://www.drivechain.info/faq/index.html#what-is-the-difference-between-drivechain-and-extension-blocks +* [3] http://www.truthcoin.info/blog/blind-merged-mining/#handling-reorganizations + +The ratchet system must keep track of sidechain "mini-headers" (see Sidechain Critical Data ("Sidechain Mini-Header")) and count the "blocks atop" maturity of the related side:blocks. + +Simon's offer to Mary to include a critical hash in exchange for payment must be *atomic*. The "ratchet" concept helps to construct a very tight connection between two things: + +# The sidechain-block-generator "Simon" paying himself the side:block's side:tx-fees (which he receives in 100 sidechain blocks (blocks atop) hence). +# "Simon" making a mainchain main:btc payment to a mainchain miner "Mary". + +Either both of the two should succeed, or else both should jointly fail. + +However, absent our intervention, there are cases in which [2, the payment to Mary] succeeds but [1, side:tx-fees] fails. One such case is when a side:block contains unusually high side:tx-fees. Here, there will be many requests to include a critical hash in exchange for payment submitted to Mary, but only one can be included in each main:block per sidechain. Without an incentive to make "forward progress", Mary is likely to include one of the highest paying requests in the next main:block (and the main:block after that, and so on). Mary will "blindly" include high-paying requests for *older* blocks, unless something prevents her from doing so. + +To address these potential issues, we utilize the concept of "Blocks_Atop" (the "side:confirmations") that we mentioned earlier. As previously mentioned, Mary will not be able to spend Simon's M8 payment until satisfying the critical data requirements as well as the blocks atop (side:confirmations) requirement. + + +==== M8 -- The two forms of M8 transactions ==== + +As previously mentioned, M8 can take two forms. The first does not require the Lightning Network, but it does have new requirements for Immediate Expiration (see above). The second inherits Immediate Expiration from the Lightning Network itself, but requires extra preparation and a different/larger message. + +Both forms require that certain Critical Data will be committed to within the coinbase of the block that the transaction is included in. For the non Lightning version, we have created a new extended serialization transaction type (very similar to how segwit handles witness data (the witness stack)). + +===== M8_V1 - No Lightning Network ===== + +M8_V1 does not require the Lightning network but does have new requirements for validation. + +A M8_V1 TxOut is expected to contain: + + 1-byte - OP_RETURN (0x6a) + 1-byte - Push the following 36 bytes (0x24) + 4-byte - Message header (0xD1617368) + 32-bytes - h* side:block hash + 5~7-bytes - BMM request identifying bytes (0x00bf00) + prevBlockRef & ChainIndex (sidechain mini-header) + + +In the first version of M8, we need to introduce the concept of Immediate Expiration (see above). In other words, we need a way for Simon to construct many payments to multiple Marys, such that only one of these is ever included; and only then if Simon's txn is expected to coincide with the finding of Simon's side:block. + +We do this by imposing validity-rules on the txn itself: + +# The txn's content, when examined, must match part of the main:block's content. Specifically, the (ChainIndex, h\*) pair of the txn, must match one of the (ChainIndex, h\*) pairs in the M7 of this main:block. +# Only one payment per sidechain per main:block is valid. In other words, if 400 people all try to bm-mine the sidechain with ChainIndex==4, then not only is it the case that only one side_4:block can be found, but it is also the case that only the corresponding M8 txn can be included (out of all of the 400 M8s which are for ChainIndex==4). +# Simon's txns must only be valid for the current block; afterward, they immediately expire. This is because Simon's intended prevBlockRef & side:block contents will most likely change from one main:block to the next. + +To impose new requirements on the transaction level (not the block level nor the TxOutput level), we borrow the "flag" trick from SegWit style transactions. If the flag is present, the transaction is examined for extra data, and if this data does not pass certain requirements, the transaction is invalid. With SegWit, this extra data is the signatures, and the extra requirements are the signatures' locations and validity. In the BMM-transactions, the extra data is the (ChainIndex, h\*) pair, which must meet the first two requirements (above) as well as the main:blocknumber, which must meet the third requirement (above). + +To impose new requirements at the transaction level, we borrow the dummy vin & "flag" trick from SegWit style transactions. If the flag is set to 2 (0010), the transaction contains Critical Data and requires that our new validation rules be met in order for the txn to be valid in a block. Unless all of the requirements for sidechain critical data transactions are met by the block it is included in, the transaction is invalid. With SegWit, this extra data is the segwit signature stack, and the extra requirements are the signatures' locations and validity. In the sidechain BMM critical data transactions, the extra data is the (ChainIndex, h\*) pair, which must meet the first two requirements (above) as well as the main:blocknumber, which must meet the third requirement (above). Note The main:blocknumber does not take up any additional space compared to a normal txn, as we reuse the locktime field for our purposes. + + + + + +This txn structure conserves main:blockspace, because it is the easiest way to refer to a previous sidechain block in 4 bytes, (prevBlockRef + FoK_nLockTime). Instead, we would need to use at least 32 bytes (prevSideBlockHash). + +These types of transactions have slightly different mempool behavior, and should probably be kept in a second mempool. These txns are received, checked immediately, and if valid they are evaluated for inclusion in a block. If they are not able to be included in the specific requested block (if the block height requested has been surpassed by the chain tip), they are discarded. In fact, after any main:block is found, everything in this "second mempool" can be discarded as new payments will be created immediately for the next block height. (This includes cases where the blockchain reorganizes.) There is no re-evaluation of the txns in this mempool ever -- they are evaluated once and then either included or discarded. To be clear, when the transaction is received we are able to evaluate its validity, and do not need to rescan these transactions again. + +Interestingly, these payments (M8) will *always* be directed to miners from non-miners. Therefore, non-mining nodes do not need to keep them in any mempool at all. Non-miner nodes can just wait for a block to be found, and check the txn then. These transactions more resemble a stock market's pit trades (in contrast, regular Bitcoin txns remind me more of paper checks). + +===== M8_V2 With Lightning ===== + +M8_V2 requires having a LN-channel open with a miner. This may not always be practical (or even possible), especially today. + +A M8_V1 TxOut is expected to contain: + + 1-byte - OP_RETURN (0x6a) + 1-byte - Push the following 68 bytes (0x44) + 4-byte - Message header (0xD0520C6E) + 32-bytes - h* side:block hash + 32-bytes - prevSideBlockHash + 5~7-bytes - BMM request identifying bytes (0x00bf00) + prevBlockRef & ChainIndex (sidechain mini-header) + + +Notice that, in M8_V1, Simon could reuse the same h\* all he wanted, because only one M8_V1 could be included per main:block per sidechain. However, on the LN no such rule can be enforced, as the goal is to push everything off-chain and include *zero* M8s. So, we will never know what the M8s were or how many had an effect on anything. + +Therefore, Simon will need to ensure that he '''gives each Mary a different h\*'''. Simon can easily do this, as he controls the side:block's contents and can simply increment a nonce -- this changes the side:block, and changes its hash (ie, changes h\*). + +With a unique h\* per Mary, and at most 1 h\* making it into a block (per sidechain), we can guarantee that only one of the M8_V2's critical data can be committed to in a single main:block. By giving each miner (who Simon has a payment channel open with) a different h*, Simon can figure out which miner followed through with the commit, and know that only one such commit went through. Furthermore, if this Simon's requested critical data is not found in a block, none of the M8_V2 payments will be spendable by the Mary(s), because none of the h\* in question have ever made it into D3 (which is always on-chain) and no blocks atop will be accumulated. + +That's probably confusing, so here is an example, in which: Simon starts with 13 BTC, Mary starts with 40 BTC, the side:block's tx-fees currently total 7.1 BTC, and Simon is keeping 0.1 BTC for himself and paying 7 BTC to Mary. + +We start with (I): + +
+    Simon 13 in, Mary 40 in ; 53 in total
+        Simon's version [signed by Mary]
+            13 ; to Simon if TimeLock=over; OR to Mary if SimonSig
+            40 ; to Mary
+        Mary's version [signed by Simon]
+            40 ; to me if TimeLock=over; OR to Simon if MarySig
+            13 ; to Simon
+
+ + +And both parties move, from there to "M8_V2" (II): + +
+    Simon 13 in, Mary 40 in ; 53 in total
+        Simon's version [signed by Mary]
+            6 ; to Simon if TimeLock=over; OR to Mary if SimonSig
+            40 ; to Mary
+            7 ; to Mary if critical data requirements met; OR to Simon if LongTimeLock=over
+        Mary's version [signed by Simon]
+            40 ; to Mary if TimeLock=over; OR to Simon if MarySig
+            6 ; to Simon
+            7 ; to Mary if critical data requirements met; OR to Simon if LongTimeLock=over
+
+ + +From here, if the h\* side:block in question is BMMed, they can proceed to (III): + +
+    Simon 13 in, Mary 40 in ; 53 in total
+        Simon's version [signed by Mary]
+            6 ; to Simon if TimeLock=over; OR to Mary if SimonSig
+            47 ; to Mary
+        Mary's version [signed by Simon]
+            47 ; to me if TimeLock=over; OR to Simon if MarySig
+            6 ; to Simon
+
+ +Although, if Simon proceeds immediately, he removes the protection of the 'ratchet'. Ie, Simon removes Mary's incentive to care about blocks being built on this side:block. If Simon's side:block is orphaned, he loses his 7 BTC. Simon can either play it safe, and wait the full 100 side:blocks before moving on (ie, moving on to the third LN txn, above); or else Simon can take the risk if he feels comfortable with it. + +If the h\* side:block is not found, then (II) and (III) are basically equivalent to each other. Simon and Mary could jointly reconstruct (I) and go back there, or they could proceed to a new version of II (with a different h\*, trying again with new side:block in the next main:block). + + + + +==Deployment== + +This BIP will be deployed by "version bits" BIP9 with the name "blindmm" and using bit 4. + +
+// Deployment of Drivechains (BIPX, BIPY)
+consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].bit = 4;
+consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].nStartTime = 1515974401; // January 15th, 2018.
+consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].nTimeout = 1547510401; // January 15th, 2019.
+
+ + +==Reference Implementation== + +See: https://github.com/drivechain-project/bitcoin/tree/mainchainBMM + +Also, for interest, see an example sidechain here: https://github.com/drivechain-project/bitcoin/tree/sidechainBMM + + +==References== + + +* http://www.drivechain.info/literature/index.html +* http://www.truthcoin.info/blog/blind-merged-mining/ +* https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-July/014789.html +* http://www.truthcoin.info/images/bmm-outline.txt + + +==Thanks== + +Thanks to everyone who contributed to the discussion, especially: ZmnSCPxj, Adam Back, Peter Todd, Dan Anderson, Sergio Demian Lerner, Matt Corallo, Sjors Provoost, Tier Nolan, Erik Aronesty, Jason Dreyzehner, Joe Miyamoto, Ben Goldhaber. + + +==Copyright== + +This BIP is licensed under the BSD 2-clause license. -- cgit v1.2.3 From 5418516065646d35e81ee956a94244dbf81ab1c1 Mon Sep 17 00:00:00 2001 From: Paul Sztorc Date: Sat, 10 Feb 2018 17:31:01 -0500 Subject: typos --- bip-blind-merged-mining.mediawiki | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/bip-blind-merged-mining.mediawiki b/bip-blind-merged-mining.mediawiki index 3aaf81b..9a919f8 100644 --- a/bip-blind-merged-mining.mediawiki +++ b/bip-blind-merged-mining.mediawiki @@ -194,12 +194,13 @@ M8_V1 does not require the Lightning network but does have new requirements for A M8_V1 TxOut is expected to contain: +
     1-byte - OP_RETURN (0x6a)
     1-byte - Push the following 36 bytes (0x24)
     4-byte - Message header (0xD1617368)
     32-bytes  - h* side:block hash
     5~7-bytes - BMM request identifying bytes (0x00bf00) + prevBlockRef & ChainIndex (sidechain mini-header)
-    
+
In the first version of M8, we need to introduce the concept of Immediate Expiration (see above). In other words, we need a way for Simon to construct many payments to multiple Marys, such that only one of these is ever included; and only then if Simon's txn is expected to coincide with the finding of Simon's side:block. @@ -225,17 +226,19 @@ Interestingly, these payments (M8) will *always* be directed to miners from non- ===== M8_V2 With Lightning ===== -M8_V2 requires having a LN-channel open with a miner. This may not always be practical (or even possible), especially today. +M8_V2 requires having a LN-channel pathway open with a miner. This may not always be practical (or even possible), especially today. A M8_V1 TxOut is expected to contain: +
     1-byte - OP_RETURN (0x6a)
     1-byte - Push the following 68 bytes (0x44)
     4-byte - Message header (0xD0520C6E)
     32-bytes  - h* side:block hash
     32-bytes  - prevSideBlockHash
     5~7-bytes - BMM request identifying bytes (0x00bf00) + prevBlockRef & ChainIndex (sidechain mini-header)
-    
+
+ Notice that, in M8_V1, Simon could reuse the same h\* all he wanted, because only one M8_V1 could be included per main:block per sidechain. However, on the LN no such rule can be enforced, as the goal is to push everything off-chain and include *zero* M8s. So, we will never know what the M8s were or how many had an effect on anything. -- cgit v1.2.3 From 17db87224d5e58aa29741cc0077f82ff07ab3803 Mon Sep 17 00:00:00 2001 From: Paul Sztorc Date: Sat, 10 Feb 2018 17:31:54 -0500 Subject: move Chris CS indicated via tweet that he felt he did not contribute enough to be a co-author --- bip-blind-merged-mining.mediawiki | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/bip-blind-merged-mining.mediawiki b/bip-blind-merged-mining.mediawiki index 9a919f8..32ffa12 100644 --- a/bip-blind-merged-mining.mediawiki +++ b/bip-blind-merged-mining.mediawiki @@ -6,7 +6,6 @@ Title: Blind Merged Mining (Consensus layer) Author: Paul Sztorc CryptAxe - Chris Stewart Comments-Summary: No comments yet. Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-??????? Status: Draft @@ -325,7 +324,7 @@ Also, for interest, see an example sidechain here: https://github.com/drivechain ==Thanks== -Thanks to everyone who contributed to the discussion, especially: ZmnSCPxj, Adam Back, Peter Todd, Dan Anderson, Sergio Demian Lerner, Matt Corallo, Sjors Provoost, Tier Nolan, Erik Aronesty, Jason Dreyzehner, Joe Miyamoto, Ben Goldhaber. +Thanks to everyone who contributed to the discussion, especially: ZmnSCPxj, Adam Back, Peter Todd, Dan Anderson, Sergio Demian Lerner, Matt Corallo, Sjors Provoost, Tier Nolan, Erik Aronesty, Jason Dreyzehner, Joe Miyamoto, Chris Stewart, Ben Goldhaber. ==Copyright== -- cgit v1.2.3 From 2a981366e7e81d6efb782ea675b7dbfa638cd24e Mon Sep 17 00:00:00 2001 From: Paul Sztorc Date: Sat, 10 Feb 2018 18:26:52 -0500 Subject: clarifications + backward compatibility --- bip-blind-merged-mining.mediawiki | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/bip-blind-merged-mining.mediawiki b/bip-blind-merged-mining.mediawiki index 32ffa12..29a735c 100644 --- a/bip-blind-merged-mining.mediawiki +++ b/bip-blind-merged-mining.mediawiki @@ -19,7 +19,7 @@ ==Abstract== -Blind Merged Mining (BMM) is a way of mining special extension blocks, ie "sidechains". It produces strong guarantees that the block is valid, for *any* arbitrary set of rules; and yet it does so without requiring miners to actually do any validation on the block whatsoever. +Blind Merged Mining (BMM) is a way of mining optional extension blocks, ie "asymmetric sidechains". It produces strong guarantees that the block is valid, for *any* arbitrary set of rules; and yet it does so without requiring miners to actually do any validation on the block whatsoever. BMM actually is a process that spans two or more chains. For an explanation of the "whole picture", please see [this post](http://www.truthcoin.info/blog/blind-merged-mining/). Here we focus on the modifications to mainchain Bitcoin. @@ -54,10 +54,12 @@ As stated above, we have two goals: [1] create and monitor an alt-chain (defined Specifically, per side:block per side:chain, we track the following 35 bytes of information: 1-byte - ChainIndex (known as "Account Number" in hashrate-escrows.md , or as "Sidechain Number") - 32-bytes - sideHeaderHash (also known as "h*" / hashCritical, the hash of the sidechain block) - 2-bytes - prevBlockRef (an index which points to this side:block's parent side:block) + 32-bytes - sideHeaderHash (also known as "h*" / hashCritical, the hash of the blockheader of the sidechain in question) + 2-bytes - prevBlockRef (an index which points to this side:blockheader's parent side:blockheader) -The **ChainIndex** indicates which sidechain this critical data is relevant to. As we may eventually have more than one sidechain, this serves as an identifier similar to the Bitcoin network's magic bytes (0xF9BEB4D9). Drivechains however only need to use 1 byte for the identifier (there is a hard limit of 256 sidechains identified as 0-255). The **sideHeaderHash** is the hash of a side:block which will receive PoW via BMM. It serves a similar function to Bitcoin's "hashMerkleRoot", in that it contains the data for its blockchain. The **prevBlockRef** forms the set of headers into a blockchain structure by making each headers refer to one parent header. It is most similar to Bitcoin's hashPrevBlock. +The **ChainIndex** indicates which sidechain this critical data is relevant to. As we may eventually have more than one sidechain, this serves as an identifier similar to the Bitcoin network's magic bytes (0xF9BEB4D9). Drivechains however only need to use 1 byte for the identifier (there is a hard limit of 256 sidechains identified as 0-255). The **sideHeaderHash** is the hash of a side:blockheader which will receive PoW via BMM. It serves a similar function to Bitcoin's "hashMerkleRoot", in that it contains the data for its blockchain. The **prevBlockRef** forms the set of headers into a blockchain structure by making each headers refer to one parent header. It is most similar to Bitcoin's hashPrevBlock. + +This data is "critical" in the sense that it is the minimum amount of data required to define a sidechain. Where does this data come from, and how does it get around? @@ -138,7 +140,7 @@ To efficiently keep track of the above data, without having to constantly load a This section introduces a new type of transaction, which allows sidechain block creators to request, and pay for, a critical hash to be included in a specific block by mainchain miners. See [the Blind Merged Mining spec](http://www.truthcoin.info/blog/blind-merged-mining/). This txn allows miners to "sell" the act of mining a sidechain block. By taking advantage of this option, miners earn tx fees for mining sidechains...truly "for free". They do not even need to run sidechain nodes, and the tx-fees they earn are in mainchain BTC. As a result, sidechains affect all miners equally and do not affect the mining ecosystem. -M8 will ultimately come in two versions. The second version will be specialized for use in the Lightning Network and must use the full 32-byte prevBlockHash (ironically, this larger transaction is cheaper for the Bitcoin network to process, as it is completely off-chain). The first version of M8, in contrast, cannot be used inside the Lightning Network, but is slightly more space-efficient (using the 2 prevBlockRef bytes to maintain chain order). It is important to make both options available to the user, because some side:nodes may be unwilling or unable to open a payment channels with each main:miner. However, in the long run we expect the lightning version to be preferred. +M8 will ultimately come in two versions. The second version will be specialized for use in the Lightning Network and must use the full 32-byte prevBlockHash (ironically, this larger transaction is cheaper for the Bitcoin network to process, as it is completely off-chain). The first version of M8, in contrast, cannot be used inside the Lightning Network, but is slightly more space-efficient (using the 2 prevBlockRef bytes to maintain chain order). It is important to make both options available to the user, because some side:nodes may be unwilling or unable to open a LN connection with each main:miner. However, in the long run we expect the lightning version to be preferred. ==== Setup ==== @@ -292,6 +294,13 @@ Although, if Simon proceeds immediately, he removes the protection of the 'ratch If the h\* side:block is not found, then (II) and (III) are basically equivalent to each other. Simon and Mary could jointly reconstruct (I) and go back there, or they could proceed to a new version of II (with a different h\*, trying again with new side:block in the next main:block). +==Backward compatibility== + +As a soft fork, older software will continue to operate without modification. As stated above, BMM asks nodes to track a set of ordered hashes, and to allow miners to "sell" the act of finding a sidechain block. Non-upgraded nodes will notice that this activity (specifically: data in coinbases, and new txns that have OP Returns and interesting message headers) is now taking place, but they will not understand any of it. Much like P2SH or a new OP Code, these old users will not be directly affected by the fork, as they will have no expectations of receiving payments of this kind. + +(As a matter of fact, the only people receiving money here all happen to be miners. So there is less reason than ever to expect compatibility problems.) + + ==Deployment== -- cgit v1.2.3 From 485b1318bd4a6f2e1378add714ce7af2cffee87b Mon Sep 17 00:00:00 2001 From: Paul Sztorc Date: Mon, 23 Apr 2018 20:50:11 -0400 Subject: m7 op return update This has been right in the code, but I kept forgetting to update the BIP. --- bip-blind-merged-mining.mediawiki | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/bip-blind-merged-mining.mediawiki b/bip-blind-merged-mining.mediawiki index 29a735c..513a4b7 100644 --- a/bip-blind-merged-mining.mediawiki +++ b/bip-blind-merged-mining.mediawiki @@ -85,12 +85,12 @@ Mainchain nodes are going to need this data later, so it must be easy to find. W ==== M7 -- "Blind-Mine the Sidechain(s)" ==== -Thus, (for n sidechains) we have a coinbase output with: +Thus, (for n sidechains) we have a coinbase output with multiple OP_RETURNs (we've changed the tx-standardness policy to allow multiple OP_RETURNs): 1-byte - OP_RETURN (0x6a) - 1-byte - Push the following (4+(n*35)) bytes (0x??) + 1-byte - Push the following 39 bytes (0x27) 4-byte - Message header (0xD3407053) - (n*(32+5))-byte - A sequence of bytes, of the three Mini-Header items (h*, prevBlockRef, ChainIndex). + 37-byte - A sequence of bytes, of the three Mini-Header items (h*, prevBlockRef, ChainIndex). ( We assume that 5 bytes are used for the Critical Data bytes (non h* parts of the Sidechain Mini-Header). For 256 sidechains, a total of 9,478 bytes [1+1+4+256\*(32+5)] are required, conveniently just below the 10 KB scriptPubKey size limit.) -- cgit v1.2.3 From cc122a76ef50b1a8607a5c4541f352a705bd112e Mon Sep 17 00:00:00 2001 From: Steve Lee Date: Tue, 29 Jan 2019 12:19:16 -0800 Subject: fixed minor typo --- bip-0079.mediawiki | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bip-0079.mediawiki b/bip-0079.mediawiki index 7d0cc57..ca6179e 100644 --- a/bip-0079.mediawiki +++ b/bip-0079.mediawiki @@ -26,7 +26,7 @@ This document is licensed under the Creative Commons CC0 1.0 Universal license. One of the most powerful blockchain analysis heuristics has been to assume all inputs of a transaction are controlled by a single party unless otherwise known (such as by the distinctive structure of a traditional coinjoin, or multisig spends that are validated onchain). Combined with other techniques (notably change-output guessing) this has lead to unexpectedly accurate tracking that has exposed bitcoin participants to unacceptable personal, business and financial risks -- undermining bitcoin's utility and fungibility -- and ultimately jeopardizing its ability to function as useful money. -We however can bust these assumption with a sender-receiver coinjoin. To prevent costless spy/DoS attacks, we require the sending party to provide a fully-valid ready-to-propagate transaction to initiate the process, that the receiver can broadcast if the sender never completes the coinjoin thus tying the cost to that of spending a utxo. Most promisingly, bustapay transactions do not have an identifiable structure so any network analysis will be not able to tell if a given transaction is a bustapay transaction or not which erodes the confidence of their entire models, providing positive externalities for the entire bitcoin ecosystem. +We however can bust these assumptions with a sender-receiver coinjoin. To prevent costless spy/DoS attacks, we require the sending party to provide a fully-valid ready-to-propagate transaction to initiate the process, that the receiver can broadcast if the sender never completes the coinjoin thus tying the cost to that of spending a utxo. Most promisingly, bustapay transactions do not have an identifiable structure so any network analysis will be not able to tell if a given transaction is a bustapay transaction or not which erodes the confidence of their entire models, providing positive externalities for the entire bitcoin ecosystem. Bustapay transactions also do not grow the receiver's count of unspent transaction outputs, and in fact gives the receiver an opportunity to better manage their utxo set, something normally only done when sending payments. Large utxo sets are often problematic and expensive, and frequently requiring privacy-destroying consolidation. Besides busting clustering assumptions, bustapay also provides a layer of obfuscation of send amounts. -- cgit v1.2.3 From ec8b400259d74de507f0e7856899397fc5a15671 Mon Sep 17 00:00:00 2001 From: "Wladimir J. van der Laan" Date: Wed, 27 Feb 2019 10:28:34 +0100 Subject: addrv2 BIP proposal --- bip-XXXX.mediawiki | 189 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 189 insertions(+) create mode 100644 bip-XXXX.mediawiki diff --git a/bip-XXXX.mediawiki b/bip-XXXX.mediawiki new file mode 100644 index 0000000..583bf8b --- /dev/null +++ b/bip-XXXX.mediawiki @@ -0,0 +1,189 @@ +
+  BIP: ???
+  Layer: Peer Services
+  Title: addrv2 message
+  Author: Wladimir J. van der Laan 
+  Comments-Summary: No comments yet.
+  Comments-URI: 
+  Status: Draft
+  Type: Standards Track
+  Created: 2019-02-27
+  License: BSD-2-Clause
+
+ +==Introduction== + +===Abstract=== + +This document proposes a new P2P message to gossip longer node addresses over the P2P network. +This is required to support new-generation Onion addresses, I2P, and potentially other networks +that have longer endpoint addresses than fit in the 128 bits of the current addr message. + +===Copyright=== + +This BIP is licensed under the 2-clause BSD license. + +===Motivation=== + +Tor v3 hidden services are part of the stable release of Tor since version 0.3.2.9. They have +various advantages compared to the old hidden services, among which better encryption and privacy +[https://gitweb.torproject.org/torspec.git/tree/rend-spec-v3.txt Tor Rendezvous Specification - Version 3]. +These services have 256 bit addresses and thus do not fit in the existing addr message, which encapsulates onion addresses in OnionCat IPv6 addresses. + +Other transport-layer protocols such as I2P have always used longer +addresses. This change would make it possible to gossip such addresses over the +P2P network, so that other peers can connect to them. + +==Specification== + +
+The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", +"SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be +interpreted as described in RFC 2119[https://tools.ietf.org/html/rfc2119 RFC 2119]. +
+ +The addrv2 message is defined as a message where pchCommand == "addrv2". +It is serialized in the standard encoding for P2P messages. +Its format is similar to the current addr message format +[https://bitcoin.org/en/developer-reference#addr Bitcoin Developer Reference: addr message], with the difference that the +fixed 16-byte IP address is replaced by a network ID and a variable-length address, and the time and services format has been changed to VARINT. + +This means that the message contains a serialized std::vector of the following structure: + +{| class="wikitable" style="width: auto; text-align: center; font-size: smaller; table-layout: fixed;" +!Type +!Name +!Description +|- +| VARINT (unsigned) +| time +| Time that this node was last seen as connected to the network. A time in Unix epoch time format, up to 64 bits wide. +|- +| VARINT (unsigned) +| services +| Service bits. A 64-wide bit field. +|- +| uint8_t +| networkID +| Network identifier. An 8-bit value that specifies which network is addressed. +|- +| std::vector +| addr +| Network address. The interpretation depends on networkID. +|- +| uint16_t +| port +| Network port. If not relevant for the network this MUST be 0. +|} + +One message can contain up to 1,000 addresses. Clients SHOULD reject messages with more addresses. + +Field addr has a variable length, with a maximum of 32 bytes (256 bits). Clients SHOULD reject +longer addresses. + +The list of reserved network IDs is as follows: + +{| class="wikitable" style="width: auto; text-align: center; font-size: smaller; table-layout: fixed;" +!Network ID +!Enumeration +!Address length (bytes) +!Description +|- +| 0x01 +| IPV4 +| 4 +| IPv4 address (globally routed internet) +|- +| 0x02 +| IPV6 +| 16 +| IPv6 address (globally routed internet) +|- +| 0x03 +| TORV2 +| 10 +| Tor v2 hidden service address +|- +| 0x04 +| TORV3 +| 32 +| Tor v3 hidden service address +|- +| 0x05 +| I2P +| 32 +| I2P overlay network address +|- +| 0x06 +| CJDNS +| 16 +| Cjdns overlay network address +|} + +To allow for future extensibility, clients MUST ignore address types that they do not know about. +Client MAY store and gossip address formats that they do not know about. Further network ID numbers MUST be reserved in a new BIP document. + +Clients SHOULD reject addresses that have a different length than specified in this table for a specific address ID, as these are meaningless. + +See the appendices for the address encodings to be used for the various networks. + +==Compatibility== + +Send addrv2 messages only, and exclusively, when the peer has a certain protocol version (or higher): + +//! gossiping using `addrv2` messages starts with this version +static const int GOSSIP_ADDRV2_VERSION = 70016; + +For older peers keep sending the legacy addr message, ignoring addresses with the newly introduced address types. + +==Reference implementation== + +The reference implementation is available at (to be done) + +==Acknowledgements== + +- Jonas Schnelli: change services field to VARINT, to make the message more compact in the likely case instead of always using 8 bytes. + +- Luke-Jr: change time field to VARINT, for post-2038 compatibility. + +- Gregory Maxwell: various suggestions regarding extensibility + +==Appendix A: Tor v2 address encoding== + +The new message introduces a separate network ID for TORV2. + +Clients MUST send Tor hidden service addresses with this network ID, with the 80-bit hidden service ID in the address field. This is the same as the representation in the legacy addr message, minus the 6 byte prefix of the OnionCat wrapping. + +Clients SHOULD ignore OnionCat (fd87:d87e:eb43::/48) addresses on receive if they come with the IPV6 network ID. + +==Appendix B: Tor v3 address encoding== + +According to the spec [https://gitweb.torproject.org/torspec.git/tree/rend-spec-v3.txt Tor Rendezvous Specification - Version 3: Encoding onion addresses], next-gen .onion addresses are encoded as follows: +
+onion_address = base32(PUBKEY | CHECKSUM | VERSION) + ".onion"
+ CHECKSUM = H(".onion checksum" | PUBKEY | VERSION)[:2]
+
+ where:
+   - PUBKEY is the 32 bytes ed25519 master pubkey of the hidden service.
+   - VERSION is an one byte version field (default value '\x03')
+   - ".onion checksum" is a constant string
+   - CHECKSUM is truncated to two bytes before inserting it in onion_address
+
+ +Tor v3 addresses MUST be sent with the TORV3 network ID, with the 32-byte PUBKEY part in the address field. As VERSION will always be '\x03' in the case of v3 addresses, this is enough to reconstruct the onion address. + +==Appendix C: I2P address encoding== + +Like Tor, I2P naming uses a base32-encoded address format[https://geti2p.net/en/docs/naming#base32 I2P: Naming and address book]. + +I2P uses 52 characters (256 bits) to represent the full SHA-256 hash, followed by .b32.i2p. + +I2P addresses MUST be sent with the I2P network ID, with the decoded SHA-256 hash as address field. + +==Appendix D: Cjdns address encoding== + +Cjdns addresses are simply IPv6 addresses in the fc00::/8 range[https://github.com/cjdelisle/cjdns/blob/6e46fa41f5647d6b414612d9d63626b0b952746b/doc/Whitepaper.md#pulling-it-all-together Cjdns whitepaper: Pulling It All Together]. They MUST be sent with the CJDNS network ID. + +==References== + + -- cgit v1.2.3 From 4f73814175c9e83c302092027d360e0bd9fa55a2 Mon Sep 17 00:00:00 2001 From: Lucas Cullen Date: Wed, 6 Mar 2019 07:51:45 +1000 Subject: Update the link to NBitcoin repo NBitcoin official repo has changed. --- bip-0032.mediawiki | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bip-0032.mediawiki b/bip-0032.mediawiki index 18b3b0c..9339307 100644 --- a/bip-0032.mediawiki +++ b/bip-0032.mediawiki @@ -294,7 +294,7 @@ Two JavaScript implementations exist: available at https://github.com/sarchar/br A PHP implementation is available at https://github.com/Bit-Wasp/bitcoin-lib-php -A C# implementation is available at https://github.com/NicolasDorier/NBitcoin (ExtKey, ExtPubKey) +A C# implementation is available at https://github.com/MetacoSA/NBitcoin (ExtKey, ExtPubKey) A Haskell implementation is available at https://github.com/haskoin/haskoin together with a CLI interface at https://github.com/np/hx -- cgit v1.2.3 From 23590c0508d6df6a68cd6fdf6e9aaa13cc77fe6a Mon Sep 17 00:00:00 2001 From: azuchi Date: Sun, 31 Mar 2019 14:19:00 +0900 Subject: [BIP197] Fix description of Refund Period Seizable Collateral script have condition that can be refund by the borrower after the Seizure Period. --- bip-0197.mediawiki | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bip-0197.mediawiki b/bip-0197.mediawiki index 8a34b04..427ff22 100644 --- a/bip-0197.mediawiki +++ b/bip-0197.mediawiki @@ -130,7 +130,7 @@ In the case of a default or the lender not accepting the borrower repayment, the In the case that either the lender or borrower don’t accept the bid, the lender can seize a percentage of the collateral. The amount is dependent on the amount of collateral locked in the Seizable Collateral and Refundable Collateral script as described in this BIP. During this period, the borrower can also refund the funds locked in the Refundable Collateral script. ===Refund Period=== -In the case that the lender does not seize the collateral locked in the Seizable Collateral script, then the borrower can refund the funds locked in the Refundable Collateral script. +In the case that the lender does not seize the collateral locked in the Seizable Collateral script, then the borrower can refund the funds locked in the Seizable Collateral script. ==Rationale== -- cgit v1.2.3 From c90088ed815ea243db7454b921461ea97113f006 Mon Sep 17 00:00:00 2001 From: Paul Sztorc Date: Thu, 4 Apr 2019 16:22:09 -0700 Subject: improved image, with examples --- bip-blind-merged-mining/witness-vs-critical.png | Bin 67570 -> 265603 bytes 1 file changed, 0 insertions(+), 0 deletions(-) diff --git a/bip-blind-merged-mining/witness-vs-critical.png b/bip-blind-merged-mining/witness-vs-critical.png index 1a2458d..77003e8 100644 Binary files a/bip-blind-merged-mining/witness-vs-critical.png and b/bip-blind-merged-mining/witness-vs-critical.png differ -- cgit v1.2.3 From bbcab029ea0c4e6237087e76102e62df5e1d530d Mon Sep 17 00:00:00 2001 From: Paul Sztorc Date: Thu, 4 Apr 2019 16:26:19 -0700 Subject: number, shorten, clarify, link to working code --- bip-blind-merged-mining.mediawiki | 253 +++++++++++--------------------------- 1 file changed, 74 insertions(+), 179 deletions(-) diff --git a/bip-blind-merged-mining.mediawiki b/bip-blind-merged-mining.mediawiki index 513a4b7..ffa883a 100644 --- a/bip-blind-merged-mining.mediawiki +++ b/bip-blind-merged-mining.mediawiki @@ -1,13 +1,13 @@
 
-    BIP: ????
+    BIP: 301
     Layer: Consensus (soft fork)
     Title: Blind Merged Mining (Consensus layer)
     Author: Paul Sztorc 
             CryptAxe 
     Comments-Summary: No comments yet.
-    Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-???????
+    Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-301
     Status: Draft
     Type: Standards Track
     Created: 2017-10-24
@@ -19,16 +19,11 @@
 ==Abstract==
 
 
-Blind Merged Mining (BMM) is a way of mining optional extension blocks, ie "asymmetric sidechains". It produces strong guarantees that the block is valid, for *any* arbitrary set of rules; and yet it does so without requiring miners to actually do any validation on the block whatsoever.
+Blind Merged Mining (BMM) is a way of mining optional extension blocks (ie, "asymmetric sidechains"). BMM produces weak guarantees that the block is valid, for *any* arbitrary set of rules; and yet it does so without requiring miners to actually do any validation on the block whatsoever.
 
-BMM actually is a process that spans two or more chains. For an explanation of the "whole picture", please see [this post](http://www.truthcoin.info/blog/blind-merged-mining/). Here we focus on the modifications to mainchain Bitcoin.
+BMM actually is a process that spans two or more chains. Here we focus on the modifications to mainchain Bitcoin. For an explanation of the "whole picture", please see [http://www.truthcoin.info/blog/blind-merged-mining/ this post].
 
-To support BMM, the mainchain is asked to accomplish two goals:
-
-# Track a set of ordered hashes (the merged-mining).
-# Allow miners to "sell" the act of finding a sidechain block (through the use of a new extended serialization transaction type).
-
-These goals are accomplished by forcing nodes to validate two new messages (M7, M8), and track data in one new database (D3).
+Our goal here, is to allow mainchain miners to trustlessly "sell" the act of finding a sidechain block.
 
 
 ==Motivation==
@@ -39,213 +34,94 @@ Regular "Merged-Mining" (MM) allows miners to reuse their hashing work to secure
 # Miners must run a full node of the other chain. (This is because [while miners can effortlessly create the block] miners will not create a valid payment to themselves, unless the block that they MM is a valid one. Therefore, miners must assemble a *valid* block first, then MM it.)
 # Miners are paid on the other chain, not on the regular BTC mainchain. For example, miners who MM Namecoin will earn NMC (and they will need to sell the NMC for BTC, before selling the BTC in order to pay for electricity).
 
-Blind Merged-Mining (BMM) attempts to address those shortcomings.
+BMM addresses both shortcomings.
 
 
 ==Specification==
 
 
-Note: This document uses the notation side:\* and main:\* in front of otherwise-ambiguous words (such as "block", "node", or "chain"), to distinguish the mainchain version from its sidechain counterpart.
-
-As stated above, we have two goals: [1] create and monitor an alt-chain (defined only by a deterministic list of hashes), and [2] allow miners to "sell" the act of finding a sidechain block (through the use of a new extended serialization transaction type).
-
-=== Sidechain Critical Data ("Sidechain Mini-Header") ===
-
-Specifically, per side:block per side:chain, we track the following 35 bytes of information:
-
-    1-byte   - ChainIndex (known as "Account Number" in hashrate-escrows.md , or as "Sidechain Number")
-    32-bytes - sideHeaderHash (also known as "h*" / hashCritical, the hash of the blockheader of the sidechain in question)
-    2-bytes  - prevBlockRef (an index which points to this side:blockheader's parent side:blockheader)
-
-The **ChainIndex** indicates which sidechain this critical data is relevant to. As we may eventually have more than one sidechain, this serves as an identifier similar to the Bitcoin network's magic bytes (0xF9BEB4D9). Drivechains however only need to use 1 byte for the identifier (there is a hard limit of 256 sidechains identified as 0-255). The **sideHeaderHash** is the hash of a side:blockheader which will receive PoW via BMM. It serves a similar function to Bitcoin's "hashMerkleRoot", in that it contains the data for its blockchain. The **prevBlockRef** forms the set of headers into a blockchain structure by making each headers refer to one parent header. It is most similar to Bitcoin's hashPrevBlock.
-
-This data is "critical" in the sense that it is the minimum amount of data required to define a sidechain.
-
-Where does this data come from, and how does it get around?
-
-==== Creating / Broadcasting This Data ====
-
-===== Creation =====
-
-By the time Blind Merged Mining can take place, the ChainIndex is globally known (it is the "Account Number" in D1 [see previous BIP], and "nSidechain" in the code). Each sidechain, when activated by soft fork, will take one of the 0-255 available indexes.
-
-The other two items, sideHeaderHash and prevBlockRef, are created by sidechain nodes. sideHeaderHash is quite straightforward -- side:nodes build side:blocks, and take the hash of these.
-
-The final item, prevBlockRef, is a little more complicated. It is an integer that counts the number of "skips" one must take in the side:chain in order to find the current side:block's parent block. In practice, this value will usually be zero. It will only be a value other than zero, in cases where invalid sidechain blocks have been mined, or when a side:node intentionally wants to orphan some side:blocks (if a side:node wants to orphan the most-recent N blocks, the value of the current block will be equal to N ; in the block after that it will be back to zero).
-
-
-
-Since the hashes themselves are already ordered by the mainchain, tracing the blockchain's path by index (prevBlockRef) will be the same as tracing it by identifying a list of hashes. In other words, the ordering given via each side:block's "prevBlockRef" will be isomorphic to an ordering given by each side:block's "prevSideHeaderHash" ... if "prevSideHeaderHash is defined to be the sidechain's equivalent of the mainchain's "prevBlockHash". It will be possible to freely convert from one to the other. See M8 to learn more about how these hashes are requested by sidechain block creators to be included in the mainchain.
-
-Now that we know what our critical data is, and how it is made, how is this data broadcast and stored?
-
-===== Broadcast =====
+Note: This document uses the notation side:\* and main:\* in front of otherwise-ambiguous words (such as "block", "node", or "chain"), to distinguish the mainchain version from its sidechain counterpart. We also use "Simon" to refer to a Sidechain Full Node, and "Mary" to refer to a mainchain miner.
 
-Mainchain nodes are going to need this data later, so it must be easy to find. We will put it into the coinbase via OP RETURN.
 
-==== M7 -- "Blind-Mine the Sidechain(s)" ====
+=== BMM Request ===
 
-Thus, (for n sidechains) we have a coinbase output with multiple OP_RETURNs (we've changed the tx-standardness policy to allow multiple OP_RETURNs):
+To buy the right to find a sidechain block, users broadcast BMM Requests.
 
-    1-byte - OP_RETURN (0x6a)
-    1-byte - Push the following 39 bytes (0x27)
-    4-byte - Message header (0xD3407053)
-    37-byte - A sequence of bytes, of the three Mini-Header items (h*, prevBlockRef, ChainIndex).
-
-( We assume that 5 bytes are used for the Critical Data bytes (non h* parts of the Sidechain Mini-Header). For 256 sidechains, a total of 9,478 bytes [1+1+4+256\*(32+5)] are required, conveniently just below the 10 KB scriptPubKey size limit.)
+Here, these can take two forms. The first does not require the Lightning Network, but it does have new requirements for Immediate Expiration (see below). The second inherits Immediate Expiration from the Lightning Network itself, but requires extra preparation and a different/larger message.
 
-This data is parsed by laying it in sequential 37-byte chunks (any remaining data --ie, some final chunk that is less than 37 bytes in length-- has no consensus meaning). 
+Both forms require that certain Critical Data will be committed to within the coinbase of the block that the transaction is included in (see BMM Accept). For the OnChain (non-Lightning) version, we have created a new extended serialization transaction type (very similar to how segwit handles witness data (the witness stack)).
 
-Each 37-byte chunk is then parsed to obtain the data outlined above (in "Description"). If two 35-byte chunks being with the same "Sidechain number" (ie, if the two chunks have the same first byte), then only the first chunk has consensus meaning.
+==== Immediate Expiration ("Fill-or-Kill") ====
 
-We are left with, at most, one (h*, prevBlockRef) pair per sidechain per block. This data is added directly to D3, a new database.
+We would like to make special guarantees to the counterparties of this transaction. Specifically, instead of Simon making a "payment" to Mary, we prefer that Simon give Mary an "offer" (which she can either accept or decline).
 
-==== D3 -- "RecentSidechains_DB" ====
+Crucially, we want Simon to safely make many offers to several different Mary's, in realtime (ie, quickly and off-chain). However, we ultimately want only one offer to be accepted, at most. In other words, we want Simon's offers to *immediately expire*. If only one offer can become a bona fide transaction, then Simon will feel comfortable making multiple offers all day long. Because all of the Simons are making many offers, the Marys collectively gain access to a large set of offers to choose from.
 
-To suit our purposes, the mainchain full nodes will need to keep track of the most recent 8000 (h\*, prevBlockRef) pairs.
+==== OnChain BMM Request ====
 
-( This 8,000 figure is a tradeoff between decentralization (costs of running the main:node) and sidechain security -- it requires attackers to merged-mine 8,000 invalid blocks consecutively, in order to cause the sidechain to fail. The mainchain burden is minimal, so this figure might be raised to 12,000 or higher. )
+OnChain BMMRs do not require the Lightning network, but they do have new requirements for validation.
 
-Therefore, D3 would look something like this:
+===== Structure =====
 
+The following data is required:
 
 
-           BlockHeight  CB_Index    SC_1   Blks_Atop_1   SC 2   Blks_Atop_2   SC 3   Blks_Atop_3
-            ---------    ------    ------   ---------   ------   ---------   ------   ---------
-       1.    401,005        2      (h*, 0)     7985     (h*, 0)        1     (h*, 0)        0
-       2.    401,006        4      (h*, 0)     7984     (h*, 0)        0     (h*, 1)     7801
-       3.    401,007        2      (h*, 0)     7983     (h*, 5)     2027     (h*, 0)        0
-       4.    401,008        2      (h*, 0)     7982     (h*, 0)     2028     (h*, 1)     7800
-      ...     ...                                                                  )
-    7999.    409,003        3      (h*, 0)        1     (h*, 0)        0     (h*, 0)        1
-    8000.    409,004        2      (h*, 0)        0     (h*, 1)        0     (h*, 0)        0
+    32-bytes  - h* sideHeaderHash
+    ?~?-bytes - critical data extended serialization
+        3-bytes - 0x00bf00 identifying bytes
+        1-byte  - nSidechain
+        2-bytes - prevSideBlockRef
+        4-bytes - prevMainHeaderBytes
 
+sideHeaderHash comes from side:chain (side:nodes build side:blocks/headers). The identifying bytes are given here. nSidechain identifies which sidechain we are BMMing. By the time Blind Merged Mining can take place, it is known globally. -When new sidechains (or "hashrate escrows") are soft-forked into existence, a new column is added to D3 (to contain any BMMing that might be done on it). - -For each sidechain we also track the field "Blocks Atop". This is the number of side:blocks that are "on top" of the specified side:block. These might be regarded as "side:confirmations" (pseudo-confirmations that are specific to each sidechain). - -D3 also contains a column (not shown) for each sidechain containing "prevSideBlockHash". This value is is either derived from prevBlockRef; or else it is given explicitly (in which case it is the converse: prevBlockRef is derived from prevSideBlockHash). - - -==== Coinbase Cache ==== - -As mentioned above, M7s cause data to be added to D3. Recent D3 data is tracked by all mainchain nodes for a period of time. - -To efficiently keep track of the above data, without having to constantly load and process entire blocks from disk, we temporarily cache enough coinbases in the chain index to maintain D3. - - -=== M8 -- Paying miners to include BMM data in their coinbase outputs === - -This section introduces a new type of transaction, which allows sidechain block creators to request, and pay for, a critical hash to be included in a specific block by mainchain miners. See [the Blind Merged Mining spec](http://www.truthcoin.info/blog/blind-merged-mining/). This txn allows miners to "sell" the act of mining a sidechain block. By taking advantage of this option, miners earn tx fees for mining sidechains...truly "for free". They do not even need to run sidechain nodes, and the tx-fees they earn are in mainchain BTC. As a result, sidechains affect all miners equally and do not affect the mining ecosystem. - -M8 will ultimately come in two versions. The second version will be specialized for use in the Lightning Network and must use the full 32-byte prevBlockHash (ironically, this larger transaction is cheaper for the Bitcoin network to process, as it is completely off-chain). The first version of M8, in contrast, cannot be used inside the Lightning Network, but is slightly more space-efficient (using the 2 prevBlockRef bytes to maintain chain order). It is important to make both options available to the user, because some side:nodes may be unwilling or unable to open a LN connection with each main:miner. However, in the long run we expect the lightning version to be preferred. - -==== Setup ==== - -We define **"Mary"** as a mainchain miner, and **"Simon"** as a sidechain node. - -The goal is to construct a payment from Simon to Mary, such that: - -# If the critical data conditions are met, **Mary** can claim the outputs of the transaction with finality. -# If the critical data conditions are not met, the outputs become immediately available again to **Simon**. - - -==== Goals (this is rather philosophical, and skippable) ==== +prevBlockRef, is a little more complicated (next section). -===== Immediate Expiration ("Fill-or-Kill") ===== +To qualify for inclusion in a block, BMM requests are subject to the following reqirements: -We would like to make special guarantees to the counterparties of this transaction. Specifically, instead of Simon making a "payment" to Mary, we prefer that Simon give Mary an "offer" (which she can either accept or decline). - -Crucially, we want Simon to safely make many offers to several different Mary's, in realtime (ie, quickly and off-chain). However, we ultimately want only one offer to be accepted, at most. In other words, we want Simon's offers to *immediately expire*. If only one offer can become a bona fide transaction, then Simon will feel comfortable making offers all day long. Because all of the Simons are making many offers, the Marys collectively gain access to a large set of offers to choose from. - -===== Forward Progress (The Need for a "Ratchet") ===== - -The "ratchet" concept is an attempt to harmonize incentives among the main and side chain(s). -We need to ensure that a sidechain is making "forward progress", without tracking too much about the sidechain such that we burden Bitcoin (see [1] and [2]) all while still allowing the sidechain to reorganize [3]. - -* [1] https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-July/014789.html -* [2] http://www.drivechain.info/faq/index.html#what-is-the-difference-between-drivechain-and-extension-blocks -* [3] http://www.truthcoin.info/blog/blind-merged-mining/#handling-reorganizations - -The ratchet system must keep track of sidechain "mini-headers" (see Sidechain Critical Data ("Sidechain Mini-Header")) and count the "blocks atop" maturity of the related side:blocks. - -Simon's offer to Mary to include a critical hash in exchange for payment must be *atomic*. The "ratchet" concept helps to construct a very tight connection between two things: - -# The sidechain-block-generator "Simon" paying himself the side:block's side:tx-fees (which he receives in 100 sidechain blocks (blocks atop) hence). -# "Simon" making a mainchain main:btc payment to a mainchain miner "Mary". - -Either both of the two should succeed, or else both should jointly fail. - -However, absent our intervention, there are cases in which [2, the payment to Mary] succeeds but [1, side:tx-fees] fails. One such case is when a side:block contains unusually high side:tx-fees. Here, there will be many requests to include a critical hash in exchange for payment submitted to Mary, but only one can be included in each main:block per sidechain. Without an incentive to make "forward progress", Mary is likely to include one of the highest paying requests in the next main:block (and the main:block after that, and so on). Mary will "blindly" include high-paying requests for *older* blocks, unless something prevents her from doing so. - -To address these potential issues, we utilize the concept of "Blocks_Atop" (the "side:confirmations") that we mentioned earlier. As previously mentioned, Mary will not be able to spend Simon's M8 payment until satisfying the critical data requirements as well as the blocks atop (side:confirmations) requirement. - - -==== M8 -- The two forms of M8 transactions ==== - -As previously mentioned, M8 can take two forms. The first does not require the Lightning Network, but it does have new requirements for Immediate Expiration (see above). The second inherits Immediate Expiration from the Lightning Network itself, but requires extra preparation and a different/larger message. - -Both forms require that certain Critical Data will be committed to within the coinbase of the block that the transaction is included in. For the non Lightning version, we have created a new extended serialization transaction type (very similar to how segwit handles witness data (the witness stack)). - -===== M8_V1 - No Lightning Network ===== - -M8_V1 does not require the Lightning network but does have new requirements for validation. - -A M8_V1 TxOut is expected to contain: - -
-    1-byte - OP_RETURN (0x6a)
-    1-byte - Push the following 36 bytes (0x24)
-    4-byte - Message header (0xD1617368)
-    32-bytes  - h* side:block hash
-    5~7-bytes - BMM request identifying bytes (0x00bf00) + prevBlockRef & ChainIndex (sidechain mini-header)
-
+# Requests must match a corresponding "BMM Accept" (see last section of BIP). +# At most, only one Request is allowed in a main:block, per sidechain. In other words, if 700 users broadcast BMM Requests for sidechain #4, then the main:miner must choose one single Request to include. +# The 4-bytes of prevMainHeaderBytes must match the last four bytes of the previous main:blockheader. Thus, Simon's txns are only be valid for the current block, in the block history that he knows about (and therefore, the current sidechain history that he knows about). -In the first version of M8, we need to introduce the concept of Immediate Expiration (see above). In other words, we need a way for Simon to construct many payments to multiple Marys, such that only one of these is ever included; and only then if Simon's txn is expected to coincide with the finding of Simon's side:block. +===== prevBlockRef ===== -We do this by imposing validity-rules on the txn itself: +prevBlockRef is an integer that counts the number of "skips" one must take in the side:chain in order to find the current side:block's parent block. This value is zero unless the sidechain is reorganizing (or skipping over invalid sidechain blocks). If a side:node wants to orphan the most-recent N blocks, the value of the current block will be equal to N; in the block after that it will be back to zero. -# The txn's content, when examined, must match part of the main:block's content. Specifically, the (ChainIndex, h\*) pair of the txn, must match one of the (ChainIndex, h\*) pairs in the M7 of this main:block. -# Only one payment per sidechain per main:block is valid. In other words, if 400 people all try to bm-mine the sidechain with ChainIndex==4, then not only is it the case that only one side_4:block can be found, but it is also the case that only the corresponding M8 txn can be included (out of all of the 400 M8s which are for ChainIndex==4). -# Simon's txns must only be valid for the current block; afterward, they immediately expire. This is because Simon's intended prevBlockRef & side:block contents will most likely change from one main:block to the next. + -To impose new requirements on the transaction level (not the block level nor the TxOutput level), we borrow the "flag" trick from SegWit style transactions. If the flag is present, the transaction is examined for extra data, and if this data does not pass certain requirements, the transaction is invalid. With SegWit, this extra data is the signatures, and the extra requirements are the signatures' locations and validity. In the BMM-transactions, the extra data is the (ChainIndex, h\*) pair, which must meet the first two requirements (above) as well as the main:blocknumber, which must meet the third requirement (above). +Above: Three blockchains, with different max length (small number), reorganization histories, and prevBlockRef numbers (larger numbers beneath blocks). The ordering given via each side:block's "prevSideBlockRef" will be isomorphic to an ordering given by each side:block's "prevSideHeaderHash" ("prevSideHeaderHash is the sidechain's equivalent of the mainchain's "prevBlockHash"). One can freely convert from one to the other. -To impose new requirements at the transaction level, we borrow the dummy vin & "flag" trick from SegWit style transactions. If the flag is set to 2 (0010), the transaction contains Critical Data and requires that our new validation rules be met in order for the txn to be valid in a block. Unless all of the requirements for sidechain critical data transactions are met by the block it is included in, the transaction is invalid. With SegWit, this extra data is the segwit signature stack, and the extra requirements are the signatures' locations and validity. In the sidechain BMM critical data transactions, the extra data is the (ChainIndex, h\*) pair, which must meet the first two requirements (above) as well as the main:blocknumber, which must meet the third requirement (above). Note The main:blocknumber does not take up any additional space compared to a normal txn, as we reuse the locktime field for our purposes. +===== Extended Serialization ===== +To impose new requirements at the transaction level, we borrow the dummy vin & "flag" trick from SegWit style transactions. Unless all of the requirements for sidechain critical data transactions are met by the block it is included in, the transaction is invalid. With SegWit, this extra data is the segwit signature stack, and the extra requirements are the signatures' locations and validity. In the sidechain BMM critical data transactions, the extra data is the (nSidechain, h\*) pair, which must meet the first two requirements (above) as well as the main:blocknumber, which must meet the third requirement (above). +Above: A chart showing normal txns, SegWit txns, and CriticalData txns. The specific SegWit txn can be seen [http://srv1.yogh.io/#tx:id:D4A99AE93DF6EE3D4E42CE69338DFC1D06CCD9B198666E98FF0588057378D3D9 here]. -This txn structure conserves main:blockspace, because it is the easiest way to refer to a previous sidechain block in 4 bytes, (prevBlockRef + FoK_nLockTime). Instead, we would need to use at least 32 bytes (prevSideBlockHash). +These types of transactions have slightly different mempool behavior, and should probably be kept in a second mempool. These txns are received, checked immediately, and if valid they are evaluated for inclusion in a block. If they are not able to be included in the specific requested block (if the block height requested has been surpassed by the chain tip), they are discarded. In fact, after any main:block is found, everything in this "second mempool" can be discarded as new payments will be created immediately for the next block height. (This includes cases where the blockchain reorganizes.) There is no re-evaluation of the txns in this mempool ever -- they are evaluated once and then either included or discarded. They never need to be rescanned. -These types of transactions have slightly different mempool behavior, and should probably be kept in a second mempool. These txns are received, checked immediately, and if valid they are evaluated for inclusion in a block. If they are not able to be included in the specific requested block (if the block height requested has been surpassed by the chain tip), they are discarded. In fact, after any main:block is found, everything in this "second mempool" can be discarded as new payments will be created immediately for the next block height. (This includes cases where the blockchain reorganizes.) There is no re-evaluation of the txns in this mempool ever -- they are evaluated once and then either included or discarded. To be clear, when the transaction is received we are able to evaluate its validity, and do not need to rescan these transactions again. +Interestingly, these payments will *always* be directed to main:miners from non-main:miners. Therefore, non-mining full nodes do not need to keep them in any mempool at all. Non-miner nodes can just wait for a block to be found, and check the txn then. These transactions more resemble a stock market's pit trade-offers (in contrast, regular Bitcoin txns are more like paper checks). -Interestingly, these payments (M8) will *always* be directed to miners from non-miners. Therefore, non-mining nodes do not need to keep them in any mempool at all. Non-miner nodes can just wait for a block to be found, and check the txn then. These transactions more resemble a stock market's pit trades (in contrast, regular Bitcoin txns remind me more of paper checks). +==== Lightning BMM Request ==== -===== M8_V2 With Lightning ===== +Lightning BMMRs require Simons to have a LN-channel pathways open with Marys. This may not always be practical (or even possible), especially today. -M8_V2 requires having a LN-channel pathway open with a miner. This may not always be practical (or even possible), especially today. +LN txns cannot make use of prevSideBlockRef, as no one knows for sure when (or if) they will be broadcast on-chain. Instead, they must use prevSideBlockHash. But they otherwise require the same data: -A M8_V1 TxOut is expected to contain: - -
-    1-byte - OP_RETURN (0x6a)
-    1-byte - Push the following 68 bytes (0x44)
-    4-byte - Message header (0xD0520C6E)
-    32-bytes  - h* side:block hash
-    32-bytes  - prevSideBlockHash
-    5~7-bytes - BMM request identifying bytes (0x00bf00) + prevBlockRef & ChainIndex (sidechain mini-header)
+
	
+    4-bytes - Message header (0xD0520C6E)	
+    1-byte - sidechain number
+    32-bytes  - h* side:block hash	
+    32-bytes  - prevSideBlockHash	
 
+Notice that, in OnChain BMMRs, Simon could reuse the same h\* all he wanted, because only one OnChain BMMR could be included per main:block per sidechain. However, on the LN no such rule can be enforced, as the goal is to push everything off-chain and include *zero* txns. So, we will never know what the Requests were, or how many had an effect on anything. -Notice that, in M8_V1, Simon could reuse the same h\* all he wanted, because only one M8_V1 could be included per main:block per sidechain. However, on the LN no such rule can be enforced, as the goal is to push everything off-chain and include *zero* M8s. So, we will never know what the M8s were or how many had an effect on anything. +Therefore, Simon will need to ensure that he '''gives each Mary a different h\*'''. Simon can easily do this, as he controls the side:block's contents and can simply increment a side:nonce -- this changes the side:block, and changes its hash (ie, changes h\*). -Therefore, Simon will need to ensure that he '''gives each Mary a different h\*'''. Simon can easily do this, as he controls the side:block's contents and can simply increment a nonce -- this changes the side:block, and changes its hash (ie, changes h\*). - -With a unique h\* per Mary, and at most 1 h\* making it into a block (per sidechain), we can guarantee that only one of the M8_V2's critical data can be committed to in a single main:block. By giving each miner (who Simon has a payment channel open with) a different h*, Simon can figure out which miner followed through with the commit, and know that only one such commit went through. Furthermore, if this Simon's requested critical data is not found in a block, none of the M8_V2 payments will be spendable by the Mary(s), because none of the h\* in question have ever made it into D3 (which is always on-chain) and no blocks atop will be accumulated. +With a unique h\* per Mary (or, more precisely, per channel), and at most 1 h\* making it into a block (per sidechain), Simon can ensure that he is charged, at most, one time. That's probably confusing, so here is an example, in which: Simon starts with 13 BTC, Mary starts with 40 BTC, the side:block's tx-fees currently total 7.1 BTC, and Simon is keeping 0.1 BTC for himself and paying 7 BTC to Mary. @@ -262,7 +138,7 @@ We start with (I):
-And both parties move, from there to "M8_V2" (II): +And both parties move, from there to (II):
     Simon 13 in, Mary 40 in ; 53 in total
@@ -289,10 +165,28 @@ From here, if the h\* side:block in question is BMMed, they can proceed to (III)
             6 ; to Simon
 
-Although, if Simon proceeds immediately, he removes the protection of the 'ratchet'. Ie, Simon removes Mary's incentive to care about blocks being built on this side:block. If Simon's side:block is orphaned, he loses his 7 BTC. Simon can either play it safe, and wait the full 100 side:blocks before moving on (ie, moving on to the third LN txn, above); or else Simon can take the risk if he feels comfortable with it. +If Simon proceeds immediately, he removes Mary's incentive to care about blocks being built on this side:block. If Simon's side:block is orphaned, he loses his 7 BTC. Simon can either play it safe, and wait for (for example) 100 side:blocks before moving on (ie, before moving on to the third LN txn, above); or else Simon can take the risk if he feels comfortable with it. If the h\* side:block is not found, then (II) and (III) are basically equivalent to each other. Simon and Mary could jointly reconstruct (I) and go back there, or they could proceed to a new version of II (with a different h\*, trying again with new side:block in the next main:block). +Now that we have described Requests, we can describe how they are accepted. + +=== BMM Accept === + +For each BMM Request that a main:miner "accepts", main:miners must place an OP Return ouput into their main:coinbase txn. (We've changed the tx-standardness policy to allow multiple OP_RETURNs.) + +The following data is required in the "accept" OP_RETURN output: + 1-byte - OP_RETURN (0x6a) + 1-byte - Push the following 36 bytes (0x24) + 4-bytes - Message header (0xD3407053) + 32-bytes - h* + ~5-bytes - BMM identifier bytes + + +[https://github.com/DriveNetTESTDRIVE/DriveNet/blob/564516653c1d876429382971a011f5f6119f7eb4/src/validation.cpp#L3377-L3470 Link to code]. + +If these OP_RETURN outputs are not present, then no BMM Requests have been accepted. (And, if they are not accepted, then they cannot be included in a main:block.) + ==Backward compatibility== @@ -310,14 +204,14 @@ This BIP will be deployed by "version bits" BIP9 with the name "blindmm" and usi
 // Deployment of Drivechains (BIPX, BIPY)
 consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].bit = 4;
-consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].nStartTime = 1515974401; // January 15th, 2018.
-consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].nTimeout = 1547510401; // January 15th, 2019.
+consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].nStartTime = 1579072881; // January 15th, 2020.
+consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].nTimeout = 1610695281; // January 15th, 2021.
 
==Reference Implementation== -See: https://github.com/drivechain-project/bitcoin/tree/mainchainBMM +See: https://github.com/DriveNetTESTDRIVE/DriveNet Also, for interest, see an example sidechain here: https://github.com/drivechain-project/bitcoin/tree/sidechainBMM @@ -339,3 +233,4 @@ Thanks to everyone who contributed to the discussion, especially: ZmnSCPxj, Adam ==Copyright== This BIP is licensed under the BSD 2-clause license. + -- cgit v1.2.3 From 2d7093ba7682d6834cda1e8bf79db8ce1794bf37 Mon Sep 17 00:00:00 2001 From: Paul Sztorc Date: Fri, 5 Apr 2019 10:02:24 -0700 Subject: spellcheck --- bip-blind-merged-mining.mediawiki | 18 ++++++++---------- 1 file changed, 8 insertions(+), 10 deletions(-) diff --git a/bip-blind-merged-mining.mediawiki b/bip-blind-merged-mining.mediawiki index ffa883a..7f29f06 100644 --- a/bip-blind-merged-mining.mediawiki +++ b/bip-blind-merged-mining.mediawiki @@ -1,4 +1,3 @@ -
 
     BIP: 301
@@ -49,7 +48,7 @@ To buy the right to find a sidechain block, users broadcast BMM Requests.
 
 Here, these can take two forms. The first does not require the Lightning Network, but it does have new requirements for Immediate Expiration (see below). The second inherits Immediate Expiration from the Lightning Network itself, but requires extra preparation and a different/larger message.
 
-Both forms require that certain Critical Data will be committed to within the coinbase of the block that the transaction is included in (see BMM Accept). For the OnChain (non-Lightning) version, we have created a new extended serialization transaction type (very similar to how segwit handles witness data (the witness stack)).
+Both forms require that certain Critical Data will be committed to within the coinbase of the block that the transaction is included in (see BMM Accept). For the OnChain (non-Lightning) version, we have created a new extended serialization transaction type (very similar to how SegWit handles witness data (the witness stack)).
 
 ==== Immediate Expiration ("Fill-or-Kill") ====
 
@@ -78,7 +77,7 @@ sideHeaderHash comes from side:chain (side:nodes build side:blocks/headers). The
 
 prevBlockRef, is a little more complicated (next section). 
 
-To qualify for inclusion in a block, BMM requests are subject to the following reqirements:
+To qualify for inclusion in a block, BMM requests are subject to the following requirements:
 
 # Requests must match a corresponding "BMM Accept" (see last section of BIP).
 # At most, only one Request is allowed in a main:block, per sidechain. In other words, if 700 users broadcast BMM Requests for sidechain #4, then the main:miner must choose one single Request to include.
@@ -94,7 +93,7 @@ Above: Three blockchains, with different max length (small number), reorganizati
 
 ===== Extended Serialization =====
 
-To impose new requirements at the transaction level, we borrow the dummy vin & "flag" trick from SegWit style transactions. Unless all of the requirements for sidechain critical data transactions are met by the block it is included in, the transaction is invalid. With SegWit, this extra data is the segwit signature stack, and the extra requirements are the signatures' locations and validity. In the sidechain BMM critical data transactions, the extra data is the (nSidechain, h\*) pair, which must meet the first two requirements (above) as well as the main:blocknumber, which must meet the third requirement (above).
+To impose new requirements at the transaction level, we borrow the dummy vin & "flag" trick from SegWit style transactions. Unless all of the requirements for sidechain critical data transactions are met by the block it is included in, the transaction is invalid. With SegWit, this extra data is the SegWit signature stack, and the extra requirements are the signatures' locations and validity. In the sidechain BMM critical data transactions, the extra data is the (nSidechain, h\*) pair, which must meet the first two requirements (above) as well as the main:blocknumber, which must meet the third requirement (above).
 
 
 
@@ -110,11 +109,11 @@ Lightning BMMRs require Simons to have a LN-channel pathways open with Marys. Th
 
 LN txns cannot make use of prevSideBlockRef, as no one knows for sure when (or if) they will be broadcast on-chain. Instead, they must use prevSideBlockHash. But they otherwise require the same data:
 
-
	
-    4-bytes - Message header (0xD0520C6E)	
+
   
+    4-bytes - Message header (0xD0520C6E)   
     1-byte - sidechain number
-    32-bytes  - h* side:block hash	
-    32-bytes  - prevSideBlockHash	
+    32-bytes  - h* side:block hash  
+    32-bytes  - prevSideBlockHash   
 
Notice that, in OnChain BMMRs, Simon could reuse the same h\* all he wanted, because only one OnChain BMMR could be included per main:block per sidechain. However, on the LN no such rule can be enforced, as the goal is to push everything off-chain and include *zero* txns. So, we will never know what the Requests were, or how many had an effect on anything. @@ -173,7 +172,7 @@ Now that we have described Requests, we can describe how they are accepted. === BMM Accept === -For each BMM Request that a main:miner "accepts", main:miners must place an OP Return ouput into their main:coinbase txn. (We've changed the tx-standardness policy to allow multiple OP_RETURNs.) +For each BMM Request that a main:miner "accepts", main:miners must place an OP Return output into their main:coinbase txn. (We've changed the tx-standardness policy to allow multiple OP_RETURNs.) The following data is required in the "accept" OP_RETURN output: 1-byte - OP_RETURN (0x6a) @@ -233,4 +232,3 @@ Thanks to everyone who contributed to the discussion, especially: ZmnSCPxj, Adam ==Copyright== This BIP is licensed under the BSD 2-clause license. - -- cgit v1.2.3 From 18f43a6b2a8423850c2cac617fa3053faf420007 Mon Sep 17 00:00:00 2001 From: Gigi <109058+dergigi@users.noreply.github.com> Date: Tue, 9 Apr 2019 22:37:48 -0500 Subject: transaction -> transactions --- bip-0016.mediawiki | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bip-0016.mediawiki b/bip-0016.mediawiki index ba828e1..0f4fb81 100644 --- a/bip-0016.mediawiki +++ b/bip-0016.mediawiki @@ -40,7 +40,7 @@ The rules for validating these outpoints when relaying transactions or consideri # Normal validation is done: an initial stack is created from the signatures and {serialized script}, and the hash of the script is computed and validation fails immediately if it does not match the hash in the outpoint. # {serialized script} is popped off the initial stack, and the transaction is validated again using the popped stack and the deserialized script as the scriptPubKey. -These new rules should only be applied when validating transactions in blocks with timestamps >= 1333238400 (Apr 1 2012) [https://github.com/bitcoin/bitcoin/commit/8f188ece3c82c4cf5d52a3363e7643c23169c0ff Remove -bip16 and -paytoscripthashtime command-line arguments]. There are transaction earlier than 1333238400 in the block chain that fail these new validation rules. [http://blockexplorer.com/tx/6a26d2ecb67f27d1fa5524763b49029d7106e91e3cc05743073461a719776192 Transaction 6a26d2ecb67f27d1fa5524763b49029d7106e91e3cc05743073461a719776192]. Older transactions must be validated under the old rules. (see the Backwards Compatibility section for details). +These new rules should only be applied when validating transactions in blocks with timestamps >= 1333238400 (Apr 1 2012) [https://github.com/bitcoin/bitcoin/commit/8f188ece3c82c4cf5d52a3363e7643c23169c0ff Remove -bip16 and -paytoscripthashtime command-line arguments]. There are transactions earlier than 1333238400 in the block chain that fail these new validation rules. [http://blockexplorer.com/tx/6a26d2ecb67f27d1fa5524763b49029d7106e91e3cc05743073461a719776192 Transaction 6a26d2ecb67f27d1fa5524763b49029d7106e91e3cc05743073461a719776192]. Older transactions must be validated under the old rules. (see the Backwards Compatibility section for details). For example, the scriptPubKey and corresponding scriptSig for a one-signature-required transaction is: -- cgit v1.2.3 From 14834fa63c9de42140d1d70ed6dab0837a202186 Mon Sep 17 00:00:00 2001 From: Gigi <109058+dergigi@users.noreply.github.com> Date: Tue, 9 Apr 2019 22:41:52 -0500 Subject: is alive any working -> is alive and working --- bip-0002.mediawiki | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bip-0002.mediawiki b/bip-0002.mediawiki index b4567c4..3bf5aec 100644 --- a/bip-0002.mediawiki +++ b/bip-0002.mediawiki @@ -240,7 +240,7 @@ What if a single merchant wishes to block a hard-fork? How about a small number of merchants (maybe only two) who sell products to each other? -* In this scenario, it would seem the previous Bitcoin is alive any working, and that the hard-fork has failed. How to resolve such a split is outside the scope of this BIP. +* In this scenario, it would seem the previous Bitcoin is alive and working, and that the hard-fork has failed. How to resolve such a split is outside the scope of this BIP. How can economic agreement veto a soft-fork? -- cgit v1.2.3 From d69e368ce3ef402e57f34ed40bf61508355ffa9d Mon Sep 17 00:00:00 2001 From: Paul Sztorc Date: Wed, 17 Apr 2019 16:59:40 -0700 Subject: typo the critical txn should start with "03", as it has version number 3 --- bip-blind-merged-mining/witness-vs-critical.png | Bin 265603 -> 268309 bytes 1 file changed, 0 insertions(+), 0 deletions(-) diff --git a/bip-blind-merged-mining/witness-vs-critical.png b/bip-blind-merged-mining/witness-vs-critical.png index 77003e8..79c84b1 100644 Binary files a/bip-blind-merged-mining/witness-vs-critical.png and b/bip-blind-merged-mining/witness-vs-critical.png differ -- cgit v1.2.3 From 0cb0d361a57f90b3cad39527347360cb206bf9e3 Mon Sep 17 00:00:00 2001 From: Gigi <109058+dergigi@users.noreply.github.com> Date: Sun, 21 Apr 2019 19:17:41 -0500 Subject: [Trivial] Fix word repetition * changes modifies -> modifies * index of of -> index of --- bip-0069.mediawiki | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/bip-0069.mediawiki b/bip-0069.mediawiki index e9f9245..3aa9463 100644 --- a/bip-0069.mediawiki +++ b/bip-0069.mediawiki @@ -78,7 +78,7 @@ N.B. All comparisons do not need to operate in constant time since they are not ===Transaction Inputs=== -Transaction inputs are defined by the hash of a previous transaction, the output index of of a UTXO from that previous transaction, the size of an unlocking script, the unlocking script, and a sequence number. [3] +Transaction inputs are defined by the hash of a previous transaction, the output index of a UTXO from that previous transaction, the size of an unlocking script, the unlocking script, and a sequence number. [3] For sorting inputs, the hash of the previous transaction and the output index within that transaction are sufficient for sorting purposes; each transaction hash has an extremely high probability of being unique in the blockchain β€” this is enforced for coinbase transactions by BIP30 β€” and output indices within a transaction are unique. For the sake of efficiency, transaction hashes should be compared first before output indices, since output indices from different transactions are often equivalent, while all bytes of the transaction hash are effectively random variables. @@ -87,7 +87,7 @@ In the event of two matching transaction hashes, the respective previous output If the previous output indices match, the inputs are considered equal. Transaction malleability will not negatively impact the correctness of this process. -Even if a wallet client follows this process using unconfirmed UTXOs as inputs and an attacker changes modifies the blockchain’s record of the hash of the previous transaction, the wallet client will include the invalidated previous transaction hash in its input data, and will still correctly sort with respect to that invalidated hash. +Even if a wallet client follows this process using unconfirmed UTXOs as inputs and an attacker modifies the blockchain’s record of the hash of the previous transaction, the wallet client will include the invalidated previous transaction hash in its input data, and will still correctly sort with respect to that invalidated hash. ===Transaction Outputs=== -- cgit v1.2.3 From 49d44b9b287f164c5b55718abe90620ddb8778d3 Mon Sep 17 00:00:00 2001 From: Torkel Rogstad Date: Tue, 30 Apr 2019 10:51:43 +0200 Subject: Update bip-0049.mediawiki Fix broken internal link --- bip-0049.mediawiki | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bip-0049.mediawiki b/bip-0049.mediawiki index 74645a1..4551b31 100644 --- a/bip-0049.mediawiki +++ b/bip-0049.mediawiki @@ -68,7 +68,7 @@ To derive the P2SH address from the above calculated public key, we use the enca ==Backwards Compatibility== -This BIP is not backwards compatible by design as described under [#considerations]. An incompatible wallet will not discover accounts at all and the user will notice that something is wrong. +This BIP is not backwards compatible by design as described under [[#considerations|considerations]]. An incompatible wallet will not discover accounts at all and the user will notice that something is wrong. ==Test vectors== -- cgit v1.2.3 From d8dea4ef095e5ce8a12f7956a9951cb5cddf8964 Mon Sep 17 00:00:00 2001 From: MarcoFalke Date: Mon, 13 May 2019 18:24:52 -0400 Subject: BIP 144: Old serialization format must be used on empty witness --- bip-0144.mediawiki | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bip-0144.mediawiki b/bip-0144.mediawiki index 75d8a1b..8ec2191 100644 --- a/bip-0144.mediawiki +++ b/bip-0144.mediawiki @@ -79,7 +79,7 @@ The serialization has the following structure: Parsers supporting this BIP will be able to distinguish between the old serialization format (without the witness) and this one. The marker byte is set to zero so that this structure will never parse as a valid transaction in a parser that does not support this BIP. If parsing were to succeed, such a transaction would contain no inputs and a single output. -If the witness is empty, the old serialization format should be used. +If the witness is empty, the old serialization format must be used. Currently, the only witness objects type supported are script witnesses which consist of a stack of byte arrays. It is encoded as a var_int item count followed by each item encoded as a var_int length followed by a string of bytes. Each txin has its own script witness. The number of script witnesses is not explicitly encoded as it is implied by txin_count. Empty script witnesses are encoded as a zero byte. The order of the script witnesses follows the same order as the associated txins. -- cgit v1.2.3 From 19d3b9dc829ca52133dd71497f0768118d7462d2 Mon Sep 17 00:00:00 2001 From: Andrew Chow Date: Mon, 13 May 2019 22:00:26 -0400 Subject: bip174: add global xpub field --- bip-0174.mediawiki | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/bip-0174.mediawiki b/bip-0174.mediawiki index f197728..8c9443e 100644 --- a/bip-0174.mediawiki +++ b/bip-0174.mediawiki @@ -118,6 +118,12 @@ The currently defined global types are as follows: *** {transaction} ** Note: Every PSBT must have a field with this type. +* Type: Extended Public Key PSBT_GLOBAL_XPUB = 0x01 +** Key: The type followed by the 78 byte serialized extended public key as defined by BIP 32. Extended public keys are those that can be used to derive public keys used in the inputs and outputs of this transaction. It should be the public key at the highest hardened derivation index so that the unhardened child keys used in the transaction can be derived. +*** {0x01}|{xpub} +** Value: The master key fingerprint as defined by BIP 32 concatenated with the derivation path of the public key. The derivation path is represented as 32 bit unsigned integer indexes concatenated with each other. The number of 32 bit unsigned integer indexes must match the depth provided in the extended public key. +*** {master key fingerprint}|{32-bit int}|...|{32-bit int} + The currently defined per-input types are defined as follows: * Type: Non-Witness UTXO PSBT_IN_NON_WITNESS_UTXO = 0x00 -- cgit v1.2.3 From e1f770e23617ddb5d9d78597c7377fd6f956f67d Mon Sep 17 00:00:00 2001 From: Andrew Chow Date: Sun, 9 Jun 2019 10:44:32 +0200 Subject: bip174: add section describing change detection --- bip-0174.mediawiki | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/bip-0174.mediawiki b/bip-0174.mediawiki index 8c9443e..1dc3b47 100644 --- a/bip-0174.mediawiki +++ b/bip-0174.mediawiki @@ -364,6 +364,23 @@ for input,i in enumerate(psbt.inputs): assert False
+====Change Detection==== + +Signers may wish to display the inputs and outputs to users for extra verification. +In such displays, signers may wish to identify which outputs are change outputs in order to omit them to avoid additional user confusion. +In order to detect change, a signer can use the BIP 32 derivation paths provided in inputs and outputs as well as the extended public keys provided globally. + +For a single key output, a signer can observe whether the master fingerprint for the public key for that output belongs to itself. +If it does, it can then derive the public key at the specified derivation path and check whether that key is the one present in that output. + +For outputs involving multiple keys, a signer can first examine the inputs that it is signing. +It should determine the general pattern of the script and internally produce a representation of the policy that the script represents. +Such a policy can include things like how many keys are present, what order they are in, how many signers are necessary, which signers are required, etc. +The signer can then use the BIP 32 derivation paths for each of the pubkeys to find which global extended public key is the one that can derive that particular public key. +To do so, the signer would extract the derivation path to the highest hardened index and use that to lookup the public key with that index and master fingerprint. +The signer would construct this script policy with extended public keys for all of the inputs and outputs. +Change outputs would then be identified as being the outputs which have the same script policy as the inputs that are being signed. + ===Combiner=== The Combiner can accept 1 or many PSBTs. -- cgit v1.2.3 From e78c384755f2bd6bf233d585a0ffd70b57eb43db Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=92=D0=B5=D0=BB=D0=B5=D1=81=D0=BB=D0=B0=D0=B2?= Date: Wed, 12 Sep 2018 20:17:33 +1000 Subject: BIP-136: Bech32 Encoded Tx Position References --- README.mediawiki | 7 ++ bip-0136.mediawiki | 276 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 283 insertions(+) create mode 100644 bip-0136.mediawiki diff --git a/README.mediawiki b/README.mediawiki index 2e28c5f..151e25c 100644 --- a/README.mediawiki +++ b/README.mediawiki @@ -658,6 +658,13 @@ Those proposing changes should consider that ultimately consent may rest with th | Sancho Panza | Informational | Draft +|- +| [[bip-0136.mediawiki|136]] +| Applications +| Bech32 Encoded Tx Position References +| ВСлСслав, Jonas Schnelli +| Informational +| Draft |- style="background-color: #cfffcf" | [[bip-0137.mediawiki|137]] | Applications diff --git a/bip-0136.mediawiki b/bip-0136.mediawiki new file mode 100644 index 0000000..e8b107d --- /dev/null +++ b/bip-0136.mediawiki @@ -0,0 +1,276 @@ +
+  BIP: 136
+  Layer: Applications
+  Title: Bech32 Encoded Tx Position References
+  Author: ВСлСслав 
+          Jonas Schnelli 
+  Comments-Summary: No comments yet.
+  Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0136
+  Status: Draft
+  Type: Informational
+  Created: 2017-07-09
+  License: BSD-2-Clause
+
+ +== Introduction == + +=== Abstract === + +This document proposes a human useable format: '''"TxRef"''' as a standard way of referencing transaction positions within the Bitcoin Blockchain. The primary purpose of this format is to allow end-users an effective and convenient way of referencing transactions that have been confirmed within the blockchain. + +''Please note: that a TxRef does not reference an actual transaction itself, rather it references a possible location within a blockchain for a transaction, that may or may not, point to an actual transaction and may change with reorganisations. We recommend that TxRef's should be not used for positions within the blockchain with a maturity less than 100 blocks.'' + +=== Copyright === + +This BIP is licensed under the 2-clause BSD license. + +=== Motivation === +Since the first version of Bitcoin, TxID's (Transaction Identifiers) that are a core part of the consensus protocol, have been routinely used to identify individual transactions between users. + +However, for many use-cases they have practical limitations: +* TxIDs are expensive for full nodes to lookup (requiring either a linear scan of the blockchain, or an expensive TxID index). +* TxIDs require third-party service for SPV wallets to lookup. +* TxIDs are very long HEX encoded values (64 characters long). + +For transactions that have been embedded in the blockchain, it is possible to reference them not by their TxID, but by their location within the blockchain itself. In this document, we propose a standard for doing this. + +=== Examples === +These examples are for Bitcoin Transactions. +* Genesis Coinbase Transaction: tx1:rqqq-qqqq-qmhu-qk (need to update) +* Transaction #2205 of Block #466793: tx1:rjk0-u5ng-4jsf-mc (need to update) + +== Specification == + +A '''confirmed transaction position reference''', or '''TxRef''' is reference to a particular location within the blockchain specified by the block height and a index within the block. + +''Please Note: All values in this specification are encoded in little-endian format.'' + +=== Transaction Position Reference Considerations === +A TxRef may reference a location that doesn't exist because: + +* The block height hasn't been mined. Or, +* The index is more than the total number of transactions included within the specified block. + +Therefore, implementers must be careful not to display TxRef's to users prematurely: + +* Applications MUST NOT display TxRef's for transactions with less than 6 confirmations. +* Application MUST show a warning for TxRef's for transactions with less than 100 confirmations. +** This warning SHOULD state that in the case of a large reorganisation, the TxRefs Displayed may point to a different transaction, or to no transaction at all. + +=== Encoding === + +TxRef uses standard Bech32'''Why use Bech32 Encoding for Confirmed Transaction References?''' The error detection and correction properties of this encoding format make it very attractive. We expect that it will be reasonable for software to correct a maximum of two characters; however, we haven’t specified this yet. encoding as defined in [https://github.com/bitcoin/bips/blob/master/bip-0173.mediawiki BIP-173] and therefore consists of: + +* Human-readable Part, or "HRP", that provides namespacing. We have chosen to distinguish between Main and Test Networks: +**For Any Mainnet Network: '''"tx"'''. +**For Any Testnet Network: '''"txtest"'''. +**Please see [https://github.com/satoshilabs/slips/blob/master/slip-0173.md SLIP-0173 : Registered human-readable parts for BIP-0173] for a full list of HRP's including these two and other relating to other projects. +* Separator: '''"1"'''. +* Data Part. + +To increase portability and readability additional separators SHOULD be added: + +* A Colon'''Why add a colon here?''' This allows it to conform better with W3C URN/URL standards. '''":"''' added after '1'. +* Hyphens'''Why hyphens to the TxRef?''' As TxRef's are short, we expect that they will be quoted via voice or written by hand. The inclusion of hyphens every 4 characters breaks the string and means people don't loose their place so easily. '''"-"''' added after every 4 characters beyond the colon. + +All non-bech32-alphabet characters after the bech32 code separator MUST be ignored/removed when parsing (except for terminating characters).'''Why strip all non-bech32-alphabet characters?''' We do not wish to expect the users to keep their TxRef's in good unicode form (hyphens, colons, invisible spaces, random unicode characters, etc). We expect them to copy, paste, write by-hand, write in a mix of character sets, etc. Parsers should automatically correct for all sorts of these common errors. + +{| class="wikitable" +|+TxRef String Format for Bitcoin Mainnet and Bitcoin Testnet: +! +!Bit +!Character +!Characters +!Value +|- +|Human Readable Part +| +|1 – 2 +|2 +|Bitcoin Mainnet: "'''tx'''", Bitcoin Testnet: "'''txtest'''" +|- +|Separator +| +|3 +|1 +|"'''1'''" +|- +|Colon +| +|4 +|1 +|"''':'''" +|- +|Data +|0 – 19 +|5 – 8 +|4 +| +|- +|Hyphen +| +|9 +|1 +|"'''-'''" +|- +|Data +|20 – 39 +|10 – 13 +|4 +| +|- +|Hyphen +| +|14 +|1 +|"'''-'''" +|- +|Data +|40 – 59 +|15 – 18 +|4 +| +|- +|Hyphen +| +|19 +|1 +|"'''-'''" +|- +|Data +|60 – 74 +|20 – 22 +|3 +| +|} + +=== Data Part === +{| class="wikitable" +|+TxRef Binary Format for Bitcoin Mainnet and Bitcoin Testnet: +! +!'''Bit''' +!'''Bit(s)''' +!'''Type''' +!'''Values''' +!'''Notes''' +|- +|Magic +|0 – 4 +|5 +|Chain Namespacing Code +|'''0x3''' for Bitcoin Mainnet, or '''0x6''' for Bitcoin Testnet +| +|- +|Version +|5 +|1 +|For Future Use +|Must be '''0x0''' +| +|- +|Block Height +|6 – 29 +|24 +|The Block Height of the Tx +|Block 0 (genesis) to block 16777215 +|Until Year ~2328 +|- +|Transaction Index +|30 – 44 +|15 +|The index of the Tx inside the block +|Tx 0 (coinbase) to Tx position 32767 +|Max Tx's in block is 16665 +|- +|Checksum +|45 – 74 +|30 +|Bech32 Checksum +| +| +|} +==== Magic Notes: ==== +The magic code provides namespacing between chains. 5-bit magic codes are used for the Bitcoin Mainnet and the Bitcoin Testnet. (it may be significantly longer for other projects/chains): + +* For Bitcoin Mainnet the magic code is: '''0x3''', leading to a '''"r"''' character. +* For Bitcoin Testnet the magic code is: '''0x6''', leading to a '''"x"''' character. + +Codes '''0x0''', '''0x1''', '''0x2''' are also reserved for future use within the Bitcoin project. + +''Any other chain MUST NOT start their magic code with any value between 0x0 and 0x6 inclusive.'' + +Other magic codes will be specified in SLIP-XXXX "TxRef for Non-Bitcoin Chains and Networks". + +=== Compatibly === +There are no known compatibility issues. + +== Rationale == + + + +== Reference implementations == +C Reference Implementation: https://github.com/jonasschnelli/bitcoin_txref_code + +Go Reference Implementation: https://github.com/kulpreet/txref + +== Appendices == + +=== Test Vectors === +There are two sets of Test Vectors included here: + +* Bech32 Encoding Test Vectors. These are to test if a implementation accepts the encoding, with the correct human readable part, and separator. +* Bitcoin TxRef Test Vectors. These test the full specification, in particular correct values for block height and the transaction index. + +==== Bech32 Encoding (for TxRef). ==== +''Please Note: All test vectors are shown to help test if a string is compliant or not. All real-life applications (such as for Bitcoin) should comply with the Bitcoin Test Vectors listed Below.'' + +The following strings have a valid Human Readable Part and Bech32 Checksum. +* TX1A12UEL5L +* tx1an83characterlonghumanreadablepartthatcontainsthenumber1andtheexcludedcharactersbio1tt5tgs +* tx1abcdef1qpzry9x8gf2tvdw0s3jn54khce6mua7lmqqqxw +* tx11qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqc8247j + +The following list gives invalid TxRef's and the reason for their invalidity. +* bc1qw508d6qejxtdg4y5r3zarvary0c5xw7kg3g4ty: Invalid human-readable part +* tx1qw508d6qejxtdg4y5r3zarvary0c5xw7kv8f3t5: Invalid checksum + +==== Bitcoin TxRef (out of date, need to be amended). ==== +The following list gives properly encoded Bitcoin TxRef's and the values in hex. (block height, transaction index) + +* tx1:rqqq-qqqq-qmhu-qk: (0x0, 0x0) +* tx1:rqqq-qull-6v87-r7: (0x0, 0x1FFF) +* tx1:r7ll-lrqq-vq5e-gg: (0x1FFFFF, 0x0) +* tx1:r7ll-llll-khym-tq: (0x1FFFFF, 0x1FFF) + +The following list give valid Bitcoin TxRef's and the values in hex. (block height, transaction index) +* tx1:rjk0-u5ng-4jsf-mc: (0x71F69, 0x89D) +* TX1RJK0U5NG4JSFMC: (0x71F69, 0x89D) +* TX1R1JK0--U5bNG4JSb----FMC: (0x71F69, 0x89D) +* tx1 rjk0 u5ng 4jsfmc: (0x71F69, 0x89D) +* tx1!rjk0\u5ng*4jsf^^mc: (0x71F69, 0x89D) + +The following list gives invalid Bitcoin TxRef's and the reason for their invalidity. +* tx1:t7ll-llll-gey7-ez: Magic 0xB instead of 0x3. (0x1FFFFF, 0x1FFF) +* tx1:rlll-llll-cgqu-n2: Version 1 instead of 0. (0x1FFFFF, 0x1FFF) +* tx1:rjk0-u5ng-gghq-fkg7: Valid Bech32, but 10x5bit packages instead of 8. +* tx1:rjk0-u5qd-s43z: Valid Bech32, but 6x5bit packages instead of 8. + +=== Bitcoin TxRef Payload Value Choice: === +Some calculations showing why we chose these particular bit-length of the block height and transaction index. + +==== Block Height Value: ==== +24-bit: between 0, and 0xFFFFFF (16,777,216 blocks). + +*There are ~52,500 blocks every year, leading to ~319 years of blocks addressable. +*Therefore before year 2328 this specification should be extended. (We think that we have plenty of time). + +==== Tx Position Value: ==== +15-bit: between 0x0, and 0x7FFF. (32,768 transactions). + +*The ''realistic'' smallest Tx is 83 Bytes: Max 12047 tx in a block. +**4B version + 1B tx_in count + 36B previous_output + 1B script length + 0B signature script + 4B sequence + 1B tx_out count + 8B amount + 1B script length + 23B pubkey script + 4B lock_time = 83B +*The ''extreme'' smallest Tx is 60 Byte's: Max 16665 tx in a block. +**4B version + 1B tx_in count + 36B previous_output + 1B script length + 0B signature script + 4B sequence + 1B tx_out count + 8B amount + 1B script length + 0B pubkey script + 4B lock_time = 60B + +== Acknowledgements == +Special Thanks to Pieter Wuille and Greg Maxwell for Bech32, a wonderful user-facing data encoding. -- cgit v1.2.3 From a8bb0a4e39a9eefa419d2f83f40b6af97699016f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=92=D0=B5=D0=BB=D0=B5=D1=81=D0=BB=D0=B0=D0=B2?= Date: Tue, 6 Nov 2018 15:35:44 +0800 Subject: Include Optional Encoded Outpoints With thanks to Daniel Pape for the work behind this idea. Please not that the test-vectors still need to be updated (again). --- README.mediawiki | 2 +- bip-0136.mediawiki | 166 ++++++++++++++++++++++++++++++++--------------------- 2 files changed, 100 insertions(+), 68 deletions(-) diff --git a/README.mediawiki b/README.mediawiki index 151e25c..11ecbb1 100644 --- a/README.mediawiki +++ b/README.mediawiki @@ -662,7 +662,7 @@ Those proposing changes should consider that ultimately consent may rest with th | [[bip-0136.mediawiki|136]] | Applications | Bech32 Encoded Tx Position References -| ВСлСслав, Jonas Schnelli +| ВСлСслав, Jonas Schnelli, Daniel Pape | Informational | Draft |- style="background-color: #cfffcf" diff --git a/bip-0136.mediawiki b/bip-0136.mediawiki index e8b107d..15bc29a 100644 --- a/bip-0136.mediawiki +++ b/bip-0136.mediawiki @@ -4,6 +4,7 @@ Title: Bech32 Encoded Tx Position References Author: ВСлСслав Jonas Schnelli + Daniel Pape Comments-Summary: No comments yet. Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0136 Status: Draft @@ -15,10 +16,9 @@ == Introduction == === Abstract === +This document proposes a convenient human useable format, '''"TxRef"''', as a standard way to refer to a transaction position within the Bitcoin Blockchain, and optionally a particular outpoint index with the referred transaction. The primary purpose of this format is to allow users to refer to a confirmed transaction (and optionally an outpoint index within) in a standard reliable and concise way. -This document proposes a human useable format: '''"TxRef"''' as a standard way of referencing transaction positions within the Bitcoin Blockchain. The primary purpose of this format is to allow end-users an effective and convenient way of referencing transactions that have been confirmed within the blockchain. - -''Please note: that a TxRef does not reference an actual transaction itself, rather it references a possible location within a blockchain for a transaction, that may or may not, point to an actual transaction and may change with reorganisations. We recommend that TxRef's should be not used for positions within the blockchain with a maturity less than 100 blocks.'' +''Please note: Unlike TxID where there is strong cryptographic link between the ID and the actual transaction. TxRef only provide a weak link to a particular transaction. TxRef locates an offset within a blockchain for a transaction, that may - or may not - point to an actual transaction, which fact may change with reorganisations. We recommend that TxRef's should be not used for positions within the blockchain having a maturity less than 100 blocks.'' === Copyright === @@ -32,24 +32,25 @@ However, for many use-cases they have practical limitations: * TxIDs require third-party service for SPV wallets to lookup. * TxIDs are very long HEX encoded values (64 characters long). -For transactions that have been embedded in the blockchain, it is possible to reference them not by their TxID, but by their location within the blockchain itself. In this document, we propose a standard for doing this. +For transactions that have been embedded in the blockchain, it is possible to reference them not by their TxID, but by their location within the blockchain itself. The encoding can be made friendly for occasional human transcription. In this document, we propose a standard for doing this. === Examples === These examples are for Bitcoin Transactions. -* Genesis Coinbase Transaction: tx1:rqqq-qqqq-qmhu-qk (need to update) -* Transaction #2205 of Block #466793: tx1:rjk0-u5ng-4jsf-mc (need to update) +* Genesis Coinbase Transaction: tx1:rqqq-qqqq-qmhu-qhp +* Transaction #2205 of Block #466793: tx1:rjk0-uqay-zsrw-hqe == Specification == -A '''confirmed transaction position reference''', or '''TxRef''' is reference to a particular location within the blockchain specified by the block height and a index within the block. +A '''confirmed transaction position reference''', or '''TxRef''', is a reference to a particular location within the blockchain, specified by the block height and a transaction index within the block, and optionally a outpoint index within the transaction. ''Please Note: All values in this specification are encoded in little-endian format.'' === Transaction Position Reference Considerations === A TxRef may reference a location that doesn't exist because: -* The block height hasn't been mined. Or, -* The index is more than the total number of transactions included within the specified block. +* The specified block hasn't yet been mined. Or, +* The transaction index is greater than the total number of transactions included within the specified block. +* The optional outpoint index is greater than the total outpoints contained within the transaction. Therefore, implementers must be careful not to display TxRef's to users prematurely: @@ -62,21 +63,23 @@ Therefore, implementers must be careful not to display TxRef's to users prematur TxRef uses standard Bech32'''Why use Bech32 Encoding for Confirmed Transaction References?''' The error detection and correction properties of this encoding format make it very attractive. We expect that it will be reasonable for software to correct a maximum of two characters; however, we haven’t specified this yet. encoding as defined in [https://github.com/bitcoin/bips/blob/master/bip-0173.mediawiki BIP-173] and therefore consists of: * Human-readable Part, or "HRP", that provides namespacing. We have chosen to distinguish between Main and Test Networks: -**For Any Mainnet Network: '''"tx"'''. -**For Any Testnet Network: '''"txtest"'''. -**Please see [https://github.com/satoshilabs/slips/blob/master/slip-0173.md SLIP-0173 : Registered human-readable parts for BIP-0173] for a full list of HRP's including these two and other relating to other projects. +** For Any Mainnet Network: '''"tx"'''. +** For Any Testnet Network: '''"txtest"'''. +** Please see [https://github.com/satoshilabs/slips/blob/master/slip-0173.md SLIP-0173 : Registered human-readable parts for BIP-0173] for a full list of HRP's including these two and others relating to other projects. * Separator: '''"1"'''. * Data Part. +Please node: other specifcations, such as [https://w3c-ccg.github.io/did-spec/ Decentralized Identifier syntax] , have implicitly encoded the information contained within the HRP elsewhere. In this case they may choose not include the HRP as specified here. + To increase portability and readability additional separators SHOULD be added: * A Colon'''Why add a colon here?''' This allows it to conform better with W3C URN/URL standards. '''":"''' added after '1'. -* Hyphens'''Why hyphens to the TxRef?''' As TxRef's are short, we expect that they will be quoted via voice or written by hand. The inclusion of hyphens every 4 characters breaks the string and means people don't loose their place so easily. '''"-"''' added after every 4 characters beyond the colon. +* Hyphens'''Why hyphens to the TxRef?''' As TxRef's are short, we expect that they will be quoted via voice or written by hand. The inclusion of hyphens every 4 characters breaks the string and means people don't lose their place so easily. '''"-"''' added after every 4 characters beyond the colon. All non-bech32-alphabet characters after the bech32 code separator MUST be ignored/removed when parsing (except for terminating characters).'''Why strip all non-bech32-alphabet characters?''' We do not wish to expect the users to keep their TxRef's in good unicode form (hyphens, colons, invisible spaces, random unicode characters, etc). We expect them to copy, paste, write by-hand, write in a mix of character sets, etc. Parsers should automatically correct for all sorts of these common errors. {| class="wikitable" -|+TxRef String Format for Bitcoin Mainnet and Bitcoin Testnet: +|+Text Encoding of the TxRef ! !Bit !Character @@ -112,39 +115,12 @@ All non-bech32-alphabet characters after the bech32 code separator MUST be ignor |9 |1 |"'''-'''" -|- -|Data -|20 – 39 -|10 – 13 -|4 -| -|- -|Hyphen -| -|14 -|1 -|"'''-'''" -|- -|Data -|40 – 59 -|15 – 18 -|4 -| -|- -|Hyphen -| -|19 -|1 -|"'''-'''" -|- -|Data -|60 – 74 -|20 – 22 -|3 -| |} +The Data - Hyphen patten is repeated for the entire length of data, ( a hyphen is inserted after every encoded 20 bits or 4 data characters). +=== Data === + +The 75 or 90 bits of data encoded in the string above are defined in this manner: -=== Data Part === {| class="wikitable" |+TxRef Binary Format for Bitcoin Mainnet and Bitcoin Testnet: ! @@ -158,7 +134,10 @@ All non-bech32-alphabet characters after the bech32 code separator MUST be ignor |0 – 4 |5 |Chain Namespacing Code -|'''0x3''' for Bitcoin Mainnet, or '''0x6''' for Bitcoin Testnet +|'''0x3''' for Bitcoin Mainnet. +'''0x4''' for Bitcoin Mainnet with Outpoint. +'''0x6''' for Bitcoin Testnet. +'''0x7''' for Bitcoin Testnet with Outpoint. | |- |Version @@ -181,27 +160,59 @@ All non-bech32-alphabet characters after the bech32 code separator MUST be ignor |The index of the Tx inside the block |Tx 0 (coinbase) to Tx position 32767 |Max Tx's in block is 16665 +|} +If the magic is '''0x4''' or '''0x7''', an optional outpoint is included in the encoding: + +{| class="wikitable" +|+Optional Outpoint Index Encoding: +! +!'''Bit''' +!'''Bit(s)''' +!'''Type''' +!'''Values''' +!'''Notes''' +|- +|Outpoint Index +|45 – 59 +|15 +|The index of the Outpoint inside the Tx +|Outpoint 0 to Outpoint Position 32767 +| +|} + +We include the 30-bit checksum last: +{| class="wikitable" +|+Bech32 Checksum Encoding: +! +!'''Bit''' +!'''Bit(s)''' +!'''Type''' +!'''Values''' +!'''Notes''' |- |Checksum -|45 – 74 +|45 – 74 or 60 – 89 |30 |Bech32 Checksum | | |} + ==== Magic Notes: ==== The magic code provides namespacing between chains. 5-bit magic codes are used for the Bitcoin Mainnet and the Bitcoin Testnet. (it may be significantly longer for other projects/chains): -* For Bitcoin Mainnet the magic code is: '''0x3''', leading to a '''"r"''' character. -* For Bitcoin Testnet the magic code is: '''0x6''', leading to a '''"x"''' character. +* For Bitcoin Mainnet the magic code is: '''0x3''', leading to an '''"r"''' character when encoded. +* For Bitcoin Mainnet with Outpoint Encoded the magic code is: '''0x4''', leading to an '''"y"''' character when encoded. +* For Bitcoin Testnet the magic code is: '''0x6''', leading to an '''"x"''' character when encoded. +* For Bitcoin Testnet with Outpoint Encoded the magic code is: '''0x7''', leading to an '''"8"''' character when encoded. -Codes '''0x0''', '''0x1''', '''0x2''' are also reserved for future use within the Bitcoin project. +Codes '''0x0''', '''0x1''', '''0x2''', '''0x5''', are also reserved for future use within the Bitcoin project. -''Any other chain MUST NOT start their magic code with any value between 0x0 and 0x6 inclusive.'' +''Any other chain MUST NOT start their magic code with any value between 0x0 and 0x7 inclusive.'' Other magic codes will be specified in SLIP-XXXX "TxRef for Non-Bitcoin Chains and Networks". -=== Compatibly === +=== Compatibility === There are no known compatibility issues. == Rationale == @@ -209,9 +220,11 @@ There are no known compatibility issues. == Reference implementations == -C Reference Implementation: https://github.com/jonasschnelli/bitcoin_txref_code +C Reference Implementation (supports version 0): https://github.com/jonasschnelli/bitcoin_txref_code -Go Reference Implementation: https://github.com/kulpreet/txref +Go Reference Implementation (supports version 0): https://github.com/kulpreet/txref + +C++ Reference Implementation (support versions 0 and 1): https://github.com/dcdpr/btcr-DID-method/blob/master/libtxref == Appendices == @@ -234,24 +247,24 @@ The following list gives invalid TxRef's and the reason for their invalidity. * bc1qw508d6qejxtdg4y5r3zarvary0c5xw7kg3g4ty: Invalid human-readable part * tx1qw508d6qejxtdg4y5r3zarvary0c5xw7kv8f3t5: Invalid checksum -==== Bitcoin TxRef (out of date, need to be amended). ==== +==== Bitcoin TxRef ==== The following list gives properly encoded Bitcoin TxRef's and the values in hex. (block height, transaction index) -* tx1:rqqq-qqqq-qmhu-qk: (0x0, 0x0) -* tx1:rqqq-qull-6v87-r7: (0x0, 0x1FFF) -* tx1:r7ll-lrqq-vq5e-gg: (0x1FFFFF, 0x0) -* tx1:r7ll-llll-khym-tq: (0x1FFFFF, 0x1FFF) +* tx1:rqqq-qqqq-qmhu-qhp: (0x0, 0x0) +* tx1:rqqq-qqll-l8xh-jkg: (0x0, 0x7FFF) +* tx1:r7ll-llqq-qghq-qr8: (0xFFFFFF, 0x0) +* tx1:r7ll-llll-l5xt-jzw: (0xFFFFFF, 0x7FFF) -The following list give valid Bitcoin TxRef's and the values in hex. (block height, transaction index) -* tx1:rjk0-u5ng-4jsf-mc: (0x71F69, 0x89D) -* TX1RJK0U5NG4JSFMC: (0x71F69, 0x89D) -* TX1R1JK0--U5bNG4JSb----FMC: (0x71F69, 0x89D) -* tx1 rjk0 u5ng 4jsfmc: (0x71F69, 0x89D) -* tx1!rjk0\u5ng*4jsf^^mc: (0x71F69, 0x89D) +The following list gives valid Bitcoin TxRef's and the values in hex. (block height, transaction index) +* tx1:rjk0-uqay-zsrw-hqe: (0x71F69, 0x89D) +* TX1RJK0UQAYZSRWHQE: (0x71F69, 0x89D) +* TX1RJK0--UQaYZSRw----HQE: (0x71F69, 0x89D) +* tx1 rjk0 uqay zsrw hqe: (0x71F69, 0x89D) +* tx1!rjk0\uqay*zsrw^^hqe: (0x71F69, 0x89D) The following list gives invalid Bitcoin TxRef's and the reason for their invalidity. -* tx1:t7ll-llll-gey7-ez: Magic 0xB instead of 0x3. (0x1FFFFF, 0x1FFF) -* tx1:rlll-llll-cgqu-n2: Version 1 instead of 0. (0x1FFFFF, 0x1FFF) +* tx1:t7ll-llll-ldup-3hh: Magic 0xB instead of 0x3. (0xFFFFFF, 0x7FFF) +* tx1:rlll-llll-lfet-r2y: Version 1 instead of 0. (0xFFFFFF, 0x7FFF) * tx1:rjk0-u5ng-gghq-fkg7: Valid Bech32, but 10x5bit packages instead of 8. * tx1:rjk0-u5qd-s43z: Valid Bech32, but 6x5bit packages instead of 8. @@ -272,5 +285,24 @@ Some calculations showing why we chose these particular bit-length of the block *The ''extreme'' smallest Tx is 60 Byte's: Max 16665 tx in a block. **4B version + 1B tx_in count + 36B previous_output + 1B script length + 0B signature script + 4B sequence + 1B tx_out count + 8B amount + 1B script length + 0B pubkey script + 4B lock_time = 60B +=== Test Vectors (version 1) === +These test vectors are extended TxRefs (version 1): + +==== Bitcoin Extended TxRef ==== +The following list gives properly encoded Bitcoin Extended TxRef's and the values in hex. (block height, transaction index, TXO index) + +* tx1:rpqq-qqqq-qqqq-q2ge-ahz: (0x0, 0x0, 0x0) +* tx1:rpqq-qqql-llqq-qshz-qhw: (0x0, 0x7FFF, 0x0) +* tx1:rp7l-lllq-qqqq-qpvh-wkq: (0xFFFFFF, 0x0, 0x0) +* tx1:rp7l-llll-llqq-qmnv-nkv: (0xFFFFFF, 0x7FFF, 0x0) + +* tx1:rpqq-qqqq-qqpq-qg2s-2w6: (0x0, 0x0, 0x1) +* tx1:rpqq-qqql-llpq-qj4t-hwk: (0x0, 0x7FFF, 0x1) +* tx1:rp7l-lllq-qqpq-qrw7-e0c: (0xFFFFFF, 0x0, 0x1) +* tx1:rp7l-llll-llpq-qe39-y05: (0xFFFFFF, 0x7FFF, 0x1) + +* tx1:rpjk-0uqa-yzu4-x0w0-kuq: (0x71F69, 0x89D, 0x1ABC) +* txtest1:xpjk-0uqa-yzu4-xgrl-pue: (0x71F69, 0x89D, 0x1ABC) (testnet, magic number = 0x6) + == Acknowledgements == Special Thanks to Pieter Wuille and Greg Maxwell for Bech32, a wonderful user-facing data encoding. -- cgit v1.2.3 From 4e2bb34d04cb2fdbb5a52fdc6a606f0dfa91adc8 Mon Sep 17 00:00:00 2001 From: "Daniel X. Pape" Date: Wed, 28 Nov 2018 22:30:09 -0800 Subject: add examples for TxRefs with Outpoints; fix some typos and wording --- bip-0136.mediawiki | 96 +++++++++++++++++++++++++++++++++--------------------- 1 file changed, 58 insertions(+), 38 deletions(-) diff --git a/bip-0136.mediawiki b/bip-0136.mediawiki index 15bc29a..f94171d 100644 --- a/bip-0136.mediawiki +++ b/bip-0136.mediawiki @@ -16,27 +16,27 @@ == Introduction == === Abstract === -This document proposes a convenient human useable format, '''"TxRef"''', as a standard way to refer to a transaction position within the Bitcoin Blockchain, and optionally a particular outpoint index with the referred transaction. The primary purpose of this format is to allow users to refer to a confirmed transaction (and optionally an outpoint index within) in a standard reliable and concise way. +This document proposes a convenient human useable format, '''"TxRef"''', as a standard way to refer to a transaction position within the Bitcoin Blockchain, and optionally a particular outpoint index within the referred transaction. The primary purpose of this format is to allow users to refer to a confirmed transaction (and optionally an outpoint index within) in a standard, reliable, and concise way. -''Please note: Unlike TxID where there is strong cryptographic link between the ID and the actual transaction. TxRef only provide a weak link to a particular transaction. TxRef locates an offset within a blockchain for a transaction, that may - or may not - point to an actual transaction, which fact may change with reorganisations. We recommend that TxRef's should be not used for positions within the blockchain having a maturity less than 100 blocks.'' +''Please note: Unlike TxID where there is strong cryptographic link between the ID and the actual transaction, TxRef only provides a weak link to a particular transaction. TxRef locates an offset within a blockchain for a transaction, that may - or may not - point to an actual transaction, which in fact may change with reorganisations. We recommend that TxRef's should be not used for positions within the blockchain having a maturity less than 100 blocks.'' === Copyright === This BIP is licensed under the 2-clause BSD license. === Motivation === -Since the first version of Bitcoin, TxID's (Transaction Identifiers) that are a core part of the consensus protocol, have been routinely used to identify individual transactions between users. +Since the first version of Bitcoin, TxID's (Transaction Identifiers) have been a core part of the consensus protocol and have been routinely used to identify individual transactions between users. However, for many use-cases they have practical limitations: * TxIDs are expensive for full nodes to lookup (requiring either a linear scan of the blockchain, or an expensive TxID index). -* TxIDs require third-party service for SPV wallets to lookup. +* TxIDs require third-party services for SPV wallets to lookup. * TxIDs are very long HEX encoded values (64 characters long). For transactions that have been embedded in the blockchain, it is possible to reference them not by their TxID, but by their location within the blockchain itself. The encoding can be made friendly for occasional human transcription. In this document, we propose a standard for doing this. === Examples === These examples are for Bitcoin Transactions. -* Genesis Coinbase Transaction: tx1:rqqq-qqqq-qmhu-qhp +* Genesis Coinbase Transaction (Transaction #0 of Block #0): tx1:rqqq-qqqq-qmhu-qhp * Transaction #2205 of Block #466793: tx1:rjk0-uqay-zsrw-hqe == Specification == @@ -69,12 +69,12 @@ TxRef uses standard Bech32'''Why use Bech32 Encoding for Confirme * Separator: '''"1"'''. * Data Part. -Please node: other specifcations, such as [https://w3c-ccg.github.io/did-spec/ Decentralized Identifier syntax] , have implicitly encoded the information contained within the HRP elsewhere. In this case they may choose not include the HRP as specified here. +Please note: other specifications, such as [https://w3c-ccg.github.io/did-spec/ the Decentralized Identifiers spec], have implicitly encoded the information contained within the HRP elsewhere. In this case they may choose to not include the HRP as specified here. To increase portability and readability additional separators SHOULD be added: * A Colon'''Why add a colon here?''' This allows it to conform better with W3C URN/URL standards. '''":"''' added after '1'. -* Hyphens'''Why hyphens to the TxRef?''' As TxRef's are short, we expect that they will be quoted via voice or written by hand. The inclusion of hyphens every 4 characters breaks the string and means people don't lose their place so easily. '''"-"''' added after every 4 characters beyond the colon. +* Hyphens'''Why hyphens within the TxRef?''' As TxRef's are short, we expect that they will be quoted via voice or written by hand. The inclusion of hyphens every 4 characters breaks up the string and means people don't lose their place so easily. '''"-"''' added after every 4 characters beyond the colon. All non-bech32-alphabet characters after the bech32 code separator MUST be ignored/removed when parsing (except for terminating characters).'''Why strip all non-bech32-alphabet characters?''' We do not wish to expect the users to keep their TxRef's in good unicode form (hyphens, colons, invisible spaces, random unicode characters, etc). We expect them to copy, paste, write by-hand, write in a mix of character sets, etc. Parsers should automatically correct for all sorts of these common errors. @@ -116,10 +116,10 @@ All non-bech32-alphabet characters after the bech32 code separator MUST be ignor |1 |"'''-'''" |} -The Data - Hyphen patten is repeated for the entire length of data, ( a hyphen is inserted after every encoded 20 bits or 4 data characters). +The Data - Hyphen pattern is repeated for the entire length of data, ( a hyphen is inserted after every encoded 20 bits or 4 data characters). === Data === -The 75 or 90 bits of data encoded in the string above are defined in this manner: +Depending on if an optional transaction outpoint is included, there can be 75 or 90 bits of data encoded in the string above. These bits are defined in this manner: {| class="wikitable" |+TxRef Binary Format for Bitcoin Mainnet and Bitcoin Testnet: @@ -130,7 +130,7 @@ The 75 or 90 bits of data encoded in the string above are defined in this manner !'''Values''' !'''Notes''' |- -|Magic +|Magic Code |0 – 4 |5 |Chain Namespacing Code @@ -161,7 +161,7 @@ The 75 or 90 bits of data encoded in the string above are defined in this manner |Tx 0 (coinbase) to Tx position 32767 |Max Tx's in block is 16665 |} -If the magic is '''0x4''' or '''0x7''', an optional outpoint is included in the encoding: +If the magic code is '''0x4''' or '''0x7''', an optional outpoint is included in the encoding: {| class="wikitable" |+Optional Outpoint Index Encoding: @@ -176,7 +176,7 @@ If the magic is '''0x4''' or '''0x7''', an optional outpoint is included in the |45 – 59 |15 |The index of the Outpoint inside the Tx -|Outpoint 0 to Outpoint Position 32767 +|Outpoint 0 to Outpoint Position 32767 | |} @@ -220,11 +220,11 @@ There are no known compatibility issues. == Reference implementations == -C Reference Implementation (supports version 0): https://github.com/jonasschnelli/bitcoin_txref_code +C Reference Implementation (supports magic codes 0x3 and 0x6): https://github.com/jonasschnelli/bitcoin_txref_code -Go Reference Implementation (supports version 0): https://github.com/kulpreet/txref +Go Reference Implementation (supports magic codes 0x3 and 0x6): https://github.com/kulpreet/txref -C++ Reference Implementation (support versions 0 and 1): https://github.com/dcdpr/btcr-DID-method/blob/master/libtxref +C++ Reference Implementation (support magic codes 0x3, 0x4, 0x6, 0x7): https://github.com/dcdpr/btcr-DID-method/ == Appendices == @@ -232,7 +232,7 @@ C++ Reference Implementation (support versions 0 and 1): https://github.com/dcdp There are two sets of Test Vectors included here: * Bech32 Encoding Test Vectors. These are to test if a implementation accepts the encoding, with the correct human readable part, and separator. -* Bitcoin TxRef Test Vectors. These test the full specification, in particular correct values for block height and the transaction index. +* Bitcoin TxRef Test Vectors. These test the full specification, in particular, correct values for block height and the transaction index. ==== Bech32 Encoding (for TxRef). ==== ''Please Note: All test vectors are shown to help test if a string is compliant or not. All real-life applications (such as for Bitcoin) should comply with the Bitcoin Test Vectors listed Below.'' @@ -247,15 +247,22 @@ The following list gives invalid TxRef's and the reason for their invalidity. * bc1qw508d6qejxtdg4y5r3zarvary0c5xw7kg3g4ty: Invalid human-readable part * tx1qw508d6qejxtdg4y5r3zarvary0c5xw7kv8f3t5: Invalid checksum -==== Bitcoin TxRef ==== -The following list gives properly encoded Bitcoin TxRef's and the values in hex. (block height, transaction index) +==== Bitcoin TxRef (mainnet and testnet) ==== +The following list gives properly encoded Bitcoin mainnet TxRef's and the values in hex. (block height, transaction index) * tx1:rqqq-qqqq-qmhu-qhp: (0x0, 0x0) * tx1:rqqq-qqll-l8xh-jkg: (0x0, 0x7FFF) * tx1:r7ll-llqq-qghq-qr8: (0xFFFFFF, 0x0) * tx1:r7ll-llll-l5xt-jzw: (0xFFFFFF, 0x7FFF) -The following list gives valid Bitcoin TxRef's and the values in hex. (block height, transaction index) +The following list gives properly encoded Bitcoin testnet TxRef's and the values in hex. (block height, transaction index) + +* txtest1:xqqq-qqqq-qkla-64l: (0x0, 0x0) +* txtest1:xqqq-qqll-l2wk-g5k: (0x0, 0x7FFF) +* txtest1:x7ll-llqq-q9lp-6pe: (0xFFFFFF, 0x0) +* txtest1:x7ll-llll-lew2-gqs: (0xFFFFFF, 0x7FFF) + +The following list gives valid (though strangely formatted) Bitcoin TxRef's and the values in hex. (block height, transaction index) * tx1:rjk0-uqay-zsrw-hqe: (0x71F69, 0x89D) * TX1RJK0UQAYZSRWHQE: (0x71F69, 0x89D) * TX1RJK0--UQaYZSRw----HQE: (0x71F69, 0x89D) @@ -268,6 +275,38 @@ The following list gives invalid Bitcoin TxRef's and the reason for their invali * tx1:rjk0-u5ng-gghq-fkg7: Valid Bech32, but 10x5bit packages instead of 8. * tx1:rjk0-u5qd-s43z: Valid Bech32, but 6x5bit packages instead of 8. +==== Bitcoin TxRef with Outpoints (mainnet and testnet) ==== +The following list gives properly encoded Bitcoin mainnet TxRef's with Outpoints and the values in hex. (block height, transaction index, TXO index) + +* tx1:yqqq-qqqq-qqqq-ksvh-26: (0x0, 0x0, 0x0) +* tx1:yqqq-qqll-lqqq-v0h2-2k: (0x0, 0x7FFF, 0x0) +* tx1:y7ll-llqq-qqqq-a5zy-tc: (0xFFFFFF, 0x0, 0x0) +* tx1:y7ll-llll-lqqq-8tee-t5: (0xFFFFFF, 0x7FFF, 0x0) + +* tx1:yqqq-qqqq-qpqq-5j9q-nz: (0x0, 0x0, 0x1) +* tx1:yqqq-qqll-lpqq-wd7a-nw: (0x0, 0x7FFF, 0x1) +* tx1:y7ll-llqq-qpqq-lktn-jq: (0xFFFFFF, 0x0, 0x1) +* tx1:y7ll-llll-lpqq-9fsw-jv: (0xFFFFFF, 0x7FFF, 0x1) + +* tx1:yjk0-uqay-zrfq-g2cg-t8: (0x71F69, 0x89D, 0x123) +* tx1:yjk0-uqay-zu4x-nk6u-pc: (0x71F69, 0x89D, 0x1ABC) + +The following list gives properly encoded Bitcoin testnet TxRef's with Outpoints and the values in hex. (block height, transaction index, TXO index) + +* txtest1:8qqq-qqqq-qqqq-cgru-fa: (0x0, 0x0, 0x0) +* txtest1:8qqq-qqll-lqqq-zhcp-f3: (0x0, 0x7FFF, 0x0) +* txtest1:87ll-llqq-qqqq-nvd0-gl: (0xFFFFFF, 0x0, 0x0) +* txtest1:87ll-llll-lqqq-fnkj-gn: (0xFFFFFF, 0x7FFF, 0x0) + +* txtest1:8qqq-qqqq-qpqq-622t-s9: (0x0, 0x0, 0x1) +* txtest1:8qqq-qqll-lpqq-q43k-sf: (0x0, 0x7FFF, 0x1) +* txtest1:87ll-llqq-qpqq-3wyc-38: (0xFFFFFF, 0x0, 0x1) +* txtest1:87ll-llll-lpqq-t3l9-3t: (0xFFFFFF, 0x7FFF, 0x1) + +* txtest1:8jk0-uqay-zrfq-xjhr-gq: (0x71F69, 0x89D, 0x123) +* txtest1:8jk0-uqay-zu4x-aw4h-zl: (0x71F69, 0x89D, 0x1ABC) + + === Bitcoin TxRef Payload Value Choice: === Some calculations showing why we chose these particular bit-length of the block height and transaction index. @@ -285,24 +324,5 @@ Some calculations showing why we chose these particular bit-length of the block *The ''extreme'' smallest Tx is 60 Byte's: Max 16665 tx in a block. **4B version + 1B tx_in count + 36B previous_output + 1B script length + 0B signature script + 4B sequence + 1B tx_out count + 8B amount + 1B script length + 0B pubkey script + 4B lock_time = 60B -=== Test Vectors (version 1) === -These test vectors are extended TxRefs (version 1): - -==== Bitcoin Extended TxRef ==== -The following list gives properly encoded Bitcoin Extended TxRef's and the values in hex. (block height, transaction index, TXO index) - -* tx1:rpqq-qqqq-qqqq-q2ge-ahz: (0x0, 0x0, 0x0) -* tx1:rpqq-qqql-llqq-qshz-qhw: (0x0, 0x7FFF, 0x0) -* tx1:rp7l-lllq-qqqq-qpvh-wkq: (0xFFFFFF, 0x0, 0x0) -* tx1:rp7l-llll-llqq-qmnv-nkv: (0xFFFFFF, 0x7FFF, 0x0) - -* tx1:rpqq-qqqq-qqpq-qg2s-2w6: (0x0, 0x0, 0x1) -* tx1:rpqq-qqql-llpq-qj4t-hwk: (0x0, 0x7FFF, 0x1) -* tx1:rp7l-lllq-qqpq-qrw7-e0c: (0xFFFFFF, 0x0, 0x1) -* tx1:rp7l-llll-llpq-qe39-y05: (0xFFFFFF, 0x7FFF, 0x1) - -* tx1:rpjk-0uqa-yzu4-x0w0-kuq: (0x71F69, 0x89D, 0x1ABC) -* txtest1:xpjk-0uqa-yzu4-xgrl-pue: (0x71F69, 0x89D, 0x1ABC) (testnet, magic number = 0x6) - == Acknowledgements == Special Thanks to Pieter Wuille and Greg Maxwell for Bech32, a wonderful user-facing data encoding. -- cgit v1.2.3 From b65ca1c094c1efb193a79d39efac6e34d390254a Mon Sep 17 00:00:00 2001 From: Fabian Jahr Date: Thu, 13 Jun 2019 12:19:57 -0400 Subject: BIP 158: remove old reference to extended filter type --- bip-0158.mediawiki | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/bip-0158.mediawiki b/bip-0158.mediawiki index 6c3202b..07f1ea1 100644 --- a/bip-0158.mediawiki +++ b/bip-0158.mediawiki @@ -27,9 +27,8 @@ enables basic wallets and applications with more advanced smart contracts. [[bip-0157.mediawiki|BIP 157]] defines a light client protocol based on deterministic filters of block content. The filters are designed to minimize the expected bandwidth consumed by light clients, downloading filters -and full blocks. This document defines two initial filter types, ''basic'' and -''extended'', to provide support for advanced applications while reducing the -filter size for regular wallets. +and full blocks. This document defines the initial filter type ''basic'' +that is designed to reduce the filter size for regular wallets. == Definitions == -- cgit v1.2.3 From 4fe45328c8ad8e23a0a3c3c6ee8133c982905147 Mon Sep 17 00:00:00 2001 From: nkohen Date: Thu, 13 Jun 2019 13:52:10 -0500 Subject: Updated Golomb-Rice Coded sets reference implementation link --- bip-0158.mediawiki | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bip-0158.mediawiki b/bip-0158.mediawiki index 6c3202b..579a246 100644 --- a/bip-0158.mediawiki +++ b/bip-0158.mediawiki @@ -348,7 +348,7 @@ Light client: [https://github.com/lightninglabs/neutrino] Full-node indexing: https://github.com/Roasbeef/btcd/tree/segwit-cbf -Golomb-Rice Coded sets: https://github.com/Roasbeef/btcutil/tree/gcs/gcs +Golomb-Rice Coded sets: https://github.com/btcsuite/btcutil/blob/master/gcs == Appendix A: Alternatives == -- cgit v1.2.3 From 71aeb9741135bb47cb74a29be22a5391c3f93e88 Mon Sep 17 00:00:00 2001 From: Janus Date: Thu, 20 Jun 2019 16:11:19 -0500 Subject: Reject BIP-0074 (three years inactivity) --- README.mediawiki | 4 ++-- bip-0074.mediawiki | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/README.mediawiki b/README.mediawiki index 11ecbb1..420ce2b 100644 --- a/README.mediawiki +++ b/README.mediawiki @@ -371,13 +371,13 @@ Those proposing changes should consider that ultimately consent may rest with th | Stephen Pair | Standard | Final -|- +|- style="background-color: #ffcfcf" | [[bip-0074.mediawiki|74]] | Applications | Allow zero value OP_RETURN in Payment Protocol | Toby Padilla | Standard -| Draft +| Rejected |- | [[bip-0075.mediawiki|75]] | Applications diff --git a/bip-0074.mediawiki b/bip-0074.mediawiki index 01fcf2c..b6e9b39 100644 --- a/bip-0074.mediawiki +++ b/bip-0074.mediawiki @@ -5,7 +5,7 @@ Author: Toby Padilla Comments-Summary: Unanimously Discourage for implementation Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0074 - Status: Draft + Status: Rejected Type: Standards Track Created: 2016-01-29 License: PD -- cgit v1.2.3 From 7ab9ef95c67051dcf6418ecd4310e63b036bc6f0 Mon Sep 17 00:00:00 2001 From: Luke Childs Date: Wed, 26 Jun 2019 16:47:04 +0700 Subject: [bip174] Fix typo in signer pseudo code --- bip-0174.mediawiki | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bip-0174.mediawiki b/bip-0174.mediawiki index f197728..cc165a5 100644 --- a/bip-0174.mediawiki +++ b/bip-0174.mediawiki @@ -337,7 +337,7 @@ sign_non_witness(script_code, i): for input,i in enumerate(psbt.inputs): if non_witness_utxo.exists: - assert(sha256d(non_witness_utxo) == psbt.tx.innput[i].prevout.hash) + assert(sha256d(non_witness_utxo) == psbt.tx.input[i].prevout.hash) if redeemScript.exists: assert(non_witness_utxo.vout[psbt.tx.input[i].prevout.n].scriptPubKey == P2SH(redeemScript)) sign_non_witness(redeemScript) -- cgit v1.2.3 From 308594e098424cfc8555b56832e07119ea5d2c7c Mon Sep 17 00:00:00 2001 From: wigy Date: Mon, 1 Jul 2019 17:07:12 +0200 Subject: Updated BIP-0138 Comments-Summary That opinion was unnoticed for more than a month now, maybe this helps getting counter-arguments. --- bip-0178.mediawiki | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bip-0178.mediawiki b/bip-0178.mediawiki index 439774e..b26efde 100644 --- a/bip-0178.mediawiki +++ b/bip-0178.mediawiki @@ -3,7 +3,7 @@ Layer: Applications Title: Version Extended WIF Author: Karl-Johan Alm - Comments-Summary: No comments yet. + Comments-Summary: One comment, with some Discouragement Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0178 Status: Draft Type: Standards Track -- cgit v1.2.3 From 97447f981fb9e2b0c4c0c8c095464c4dbd91da29 Mon Sep 17 00:00:00 2001 From: Jonathan Underwood Date: Wed, 10 Jul 2019 08:30:23 +0900 Subject: BIP174: Include suggested sighash check --- bip-0174.mediawiki | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/bip-0174.mediawiki b/bip-0174.mediawiki index f197728..b352c3b 100644 --- a/bip-0174.mediawiki +++ b/bip-0174.mediawiki @@ -319,6 +319,8 @@ For a Signer to only produce valid signatures for what it expects to sign, it mu * If a witness UTXO is provided, no non-witness signature may be created * If a redeemScript is provided, the scriptPubKey must be for that redeemScript * If a witnessScript is provided, the scriptPubKey or the redeemScript must be for that witnessScript +* If a sighash type is provided, the signer must check that the sighash is acceptable. If unacceptable, they must fail. +* If a sighash type is not provided, the signer should sign using SIGHASH_ALL, but may use any sighash type they wish. =====Simple Signer Algorithm===== @@ -326,13 +328,17 @@ A simple signer can use the following algorithm to determine what and how to sig
 sign_witness(script_code, i):
-    for key in psbt.inputs[i].keys:
-        if IsMine(key):
+    for key, sighash_type in psbt.inputs[i].items:
+        if sighash_type == None:
+            sighash_type = SIGHASH_ALL
+        if IsMine(key) and IsAcceptable(sighash_type):
             sign(witness_sighash(script_code, i, input))
 
 sign_non_witness(script_code, i):
-    for key in psbt.inputs[i].keys:
-        if IsMine(key):
+    for key, sighash_type in psbt.inputs[i].items:
+        if sighash_type == None:
+            sighash_type = SIGHASH_ALL
+        if IsMine(key) and IsAcceptable(sighash_type):
             sign(non_witness_sighash(script_code, i, input))
 
 for input,i in enumerate(psbt.inputs):
-- 
cgit v1.2.3


From 34534b5ee1cf49cf67005b5457fce110ea6b0f4b Mon Sep 17 00:00:00 2001
From: "Wladimir J. van der Laan" 
Date: Thu, 18 Jul 2019 00:17:55 +0200
Subject: bip155: fill in BIP number

---
 bip-0155.mediawiki | 189 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 bip-XXXX.mediawiki | 189 -----------------------------------------------------
 2 files changed, 189 insertions(+), 189 deletions(-)
 create mode 100644 bip-0155.mediawiki
 delete mode 100644 bip-XXXX.mediawiki

diff --git a/bip-0155.mediawiki b/bip-0155.mediawiki
new file mode 100644
index 0000000..27bec17
--- /dev/null
+++ b/bip-0155.mediawiki
@@ -0,0 +1,189 @@
+
+  BIP: 155
+  Layer: Peer Services
+  Title: addrv2 message
+  Author: Wladimir J. van der Laan 
+  Comments-Summary: No comments yet.
+  Comments-URI: 
+  Status: Draft
+  Type: Standards Track
+  Created: 2019-02-27
+  License: BSD-2-Clause
+
+ +==Introduction== + +===Abstract=== + +This document proposes a new P2P message to gossip longer node addresses over the P2P network. +This is required to support new-generation Onion addresses, I2P, and potentially other networks +that have longer endpoint addresses than fit in the 128 bits of the current addr message. + +===Copyright=== + +This BIP is licensed under the 2-clause BSD license. + +===Motivation=== + +Tor v3 hidden services are part of the stable release of Tor since version 0.3.2.9. They have +various advantages compared to the old hidden services, among which better encryption and privacy +[https://gitweb.torproject.org/torspec.git/tree/rend-spec-v3.txt Tor Rendezvous Specification - Version 3]. +These services have 256 bit addresses and thus do not fit in the existing addr message, which encapsulates onion addresses in OnionCat IPv6 addresses. + +Other transport-layer protocols such as I2P have always used longer +addresses. This change would make it possible to gossip such addresses over the +P2P network, so that other peers can connect to them. + +==Specification== + +
+The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", +"SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be +interpreted as described in RFC 2119[https://tools.ietf.org/html/rfc2119 RFC 2119]. +
+ +The addrv2 message is defined as a message where pchCommand == "addrv2". +It is serialized in the standard encoding for P2P messages. +Its format is similar to the current addr message format +[https://bitcoin.org/en/developer-reference#addr Bitcoin Developer Reference: addr message], with the difference that the +fixed 16-byte IP address is replaced by a network ID and a variable-length address, and the time and services format has been changed to VARINT. + +This means that the message contains a serialized std::vector of the following structure: + +{| class="wikitable" style="width: auto; text-align: center; font-size: smaller; table-layout: fixed;" +!Type +!Name +!Description +|- +| VARINT (unsigned) +| time +| Time that this node was last seen as connected to the network. A time in Unix epoch time format, up to 64 bits wide. +|- +| VARINT (unsigned) +| services +| Service bits. A 64-wide bit field. +|- +| uint8_t +| networkID +| Network identifier. An 8-bit value that specifies which network is addressed. +|- +| std::vector +| addr +| Network address. The interpretation depends on networkID. +|- +| uint16_t +| port +| Network port. If not relevant for the network this MUST be 0. +|} + +One message can contain up to 1,000 addresses. Clients SHOULD reject messages with more addresses. + +Field addr has a variable length, with a maximum of 32 bytes (256 bits). Clients SHOULD reject +longer addresses. + +The list of reserved network IDs is as follows: + +{| class="wikitable" style="width: auto; text-align: center; font-size: smaller; table-layout: fixed;" +!Network ID +!Enumeration +!Address length (bytes) +!Description +|- +| 0x01 +| IPV4 +| 4 +| IPv4 address (globally routed internet) +|- +| 0x02 +| IPV6 +| 16 +| IPv6 address (globally routed internet) +|- +| 0x03 +| TORV2 +| 10 +| Tor v2 hidden service address +|- +| 0x04 +| TORV3 +| 32 +| Tor v3 hidden service address +|- +| 0x05 +| I2P +| 32 +| I2P overlay network address +|- +| 0x06 +| CJDNS +| 16 +| Cjdns overlay network address +|} + +To allow for future extensibility, clients MUST ignore address types that they do not know about. +Client MAY store and gossip address formats that they do not know about. Further network ID numbers MUST be reserved in a new BIP document. + +Clients SHOULD reject addresses that have a different length than specified in this table for a specific address ID, as these are meaningless. + +See the appendices for the address encodings to be used for the various networks. + +==Compatibility== + +Send addrv2 messages only, and exclusively, when the peer has a certain protocol version (or higher): + +//! gossiping using `addrv2` messages starts with this version +static const int GOSSIP_ADDRV2_VERSION = 70016; + +For older peers keep sending the legacy addr message, ignoring addresses with the newly introduced address types. + +==Reference implementation== + +The reference implementation is available at (to be done) + +==Acknowledgements== + +- Jonas Schnelli: change services field to VARINT, to make the message more compact in the likely case instead of always using 8 bytes. + +- Luke-Jr: change time field to VARINT, for post-2038 compatibility. + +- Gregory Maxwell: various suggestions regarding extensibility + +==Appendix A: Tor v2 address encoding== + +The new message introduces a separate network ID for TORV2. + +Clients MUST send Tor hidden service addresses with this network ID, with the 80-bit hidden service ID in the address field. This is the same as the representation in the legacy addr message, minus the 6 byte prefix of the OnionCat wrapping. + +Clients SHOULD ignore OnionCat (fd87:d87e:eb43::/48) addresses on receive if they come with the IPV6 network ID. + +==Appendix B: Tor v3 address encoding== + +According to the spec [https://gitweb.torproject.org/torspec.git/tree/rend-spec-v3.txt Tor Rendezvous Specification - Version 3: Encoding onion addresses], next-gen .onion addresses are encoded as follows: +
+onion_address = base32(PUBKEY | CHECKSUM | VERSION) + ".onion"
+ CHECKSUM = H(".onion checksum" | PUBKEY | VERSION)[:2]
+
+ where:
+   - PUBKEY is the 32 bytes ed25519 master pubkey of the hidden service.
+   - VERSION is an one byte version field (default value '\x03')
+   - ".onion checksum" is a constant string
+   - CHECKSUM is truncated to two bytes before inserting it in onion_address
+
+ +Tor v3 addresses MUST be sent with the TORV3 network ID, with the 32-byte PUBKEY part in the address field. As VERSION will always be '\x03' in the case of v3 addresses, this is enough to reconstruct the onion address. + +==Appendix C: I2P address encoding== + +Like Tor, I2P naming uses a base32-encoded address format[https://geti2p.net/en/docs/naming#base32 I2P: Naming and address book]. + +I2P uses 52 characters (256 bits) to represent the full SHA-256 hash, followed by .b32.i2p. + +I2P addresses MUST be sent with the I2P network ID, with the decoded SHA-256 hash as address field. + +==Appendix D: Cjdns address encoding== + +Cjdns addresses are simply IPv6 addresses in the fc00::/8 range[https://github.com/cjdelisle/cjdns/blob/6e46fa41f5647d6b414612d9d63626b0b952746b/doc/Whitepaper.md#pulling-it-all-together Cjdns whitepaper: Pulling It All Together]. They MUST be sent with the CJDNS network ID. + +==References== + + diff --git a/bip-XXXX.mediawiki b/bip-XXXX.mediawiki deleted file mode 100644 index 583bf8b..0000000 --- a/bip-XXXX.mediawiki +++ /dev/null @@ -1,189 +0,0 @@ -
-  BIP: ???
-  Layer: Peer Services
-  Title: addrv2 message
-  Author: Wladimir J. van der Laan 
-  Comments-Summary: No comments yet.
-  Comments-URI: 
-  Status: Draft
-  Type: Standards Track
-  Created: 2019-02-27
-  License: BSD-2-Clause
-
- -==Introduction== - -===Abstract=== - -This document proposes a new P2P message to gossip longer node addresses over the P2P network. -This is required to support new-generation Onion addresses, I2P, and potentially other networks -that have longer endpoint addresses than fit in the 128 bits of the current addr message. - -===Copyright=== - -This BIP is licensed under the 2-clause BSD license. - -===Motivation=== - -Tor v3 hidden services are part of the stable release of Tor since version 0.3.2.9. They have -various advantages compared to the old hidden services, among which better encryption and privacy -[https://gitweb.torproject.org/torspec.git/tree/rend-spec-v3.txt Tor Rendezvous Specification - Version 3]. -These services have 256 bit addresses and thus do not fit in the existing addr message, which encapsulates onion addresses in OnionCat IPv6 addresses. - -Other transport-layer protocols such as I2P have always used longer -addresses. This change would make it possible to gossip such addresses over the -P2P network, so that other peers can connect to them. - -==Specification== - -
-The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", -"SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be -interpreted as described in RFC 2119[https://tools.ietf.org/html/rfc2119 RFC 2119]. -
- -The addrv2 message is defined as a message where pchCommand == "addrv2". -It is serialized in the standard encoding for P2P messages. -Its format is similar to the current addr message format -[https://bitcoin.org/en/developer-reference#addr Bitcoin Developer Reference: addr message], with the difference that the -fixed 16-byte IP address is replaced by a network ID and a variable-length address, and the time and services format has been changed to VARINT. - -This means that the message contains a serialized std::vector of the following structure: - -{| class="wikitable" style="width: auto; text-align: center; font-size: smaller; table-layout: fixed;" -!Type -!Name -!Description -|- -| VARINT (unsigned) -| time -| Time that this node was last seen as connected to the network. A time in Unix epoch time format, up to 64 bits wide. -|- -| VARINT (unsigned) -| services -| Service bits. A 64-wide bit field. -|- -| uint8_t -| networkID -| Network identifier. An 8-bit value that specifies which network is addressed. -|- -| std::vector -| addr -| Network address. The interpretation depends on networkID. -|- -| uint16_t -| port -| Network port. If not relevant for the network this MUST be 0. -|} - -One message can contain up to 1,000 addresses. Clients SHOULD reject messages with more addresses. - -Field addr has a variable length, with a maximum of 32 bytes (256 bits). Clients SHOULD reject -longer addresses. - -The list of reserved network IDs is as follows: - -{| class="wikitable" style="width: auto; text-align: center; font-size: smaller; table-layout: fixed;" -!Network ID -!Enumeration -!Address length (bytes) -!Description -|- -| 0x01 -| IPV4 -| 4 -| IPv4 address (globally routed internet) -|- -| 0x02 -| IPV6 -| 16 -| IPv6 address (globally routed internet) -|- -| 0x03 -| TORV2 -| 10 -| Tor v2 hidden service address -|- -| 0x04 -| TORV3 -| 32 -| Tor v3 hidden service address -|- -| 0x05 -| I2P -| 32 -| I2P overlay network address -|- -| 0x06 -| CJDNS -| 16 -| Cjdns overlay network address -|} - -To allow for future extensibility, clients MUST ignore address types that they do not know about. -Client MAY store and gossip address formats that they do not know about. Further network ID numbers MUST be reserved in a new BIP document. - -Clients SHOULD reject addresses that have a different length than specified in this table for a specific address ID, as these are meaningless. - -See the appendices for the address encodings to be used for the various networks. - -==Compatibility== - -Send addrv2 messages only, and exclusively, when the peer has a certain protocol version (or higher): - -//! gossiping using `addrv2` messages starts with this version -static const int GOSSIP_ADDRV2_VERSION = 70016; - -For older peers keep sending the legacy addr message, ignoring addresses with the newly introduced address types. - -==Reference implementation== - -The reference implementation is available at (to be done) - -==Acknowledgements== - -- Jonas Schnelli: change services field to VARINT, to make the message more compact in the likely case instead of always using 8 bytes. - -- Luke-Jr: change time field to VARINT, for post-2038 compatibility. - -- Gregory Maxwell: various suggestions regarding extensibility - -==Appendix A: Tor v2 address encoding== - -The new message introduces a separate network ID for TORV2. - -Clients MUST send Tor hidden service addresses with this network ID, with the 80-bit hidden service ID in the address field. This is the same as the representation in the legacy addr message, minus the 6 byte prefix of the OnionCat wrapping. - -Clients SHOULD ignore OnionCat (fd87:d87e:eb43::/48) addresses on receive if they come with the IPV6 network ID. - -==Appendix B: Tor v3 address encoding== - -According to the spec [https://gitweb.torproject.org/torspec.git/tree/rend-spec-v3.txt Tor Rendezvous Specification - Version 3: Encoding onion addresses], next-gen .onion addresses are encoded as follows: -
-onion_address = base32(PUBKEY | CHECKSUM | VERSION) + ".onion"
- CHECKSUM = H(".onion checksum" | PUBKEY | VERSION)[:2]
-
- where:
-   - PUBKEY is the 32 bytes ed25519 master pubkey of the hidden service.
-   - VERSION is an one byte version field (default value '\x03')
-   - ".onion checksum" is a constant string
-   - CHECKSUM is truncated to two bytes before inserting it in onion_address
-
- -Tor v3 addresses MUST be sent with the TORV3 network ID, with the 32-byte PUBKEY part in the address field. As VERSION will always be '\x03' in the case of v3 addresses, this is enough to reconstruct the onion address. - -==Appendix C: I2P address encoding== - -Like Tor, I2P naming uses a base32-encoded address format[https://geti2p.net/en/docs/naming#base32 I2P: Naming and address book]. - -I2P uses 52 characters (256 bits) to represent the full SHA-256 hash, followed by .b32.i2p. - -I2P addresses MUST be sent with the I2P network ID, with the decoded SHA-256 hash as address field. - -==Appendix D: Cjdns address encoding== - -Cjdns addresses are simply IPv6 addresses in the fc00::/8 range[https://github.com/cjdelisle/cjdns/blob/6e46fa41f5647d6b414612d9d63626b0b952746b/doc/Whitepaper.md#pulling-it-all-together Cjdns whitepaper: Pulling It All Together]. They MUST be sent with the CJDNS network ID. - -==References== - - -- cgit v1.2.3 From e99347806027c63ac459c35ccda4abb1fa0b83c0 Mon Sep 17 00:00:00 2001 From: "Wladimir J. van der Laan" Date: Thu, 18 Jul 2019 00:33:26 +0200 Subject: bip155: Fix comments URL --- bip-0155.mediawiki | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bip-0155.mediawiki b/bip-0155.mediawiki index 27bec17..5914241 100644 --- a/bip-0155.mediawiki +++ b/bip-0155.mediawiki @@ -4,7 +4,7 @@ Title: addrv2 message Author: Wladimir J. van der Laan Comments-Summary: No comments yet. - Comments-URI: + Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0155 Status: Draft Type: Standards Track Created: 2019-02-27 -- cgit v1.2.3 From f5174192e2d5fb3dfa7232be62d1b9d586a7f8d6 Mon Sep 17 00:00:00 2001 From: "Wladimir J. van der Laan" Date: Thu, 18 Jul 2019 00:37:29 +0200 Subject: bip155: Add table entry --- README.mediawiki | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/README.mediawiki b/README.mediawiki index 9ba9615..57f79ef 100644 --- a/README.mediawiki +++ b/README.mediawiki @@ -750,6 +750,13 @@ Those proposing changes should consider that ultimately consent may rest with th | Standard | Draft |- +| [[bip-0155.mediawiki|155]] +| Peer Services +| addrv2 message +| Wladimir J. van der Laan +| Standard +| Draft +|- | [[bip-0156.mediawiki|156]] | Peer Services | Dandelion - Privacy Enhancing Routing -- cgit v1.2.3 From 39a23b7cbed59dd922604b62502ad17379530036 Mon Sep 17 00:00:00 2001 From: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com> Date: Thu, 18 Jul 2019 13:24:32 +0300 Subject: BIP 173: Remove duplicated 'the' --- bip-0173.mediawiki | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bip-0173.mediawiki b/bip-0173.mediawiki index ad6c58b..c3ee060 100644 --- a/bip-0173.mediawiki +++ b/bip-0173.mediawiki @@ -209,7 +209,7 @@ implementations' assumptions about lengths), but still be visually distinct. for testnet. * The data-part values: ** 1 byte: the witness version -** A conversion of the the 2-to-40-byte witness program (as defined by [https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki BIP141]) to base32: +** A conversion of the 2-to-40-byte witness program (as defined by [https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki BIP141]) to base32: *** Start with the bits of the witness program, most significant bit per byte first. *** Re-arrange those bits into groups of 5, and pad with zeroes at the end if needed. *** Translate those bits to characters using the table above. -- cgit v1.2.3 From 42cc0c724d0daebc7277e71382f91072de6aa9ad Mon Sep 17 00:00:00 2001 From: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com> Date: Thu, 18 Jul 2019 13:30:06 +0300 Subject: BIP 143: Remove duplicated 'the' --- bip-0143.mediawiki | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bip-0143.mediawiki b/bip-0143.mediawiki index ed07c82..81763a0 100644 --- a/bip-0143.mediawiki +++ b/bip-0143.mediawiki @@ -391,7 +391,7 @@ This example shows how unexecuted OP_CODESEPARATOR is processed, an 02 4730440220032521802a76ad7bf74d0e2c218b72cf0cbc867066e2e53db905ba37f130397e02207709e2188ed7f08f4c952d9d13986da504502b8c3be59617e043552f506c46ff83 275163ab68210392972e2eb617b2388771abe27235fd5ac44af8e61693261550447a4c3e39da98ac nLockTime: 00000000 - Since SINGLE|ANYONECANPAY does not commit to the input index, the signatures are still valid when the the input-output pairs are swapped: + Since SINGLE|ANYONECANPAY does not commit to the input index, the signatures are still valid when the input-output pairs are swapped: 0100000000010280e68831516392fcd100d186b3c2c7b95c80b53c77e77c35ba03a66b429a2a1b0000000000ffffffffe9b542c5176808107ff1df906f46bb1f2583b16112b95ee5380665ba7fcfc0010000000000ffffffff0280969800000000001976a9146648a8cd4531e1ec47f35916de8e259237294d1e88ac80969800000000001976a914de4b231626ef508c9a74a8517e6783c0546d6b2888ac024730440220032521802a76ad7bf74d0e2c218b72cf0cbc867066e2e53db905ba37f130397e02207709e2188ed7f08f4c952d9d13986da504502b8c3be59617e043552f506c46ff83275163ab68210392972e2eb617b2388771abe27235fd5ac44af8e61693261550447a4c3e39da98ac02483045022100f6a10b8604e6dc910194b79ccfc93e1bc0ec7c03453caaa8987f7d6c3413566002206216229ede9b4d6ec2d325be245c5b508ff0339bf1794078e20bfe0babc7ffe683270063ab68210392972e2eb617b2388771abe27235fd5ac44af8e61693261550447a4c3e39da98ac00000000 nVersion: 01000000 marker: 00 -- cgit v1.2.3 From 4413bdc46ffa779987ff2ffb9ac9257744e35316 Mon Sep 17 00:00:00 2001 From: Clark Moody Date: Thu, 14 Feb 2019 12:25:38 -0600 Subject: Add version bytes section --- bip-0049.mediawiki | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/bip-0049.mediawiki b/bip-0049.mediawiki index 74645a1..846aefa 100644 --- a/bip-0049.mediawiki +++ b/bip-0049.mediawiki @@ -66,6 +66,12 @@ To derive the P2SH address from the above calculated public key, we use the enca scriptPubKey: HASH160 <20-byte-script-hash> EQUAL (0xA914{20-byte-script-hash}87) + +===Extended Key Version=== + +When serializing extended keys, this scheme uses alternate version bytes. Extended public keys use 0x049d7cb2 to produce a "ypub" prefix, and private keys use 0x049d7878 to produce a "yprv" prefix. Testnet uses 0x044a5262 "upub" and 0x044a4e28 "uprv." + + ==Backwards Compatibility== This BIP is not backwards compatible by design as described under [#considerations]. An incompatible wallet will not discover accounts at all and the user will notice that something is wrong. -- cgit v1.2.3 From 75ae8418da7697f923efcd55a33936b1b8275c80 Mon Sep 17 00:00:00 2001 From: Clark Moody Date: Thu, 14 Feb 2019 17:36:42 -0600 Subject: Link to SLIP-0132 --- bip-0049.mediawiki | 2 ++ 1 file changed, 2 insertions(+) diff --git a/bip-0049.mediawiki b/bip-0049.mediawiki index 846aefa..c7c383e 100644 --- a/bip-0049.mediawiki +++ b/bip-0049.mediawiki @@ -71,6 +71,8 @@ To derive the P2SH address from the above calculated public key, we use the enca When serializing extended keys, this scheme uses alternate version bytes. Extended public keys use 0x049d7cb2 to produce a "ypub" prefix, and private keys use 0x049d7878 to produce a "yprv" prefix. Testnet uses 0x044a5262 "upub" and 0x044a4e28 "uprv." +Additional registered version bytes are listed in [[https://github.com/satoshilabs/slips/blob/master/slip-0132.md|SLIP-0132]]. + ==Backwards Compatibility== -- cgit v1.2.3 From 5b74e93fcba783c0f0f95abba3810e514c40cd9d Mon Sep 17 00:00:00 2001 From: Clark Moody Date: Tue, 23 Jul 2019 11:13:15 -0500 Subject: Update test vectors with new version bytes --- bip-0049.mediawiki | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/bip-0049.mediawiki b/bip-0049.mediawiki index c7c383e..c40eab9 100644 --- a/bip-0049.mediawiki +++ b/bip-0049.mediawiki @@ -83,10 +83,11 @@ This BIP is not backwards compatible by design as described under [#consideratio
   masterseedWords = abandon abandon abandon abandon abandon abandon abandon abandon abandon abandon abandon about
-  masterseed = tprv8ZgxMBicQKsPe5YMU9gHen4Ez3ApihUfykaqUorj9t6FDqy3nP6eoXiAo2ssvpAjoLroQxHqr3R5nE3a5dU3DHTjTgJDd7zrbniJr6nrCzd (testnet)
+  masterseed = uprv8tXDerPXZ1QsVNjUJWTurs9kA1KGfKUAts74GCkcXtU8GwnH33GDRbNJpEqTvipfCyycARtQJhmdfWf8oKt41X9LL1zeD2pLsWmxEk3VAwd (testnet)
 
   // Account 0, root = m/49'/1'/0'
-  account0Xpriv = tprv8gRrNu65W2Msef2BdBSUgFdRTGzC8EwVXnV7UGS3faeXtuMVtGfEdidVeGbThs4ELEoayCAzZQ4uUji9DUiAs7erdVskqju7hrBcDvDsdbY (testnet)
+  account0Xpriv = uprv91G7gZkzehuMVxDJTYE6tLivdF8e4rvzSu1LFfKw3b2Qx1Aj8vpoFnHdfUZ3hmi9jsvPifmZ24RTN2KhwB8BfMLTVqaBReibyaFFcTP1s9n (testnet)
+  account0Xpub = upub5EFU65HtV5TeiSHmZZm7FUffBGy8UKeqp7vw43jYbvZPpoVsgU93oac7Wk3u6moKegAEWtGNF8DehrnHtv21XXEMYRUocHqguyjknFHYfgY (testnet)
 
   // Account 0, first receiving private key = m/49'/1'/0'/0/0
   account0recvPrivateKey = cULrpoZGXiuC19Uhvykx7NugygA3k86b3hmdCeyvHYQZSxojGyXJ
-- 
cgit v1.2.3


From 329df0b8368e04c92780cb039f1c5fb9f0cd386c Mon Sep 17 00:00:00 2001
From: Paul Sztorc 
Date: Tue, 23 Jul 2019 09:14:33 -0700
Subject: Update dates/links to merge

---
 bip-blind-merged-mining.mediawiki | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/bip-blind-merged-mining.mediawiki b/bip-blind-merged-mining.mediawiki
index 7f29f06..1a6e999 100644
--- a/bip-blind-merged-mining.mediawiki
+++ b/bip-blind-merged-mining.mediawiki
@@ -9,7 +9,7 @@
     Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-301
     Status: Draft
     Type: Standards Track
-    Created: 2017-10-24
+    Created: 2019-07-23
     License: BSD-2-Clause
 
 
@@ -87,7 +87,7 @@ To qualify for inclusion in a block, BMM requests are subject to the following r prevBlockRef is an integer that counts the number of "skips" one must take in the side:chain in order to find the current side:block's parent block. This value is zero unless the sidechain is reorganizing (or skipping over invalid sidechain blocks). If a side:node wants to orphan the most-recent N blocks, the value of the current block will be equal to N; in the block after that it will be back to zero. - + Above: Three blockchains, with different max length (small number), reorganization histories, and prevBlockRef numbers (larger numbers beneath blocks). The ordering given via each side:block's "prevSideBlockRef" will be isomorphic to an ordering given by each side:block's "prevSideHeaderHash" ("prevSideHeaderHash is the sidechain's equivalent of the mainchain's "prevBlockHash"). One can freely convert from one to the other. @@ -95,7 +95,7 @@ Above: Three blockchains, with different max length (small number), reorganizati To impose new requirements at the transaction level, we borrow the dummy vin & "flag" trick from SegWit style transactions. Unless all of the requirements for sidechain critical data transactions are met by the block it is included in, the transaction is invalid. With SegWit, this extra data is the SegWit signature stack, and the extra requirements are the signatures' locations and validity. In the sidechain BMM critical data transactions, the extra data is the (nSidechain, h\*) pair, which must meet the first two requirements (above) as well as the main:blocknumber, which must meet the third requirement (above). - + Above: A chart showing normal txns, SegWit txns, and CriticalData txns. The specific SegWit txn can be seen [http://srv1.yogh.io/#tx:id:D4A99AE93DF6EE3D4E42CE69338DFC1D06CCD9B198666E98FF0588057378D3D9 here]. -- cgit v1.2.3 From 6ca33dc63512a47f80321e2ae0222d5532571d7e Mon Sep 17 00:00:00 2001 From: Paul Sztorc Date: Tue, 23 Jul 2019 09:18:18 -0700 Subject: Rename bip-blind-merged-mining.mediawiki to bip-0301.mediawiki --- bip-0301.mediawiki | 234 ++++++++++++++++++++++++++++++++++++++ bip-blind-merged-mining.mediawiki | 234 -------------------------------------- 2 files changed, 234 insertions(+), 234 deletions(-) create mode 100644 bip-0301.mediawiki delete mode 100644 bip-blind-merged-mining.mediawiki diff --git a/bip-0301.mediawiki b/bip-0301.mediawiki new file mode 100644 index 0000000..1a6e999 --- /dev/null +++ b/bip-0301.mediawiki @@ -0,0 +1,234 @@ +
+
+    BIP: 301
+    Layer: Consensus (soft fork)
+    Title: Blind Merged Mining (Consensus layer)
+    Author: Paul Sztorc 
+            CryptAxe 
+    Comments-Summary: No comments yet.
+    Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-301
+    Status: Draft
+    Type: Standards Track
+    Created: 2019-07-23
+    License: BSD-2-Clause
+
+
+ + +==Abstract== + + +Blind Merged Mining (BMM) is a way of mining optional extension blocks (ie, "asymmetric sidechains"). BMM produces weak guarantees that the block is valid, for *any* arbitrary set of rules; and yet it does so without requiring miners to actually do any validation on the block whatsoever. + +BMM actually is a process that spans two or more chains. Here we focus on the modifications to mainchain Bitcoin. For an explanation of the "whole picture", please see [http://www.truthcoin.info/blog/blind-merged-mining/ this post]. + +Our goal here, is to allow mainchain miners to trustlessly "sell" the act of finding a sidechain block. + + +==Motivation== + + +Regular "Merged-Mining" (MM) allows miners to reuse their hashing work to secure other chains (for example, as in Namecoin). However, traditional MM has two drawbacks: + +# Miners must run a full node of the other chain. (This is because [while miners can effortlessly create the block] miners will not create a valid payment to themselves, unless the block that they MM is a valid one. Therefore, miners must assemble a *valid* block first, then MM it.) +# Miners are paid on the other chain, not on the regular BTC mainchain. For example, miners who MM Namecoin will earn NMC (and they will need to sell the NMC for BTC, before selling the BTC in order to pay for electricity). + +BMM addresses both shortcomings. + + +==Specification== + + +Note: This document uses the notation side:\* and main:\* in front of otherwise-ambiguous words (such as "block", "node", or "chain"), to distinguish the mainchain version from its sidechain counterpart. We also use "Simon" to refer to a Sidechain Full Node, and "Mary" to refer to a mainchain miner. + + +=== BMM Request === + +To buy the right to find a sidechain block, users broadcast BMM Requests. + +Here, these can take two forms. The first does not require the Lightning Network, but it does have new requirements for Immediate Expiration (see below). The second inherits Immediate Expiration from the Lightning Network itself, but requires extra preparation and a different/larger message. + +Both forms require that certain Critical Data will be committed to within the coinbase of the block that the transaction is included in (see BMM Accept). For the OnChain (non-Lightning) version, we have created a new extended serialization transaction type (very similar to how SegWit handles witness data (the witness stack)). + +==== Immediate Expiration ("Fill-or-Kill") ==== + +We would like to make special guarantees to the counterparties of this transaction. Specifically, instead of Simon making a "payment" to Mary, we prefer that Simon give Mary an "offer" (which she can either accept or decline). + +Crucially, we want Simon to safely make many offers to several different Mary's, in realtime (ie, quickly and off-chain). However, we ultimately want only one offer to be accepted, at most. In other words, we want Simon's offers to *immediately expire*. If only one offer can become a bona fide transaction, then Simon will feel comfortable making multiple offers all day long. Because all of the Simons are making many offers, the Marys collectively gain access to a large set of offers to choose from. + +==== OnChain BMM Request ==== + +OnChain BMMRs do not require the Lightning network, but they do have new requirements for validation. + +===== Structure ===== + +The following data is required: + +
+    32-bytes  - h* sideHeaderHash
+    ?~?-bytes - critical data extended serialization
+        3-bytes - 0x00bf00 identifying bytes
+        1-byte  - nSidechain
+        2-bytes - prevSideBlockRef
+        4-bytes - prevMainHeaderBytes
+
+ +sideHeaderHash comes from side:chain (side:nodes build side:blocks/headers). The identifying bytes are given here. nSidechain identifies which sidechain we are BMMing. By the time Blind Merged Mining can take place, it is known globally. + +prevBlockRef, is a little more complicated (next section). + +To qualify for inclusion in a block, BMM requests are subject to the following requirements: + +# Requests must match a corresponding "BMM Accept" (see last section of BIP). +# At most, only one Request is allowed in a main:block, per sidechain. In other words, if 700 users broadcast BMM Requests for sidechain #4, then the main:miner must choose one single Request to include. +# The 4-bytes of prevMainHeaderBytes must match the last four bytes of the previous main:blockheader. Thus, Simon's txns are only be valid for the current block, in the block history that he knows about (and therefore, the current sidechain history that he knows about). + +===== prevBlockRef ===== + +prevBlockRef is an integer that counts the number of "skips" one must take in the side:chain in order to find the current side:block's parent block. This value is zero unless the sidechain is reorganizing (or skipping over invalid sidechain blocks). If a side:node wants to orphan the most-recent N blocks, the value of the current block will be equal to N; in the block after that it will be back to zero. + + + +Above: Three blockchains, with different max length (small number), reorganization histories, and prevBlockRef numbers (larger numbers beneath blocks). The ordering given via each side:block's "prevSideBlockRef" will be isomorphic to an ordering given by each side:block's "prevSideHeaderHash" ("prevSideHeaderHash is the sidechain's equivalent of the mainchain's "prevBlockHash"). One can freely convert from one to the other. + +===== Extended Serialization ===== + +To impose new requirements at the transaction level, we borrow the dummy vin & "flag" trick from SegWit style transactions. Unless all of the requirements for sidechain critical data transactions are met by the block it is included in, the transaction is invalid. With SegWit, this extra data is the SegWit signature stack, and the extra requirements are the signatures' locations and validity. In the sidechain BMM critical data transactions, the extra data is the (nSidechain, h\*) pair, which must meet the first two requirements (above) as well as the main:blocknumber, which must meet the third requirement (above). + + + +Above: A chart showing normal txns, SegWit txns, and CriticalData txns. The specific SegWit txn can be seen [http://srv1.yogh.io/#tx:id:D4A99AE93DF6EE3D4E42CE69338DFC1D06CCD9B198666E98FF0588057378D3D9 here]. + +These types of transactions have slightly different mempool behavior, and should probably be kept in a second mempool. These txns are received, checked immediately, and if valid they are evaluated for inclusion in a block. If they are not able to be included in the specific requested block (if the block height requested has been surpassed by the chain tip), they are discarded. In fact, after any main:block is found, everything in this "second mempool" can be discarded as new payments will be created immediately for the next block height. (This includes cases where the blockchain reorganizes.) There is no re-evaluation of the txns in this mempool ever -- they are evaluated once and then either included or discarded. They never need to be rescanned. + +Interestingly, these payments will *always* be directed to main:miners from non-main:miners. Therefore, non-mining full nodes do not need to keep them in any mempool at all. Non-miner nodes can just wait for a block to be found, and check the txn then. These transactions more resemble a stock market's pit trade-offers (in contrast, regular Bitcoin txns are more like paper checks). + +==== Lightning BMM Request ==== + +Lightning BMMRs require Simons to have a LN-channel pathways open with Marys. This may not always be practical (or even possible), especially today. + +LN txns cannot make use of prevSideBlockRef, as no one knows for sure when (or if) they will be broadcast on-chain. Instead, they must use prevSideBlockHash. But they otherwise require the same data: + +
   
+    4-bytes - Message header (0xD0520C6E)   
+    1-byte - sidechain number
+    32-bytes  - h* side:block hash  
+    32-bytes  - prevSideBlockHash   
+
+ +Notice that, in OnChain BMMRs, Simon could reuse the same h\* all he wanted, because only one OnChain BMMR could be included per main:block per sidechain. However, on the LN no such rule can be enforced, as the goal is to push everything off-chain and include *zero* txns. So, we will never know what the Requests were, or how many had an effect on anything. + +Therefore, Simon will need to ensure that he '''gives each Mary a different h\*'''. Simon can easily do this, as he controls the side:block's contents and can simply increment a side:nonce -- this changes the side:block, and changes its hash (ie, changes h\*). + +With a unique h\* per Mary (or, more precisely, per channel), and at most 1 h\* making it into a block (per sidechain), Simon can ensure that he is charged, at most, one time. + +That's probably confusing, so here is an example, in which: Simon starts with 13 BTC, Mary starts with 40 BTC, the side:block's tx-fees currently total 7.1 BTC, and Simon is keeping 0.1 BTC for himself and paying 7 BTC to Mary. + +We start with (I): + +
+    Simon 13 in, Mary 40 in ; 53 in total
+        Simon's version [signed by Mary]
+            13 ; to Simon if TimeLock=over; OR to Mary if SimonSig
+            40 ; to Mary
+        Mary's version [signed by Simon]
+            40 ; to me if TimeLock=over; OR to Simon if MarySig
+            13 ; to Simon
+
+ + +And both parties move, from there to (II): + +
+    Simon 13 in, Mary 40 in ; 53 in total
+        Simon's version [signed by Mary]
+            6 ; to Simon if TimeLock=over; OR to Mary if SimonSig
+            40 ; to Mary
+            7 ; to Mary if critical data requirements met; OR to Simon if LongTimeLock=over
+        Mary's version [signed by Simon]
+            40 ; to Mary if TimeLock=over; OR to Simon if MarySig
+            6 ; to Simon
+            7 ; to Mary if critical data requirements met; OR to Simon if LongTimeLock=over
+
+ + +From here, if the h\* side:block in question is BMMed, they can proceed to (III): + +
+    Simon 13 in, Mary 40 in ; 53 in total
+        Simon's version [signed by Mary]
+            6 ; to Simon if TimeLock=over; OR to Mary if SimonSig
+            47 ; to Mary
+        Mary's version [signed by Simon]
+            47 ; to me if TimeLock=over; OR to Simon if MarySig
+            6 ; to Simon
+
+ +If Simon proceeds immediately, he removes Mary's incentive to care about blocks being built on this side:block. If Simon's side:block is orphaned, he loses his 7 BTC. Simon can either play it safe, and wait for (for example) 100 side:blocks before moving on (ie, before moving on to the third LN txn, above); or else Simon can take the risk if he feels comfortable with it. + +If the h\* side:block is not found, then (II) and (III) are basically equivalent to each other. Simon and Mary could jointly reconstruct (I) and go back there, or they could proceed to a new version of II (with a different h\*, trying again with new side:block in the next main:block). + +Now that we have described Requests, we can describe how they are accepted. + +=== BMM Accept === + +For each BMM Request that a main:miner "accepts", main:miners must place an OP Return output into their main:coinbase txn. (We've changed the tx-standardness policy to allow multiple OP_RETURNs.) + +The following data is required in the "accept" OP_RETURN output: + 1-byte - OP_RETURN (0x6a) + 1-byte - Push the following 36 bytes (0x24) + 4-bytes - Message header (0xD3407053) + 32-bytes - h* + ~5-bytes - BMM identifier bytes + + +[https://github.com/DriveNetTESTDRIVE/DriveNet/blob/564516653c1d876429382971a011f5f6119f7eb4/src/validation.cpp#L3377-L3470 Link to code]. + +If these OP_RETURN outputs are not present, then no BMM Requests have been accepted. (And, if they are not accepted, then they cannot be included in a main:block.) + + +==Backward compatibility== + +As a soft fork, older software will continue to operate without modification. As stated above, BMM asks nodes to track a set of ordered hashes, and to allow miners to "sell" the act of finding a sidechain block. Non-upgraded nodes will notice that this activity (specifically: data in coinbases, and new txns that have OP Returns and interesting message headers) is now taking place, but they will not understand any of it. Much like P2SH or a new OP Code, these old users will not be directly affected by the fork, as they will have no expectations of receiving payments of this kind. + +(As a matter of fact, the only people receiving money here all happen to be miners. So there is less reason than ever to expect compatibility problems.) + + + + +==Deployment== + +This BIP will be deployed by "version bits" BIP9 with the name "blindmm" and using bit 4. + +
+// Deployment of Drivechains (BIPX, BIPY)
+consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].bit = 4;
+consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].nStartTime = 1579072881; // January 15th, 2020.
+consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].nTimeout = 1610695281; // January 15th, 2021.
+
+ + +==Reference Implementation== + +See: https://github.com/DriveNetTESTDRIVE/DriveNet + +Also, for interest, see an example sidechain here: https://github.com/drivechain-project/bitcoin/tree/sidechainBMM + + +==References== + + +* http://www.drivechain.info/literature/index.html +* http://www.truthcoin.info/blog/blind-merged-mining/ +* https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-July/014789.html +* http://www.truthcoin.info/images/bmm-outline.txt + + +==Thanks== + +Thanks to everyone who contributed to the discussion, especially: ZmnSCPxj, Adam Back, Peter Todd, Dan Anderson, Sergio Demian Lerner, Matt Corallo, Sjors Provoost, Tier Nolan, Erik Aronesty, Jason Dreyzehner, Joe Miyamoto, Chris Stewart, Ben Goldhaber. + + +==Copyright== + +This BIP is licensed under the BSD 2-clause license. diff --git a/bip-blind-merged-mining.mediawiki b/bip-blind-merged-mining.mediawiki deleted file mode 100644 index 1a6e999..0000000 --- a/bip-blind-merged-mining.mediawiki +++ /dev/null @@ -1,234 +0,0 @@ -
-
-    BIP: 301
-    Layer: Consensus (soft fork)
-    Title: Blind Merged Mining (Consensus layer)
-    Author: Paul Sztorc 
-            CryptAxe 
-    Comments-Summary: No comments yet.
-    Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-301
-    Status: Draft
-    Type: Standards Track
-    Created: 2019-07-23
-    License: BSD-2-Clause
-
-
- - -==Abstract== - - -Blind Merged Mining (BMM) is a way of mining optional extension blocks (ie, "asymmetric sidechains"). BMM produces weak guarantees that the block is valid, for *any* arbitrary set of rules; and yet it does so without requiring miners to actually do any validation on the block whatsoever. - -BMM actually is a process that spans two or more chains. Here we focus on the modifications to mainchain Bitcoin. For an explanation of the "whole picture", please see [http://www.truthcoin.info/blog/blind-merged-mining/ this post]. - -Our goal here, is to allow mainchain miners to trustlessly "sell" the act of finding a sidechain block. - - -==Motivation== - - -Regular "Merged-Mining" (MM) allows miners to reuse their hashing work to secure other chains (for example, as in Namecoin). However, traditional MM has two drawbacks: - -# Miners must run a full node of the other chain. (This is because [while miners can effortlessly create the block] miners will not create a valid payment to themselves, unless the block that they MM is a valid one. Therefore, miners must assemble a *valid* block first, then MM it.) -# Miners are paid on the other chain, not on the regular BTC mainchain. For example, miners who MM Namecoin will earn NMC (and they will need to sell the NMC for BTC, before selling the BTC in order to pay for electricity). - -BMM addresses both shortcomings. - - -==Specification== - - -Note: This document uses the notation side:\* and main:\* in front of otherwise-ambiguous words (such as "block", "node", or "chain"), to distinguish the mainchain version from its sidechain counterpart. We also use "Simon" to refer to a Sidechain Full Node, and "Mary" to refer to a mainchain miner. - - -=== BMM Request === - -To buy the right to find a sidechain block, users broadcast BMM Requests. - -Here, these can take two forms. The first does not require the Lightning Network, but it does have new requirements for Immediate Expiration (see below). The second inherits Immediate Expiration from the Lightning Network itself, but requires extra preparation and a different/larger message. - -Both forms require that certain Critical Data will be committed to within the coinbase of the block that the transaction is included in (see BMM Accept). For the OnChain (non-Lightning) version, we have created a new extended serialization transaction type (very similar to how SegWit handles witness data (the witness stack)). - -==== Immediate Expiration ("Fill-or-Kill") ==== - -We would like to make special guarantees to the counterparties of this transaction. Specifically, instead of Simon making a "payment" to Mary, we prefer that Simon give Mary an "offer" (which she can either accept or decline). - -Crucially, we want Simon to safely make many offers to several different Mary's, in realtime (ie, quickly and off-chain). However, we ultimately want only one offer to be accepted, at most. In other words, we want Simon's offers to *immediately expire*. If only one offer can become a bona fide transaction, then Simon will feel comfortable making multiple offers all day long. Because all of the Simons are making many offers, the Marys collectively gain access to a large set of offers to choose from. - -==== OnChain BMM Request ==== - -OnChain BMMRs do not require the Lightning network, but they do have new requirements for validation. - -===== Structure ===== - -The following data is required: - -
-    32-bytes  - h* sideHeaderHash
-    ?~?-bytes - critical data extended serialization
-        3-bytes - 0x00bf00 identifying bytes
-        1-byte  - nSidechain
-        2-bytes - prevSideBlockRef
-        4-bytes - prevMainHeaderBytes
-
- -sideHeaderHash comes from side:chain (side:nodes build side:blocks/headers). The identifying bytes are given here. nSidechain identifies which sidechain we are BMMing. By the time Blind Merged Mining can take place, it is known globally. - -prevBlockRef, is a little more complicated (next section). - -To qualify for inclusion in a block, BMM requests are subject to the following requirements: - -# Requests must match a corresponding "BMM Accept" (see last section of BIP). -# At most, only one Request is allowed in a main:block, per sidechain. In other words, if 700 users broadcast BMM Requests for sidechain #4, then the main:miner must choose one single Request to include. -# The 4-bytes of prevMainHeaderBytes must match the last four bytes of the previous main:blockheader. Thus, Simon's txns are only be valid for the current block, in the block history that he knows about (and therefore, the current sidechain history that he knows about). - -===== prevBlockRef ===== - -prevBlockRef is an integer that counts the number of "skips" one must take in the side:chain in order to find the current side:block's parent block. This value is zero unless the sidechain is reorganizing (or skipping over invalid sidechain blocks). If a side:node wants to orphan the most-recent N blocks, the value of the current block will be equal to N; in the block after that it will be back to zero. - - - -Above: Three blockchains, with different max length (small number), reorganization histories, and prevBlockRef numbers (larger numbers beneath blocks). The ordering given via each side:block's "prevSideBlockRef" will be isomorphic to an ordering given by each side:block's "prevSideHeaderHash" ("prevSideHeaderHash is the sidechain's equivalent of the mainchain's "prevBlockHash"). One can freely convert from one to the other. - -===== Extended Serialization ===== - -To impose new requirements at the transaction level, we borrow the dummy vin & "flag" trick from SegWit style transactions. Unless all of the requirements for sidechain critical data transactions are met by the block it is included in, the transaction is invalid. With SegWit, this extra data is the SegWit signature stack, and the extra requirements are the signatures' locations and validity. In the sidechain BMM critical data transactions, the extra data is the (nSidechain, h\*) pair, which must meet the first two requirements (above) as well as the main:blocknumber, which must meet the third requirement (above). - - - -Above: A chart showing normal txns, SegWit txns, and CriticalData txns. The specific SegWit txn can be seen [http://srv1.yogh.io/#tx:id:D4A99AE93DF6EE3D4E42CE69338DFC1D06CCD9B198666E98FF0588057378D3D9 here]. - -These types of transactions have slightly different mempool behavior, and should probably be kept in a second mempool. These txns are received, checked immediately, and if valid they are evaluated for inclusion in a block. If they are not able to be included in the specific requested block (if the block height requested has been surpassed by the chain tip), they are discarded. In fact, after any main:block is found, everything in this "second mempool" can be discarded as new payments will be created immediately for the next block height. (This includes cases where the blockchain reorganizes.) There is no re-evaluation of the txns in this mempool ever -- they are evaluated once and then either included or discarded. They never need to be rescanned. - -Interestingly, these payments will *always* be directed to main:miners from non-main:miners. Therefore, non-mining full nodes do not need to keep them in any mempool at all. Non-miner nodes can just wait for a block to be found, and check the txn then. These transactions more resemble a stock market's pit trade-offers (in contrast, regular Bitcoin txns are more like paper checks). - -==== Lightning BMM Request ==== - -Lightning BMMRs require Simons to have a LN-channel pathways open with Marys. This may not always be practical (or even possible), especially today. - -LN txns cannot make use of prevSideBlockRef, as no one knows for sure when (or if) they will be broadcast on-chain. Instead, they must use prevSideBlockHash. But they otherwise require the same data: - -
   
-    4-bytes - Message header (0xD0520C6E)   
-    1-byte - sidechain number
-    32-bytes  - h* side:block hash  
-    32-bytes  - prevSideBlockHash   
-
- -Notice that, in OnChain BMMRs, Simon could reuse the same h\* all he wanted, because only one OnChain BMMR could be included per main:block per sidechain. However, on the LN no such rule can be enforced, as the goal is to push everything off-chain and include *zero* txns. So, we will never know what the Requests were, or how many had an effect on anything. - -Therefore, Simon will need to ensure that he '''gives each Mary a different h\*'''. Simon can easily do this, as he controls the side:block's contents and can simply increment a side:nonce -- this changes the side:block, and changes its hash (ie, changes h\*). - -With a unique h\* per Mary (or, more precisely, per channel), and at most 1 h\* making it into a block (per sidechain), Simon can ensure that he is charged, at most, one time. - -That's probably confusing, so here is an example, in which: Simon starts with 13 BTC, Mary starts with 40 BTC, the side:block's tx-fees currently total 7.1 BTC, and Simon is keeping 0.1 BTC for himself and paying 7 BTC to Mary. - -We start with (I): - -
-    Simon 13 in, Mary 40 in ; 53 in total
-        Simon's version [signed by Mary]
-            13 ; to Simon if TimeLock=over; OR to Mary if SimonSig
-            40 ; to Mary
-        Mary's version [signed by Simon]
-            40 ; to me if TimeLock=over; OR to Simon if MarySig
-            13 ; to Simon
-
- - -And both parties move, from there to (II): - -
-    Simon 13 in, Mary 40 in ; 53 in total
-        Simon's version [signed by Mary]
-            6 ; to Simon if TimeLock=over; OR to Mary if SimonSig
-            40 ; to Mary
-            7 ; to Mary if critical data requirements met; OR to Simon if LongTimeLock=over
-        Mary's version [signed by Simon]
-            40 ; to Mary if TimeLock=over; OR to Simon if MarySig
-            6 ; to Simon
-            7 ; to Mary if critical data requirements met; OR to Simon if LongTimeLock=over
-
- - -From here, if the h\* side:block in question is BMMed, they can proceed to (III): - -
-    Simon 13 in, Mary 40 in ; 53 in total
-        Simon's version [signed by Mary]
-            6 ; to Simon if TimeLock=over; OR to Mary if SimonSig
-            47 ; to Mary
-        Mary's version [signed by Simon]
-            47 ; to me if TimeLock=over; OR to Simon if MarySig
-            6 ; to Simon
-
- -If Simon proceeds immediately, he removes Mary's incentive to care about blocks being built on this side:block. If Simon's side:block is orphaned, he loses his 7 BTC. Simon can either play it safe, and wait for (for example) 100 side:blocks before moving on (ie, before moving on to the third LN txn, above); or else Simon can take the risk if he feels comfortable with it. - -If the h\* side:block is not found, then (II) and (III) are basically equivalent to each other. Simon and Mary could jointly reconstruct (I) and go back there, or they could proceed to a new version of II (with a different h\*, trying again with new side:block in the next main:block). - -Now that we have described Requests, we can describe how they are accepted. - -=== BMM Accept === - -For each BMM Request that a main:miner "accepts", main:miners must place an OP Return output into their main:coinbase txn. (We've changed the tx-standardness policy to allow multiple OP_RETURNs.) - -The following data is required in the "accept" OP_RETURN output: - 1-byte - OP_RETURN (0x6a) - 1-byte - Push the following 36 bytes (0x24) - 4-bytes - Message header (0xD3407053) - 32-bytes - h* - ~5-bytes - BMM identifier bytes - - -[https://github.com/DriveNetTESTDRIVE/DriveNet/blob/564516653c1d876429382971a011f5f6119f7eb4/src/validation.cpp#L3377-L3470 Link to code]. - -If these OP_RETURN outputs are not present, then no BMM Requests have been accepted. (And, if they are not accepted, then they cannot be included in a main:block.) - - -==Backward compatibility== - -As a soft fork, older software will continue to operate without modification. As stated above, BMM asks nodes to track a set of ordered hashes, and to allow miners to "sell" the act of finding a sidechain block. Non-upgraded nodes will notice that this activity (specifically: data in coinbases, and new txns that have OP Returns and interesting message headers) is now taking place, but they will not understand any of it. Much like P2SH or a new OP Code, these old users will not be directly affected by the fork, as they will have no expectations of receiving payments of this kind. - -(As a matter of fact, the only people receiving money here all happen to be miners. So there is less reason than ever to expect compatibility problems.) - - - - -==Deployment== - -This BIP will be deployed by "version bits" BIP9 with the name "blindmm" and using bit 4. - -
-// Deployment of Drivechains (BIPX, BIPY)
-consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].bit = 4;
-consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].nStartTime = 1579072881; // January 15th, 2020.
-consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].nTimeout = 1610695281; // January 15th, 2021.
-
- - -==Reference Implementation== - -See: https://github.com/DriveNetTESTDRIVE/DriveNet - -Also, for interest, see an example sidechain here: https://github.com/drivechain-project/bitcoin/tree/sidechainBMM - - -==References== - - -* http://www.drivechain.info/literature/index.html -* http://www.truthcoin.info/blog/blind-merged-mining/ -* https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-July/014789.html -* http://www.truthcoin.info/images/bmm-outline.txt - - -==Thanks== - -Thanks to everyone who contributed to the discussion, especially: ZmnSCPxj, Adam Back, Peter Todd, Dan Anderson, Sergio Demian Lerner, Matt Corallo, Sjors Provoost, Tier Nolan, Erik Aronesty, Jason Dreyzehner, Joe Miyamoto, Chris Stewart, Ben Goldhaber. - - -==Copyright== - -This BIP is licensed under the BSD 2-clause license. -- cgit v1.2.3 From 1ff5d5d8253191bead9ffc3924c668292747777b Mon Sep 17 00:00:00 2001 From: Paul Sztorc Date: Tue, 23 Jul 2019 09:20:44 -0700 Subject: rename folder --- bip-0301/images.txt | 1 + bip-blind-merged-mining/images.txt | 1 - 2 files changed, 1 insertion(+), 1 deletion(-) create mode 100644 bip-0301/images.txt delete mode 100644 bip-blind-merged-mining/images.txt diff --git a/bip-0301/images.txt b/bip-0301/images.txt new file mode 100644 index 0000000..2fbbf63 --- /dev/null +++ b/bip-0301/images.txt @@ -0,0 +1 @@ +Images used as reference in the documentation. diff --git a/bip-blind-merged-mining/images.txt b/bip-blind-merged-mining/images.txt deleted file mode 100644 index 2fbbf63..0000000 --- a/bip-blind-merged-mining/images.txt +++ /dev/null @@ -1 +0,0 @@ -Images used as reference in the documentation. -- cgit v1.2.3 From 42e96858d5e13793abbf9820ed38720d64783cda Mon Sep 17 00:00:00 2001 From: Paul Sztorc Date: Tue, 23 Jul 2019 09:26:15 -0700 Subject: move images --- bip-0301/bmm-dots-examples.png | Bin 0 -> 41116 bytes bip-0301/witness-vs-critical.png | Bin 0 -> 268309 bytes 2 files changed, 0 insertions(+), 0 deletions(-) create mode 100644 bip-0301/bmm-dots-examples.png create mode 100644 bip-0301/witness-vs-critical.png diff --git a/bip-0301/bmm-dots-examples.png b/bip-0301/bmm-dots-examples.png new file mode 100644 index 0000000..70f11f6 Binary files /dev/null and b/bip-0301/bmm-dots-examples.png differ diff --git a/bip-0301/witness-vs-critical.png b/bip-0301/witness-vs-critical.png new file mode 100644 index 0000000..79c84b1 Binary files /dev/null and b/bip-0301/witness-vs-critical.png differ -- cgit v1.2.3 From 69670c9da20ac696ad696ba07e58093796a8c5bb Mon Sep 17 00:00:00 2001 From: Paul Sztorc Date: Tue, 23 Jul 2019 09:26:45 -0700 Subject: cleanup 1 --- bip-blind-merged-mining/bmm-dots-examples.png | Bin 41116 -> 0 bytes 1 file changed, 0 insertions(+), 0 deletions(-) delete mode 100644 bip-blind-merged-mining/bmm-dots-examples.png diff --git a/bip-blind-merged-mining/bmm-dots-examples.png b/bip-blind-merged-mining/bmm-dots-examples.png deleted file mode 100644 index 70f11f6..0000000 Binary files a/bip-blind-merged-mining/bmm-dots-examples.png and /dev/null differ -- cgit v1.2.3 From 4cd4e53752496e155243693fffbf79c7ed2daa9d Mon Sep 17 00:00:00 2001 From: Paul Sztorc Date: Tue, 23 Jul 2019 09:26:54 -0700 Subject: cleanup 2 --- bip-blind-merged-mining/witness-vs-critical.png | Bin 268309 -> 0 bytes 1 file changed, 0 insertions(+), 0 deletions(-) delete mode 100644 bip-blind-merged-mining/witness-vs-critical.png diff --git a/bip-blind-merged-mining/witness-vs-critical.png b/bip-blind-merged-mining/witness-vs-critical.png deleted file mode 100644 index 79c84b1..0000000 Binary files a/bip-blind-merged-mining/witness-vs-critical.png and /dev/null differ -- cgit v1.2.3 From fd4aab24a4383480edbaa750834337292dc03004 Mon Sep 17 00:00:00 2001 From: DanielWeigl Date: Wed, 24 Jul 2019 10:28:09 +0200 Subject: =?UTF-8?q?Set=20Status=20f=C3=BCr=20BIP49=20to=20'Final'?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit it gets used by electrum, trezor and other wallets - Reflect bip49 status change to active also in README.mediawiki - Also fix email address (private one is more stable, old one isnt monitored anymore) --- README.mediawiki | 4 ++-- bip-0049.mediawiki | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/README.mediawiki b/README.mediawiki index fc92b63..9425f7c 100644 --- a/README.mediawiki +++ b/README.mediawiki @@ -258,13 +258,13 @@ Those proposing changes should consider that ultimately consent may rest with th | Justus Ranvier | Informational | Draft -|- +|- style="background-color: #cfffcf" | [[bip-0049.mediawiki|49]] | Applications | Derivation scheme for P2WPKH-nested-in-P2SH based accounts | Daniel Weigl | Informational -| Draft +| Final |- style="background-color: #cfffcf" | [[bip-0050.mediawiki|50]] | diff --git a/bip-0049.mediawiki b/bip-0049.mediawiki index 74645a1..cc3ea1d 100644 --- a/bip-0049.mediawiki +++ b/bip-0049.mediawiki @@ -2,10 +2,10 @@ BIP: 49 Layer: Applications Title: Derivation scheme for P2WPKH-nested-in-P2SH based accounts - Author: Daniel Weigl + Author: Daniel Weigl Comments-Summary: No comments yet. Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0049 - Status: Draft + Status: Final Type: Informational Created: 2016-05-19 License: PD -- cgit v1.2.3 From 6a36d8e1b4e763bf42c8403f598e0bf07048408f Mon Sep 17 00:00:00 2001 From: Andrew Chow Date: Thu, 25 Jul 2019 13:37:06 -0400 Subject: BIP 174: global xpubs serialization test vector --- bip-0174.mediawiki | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/bip-0174.mediawiki b/bip-0174.mediawiki index b4a6407..c0fb559 100644 --- a/bip-0174.mediawiki +++ b/bip-0174.mediawiki @@ -546,6 +546,10 @@ The following are valid PSBTs: ** Bytes in Hex:
70736274ff0100550200000001279a2323a5dfb51fc45f220fa58b0fc13e1e3342792a85d7e36cd6333b5cbc390000000000ffffffff01a05aea0b000000001976a914ffe9c0061097cc3b636f2cb0460fa4fc427d2b4588ac0000000000010120955eea0b0000000017a9146345200f68d189e1adc0df1c4d16ea8f14c0dbeb87220203b1341ccba7683b6af4f1238cd6e97e7167d569fac47f1e48d47541844355bd4646304302200424b58effaaa694e1559ea5c93bbfd4a89064224055cdf070b6771469442d07021f5c8eb0fea6516d60b8acb33ad64ede60e8785bfb3aa94b99bdf86151db9a9a010104220020771fd18ad459666dd49f3d564e3dbc42f4c84774e360ada16816a8ed488d5681010547522103b1341ccba7683b6af4f1238cd6e97e7167d569fac47f1e48d47541844355bd462103de55d1e1dac805e3f8a58c1fbf9b94c02f3dbaafe127fefca4995f26f82083bd52ae220603b1341ccba7683b6af4f1238cd6e97e7167d569fac47f1e48d47541844355bd4610b4a6ba67000000800000008004000080220603de55d1e1dac805e3f8a58c1fbf9b94c02f3dbaafe127fefca4995f26f82083bd10b4a6ba670000008000000080050000800000
** Base64 String:
cHNidP8BAFUCAAAAASeaIyOl37UfxF8iD6WLD8E+HjNCeSqF1+Ns1jM7XLw5AAAAAAD/////AaBa6gsAAAAAGXapFP/pwAYQl8w7Y28ssEYPpPxCfStFiKwAAAAAAAEBIJVe6gsAAAAAF6kUY0UgD2jRieGtwN8cTRbqjxTA2+uHIgIDsTQcy6doO2r08SOM1ul+cWfVafrEfx5I1HVBhENVvUZGMEMCIAQktY7/qqaU4VWepck7v9SokGQiQFXN8HC2dxRpRC0HAh9cjrD+plFtYLisszrWTt5g6Hhb+zqpS5m9+GFR25qaAQEEIgAgdx/RitRZZm3Unz1WTj28QvTIR3TjYK2haBao7UiNVoEBBUdSIQOxNBzLp2g7avTxI4zW6X5xZ9Vp+sR/HkjUdUGEQ1W9RiED3lXR4drIBeP4pYwfv5uUwC89uq/hJ/78pJlfJvggg71SriIGA7E0HMunaDtq9PEjjNbpfnFn1Wn6xH8eSNR1QYRDVb1GELSmumcAAACAAAAAgAQAAIAiBgPeVdHh2sgF4/iljB+/m5TALz26r+En/vykmV8m+CCDvRC0prpnAAAAgAAAAIAFAACAAAA=
+* Case: PSBT with one P2WSH input of a 2-of-2 multisig. witnessScript, keypaths, and global xpubs are available. Contains no signatures. Outputs filled. +** Bytes in Hex:
70736274ff01005202000000019dfc6628c26c5899fe1bd3dc338665bfd55d7ada10f6220973df2d386dec12760100000000ffffffff01f03dcd1d000000001600147b3a00bfdc14d27795c2b74901d09da6ef133579000000004f01043587cf02da3fd0088000000097048b1ad0445b1ec8275517727c87b4e4ebc18a203ffa0f94c01566bd38e9000351b743887ee1d40dc32a6043724f2d6459b3b5a4d73daec8fbae0472f3bc43e20cd90c6a4fae000080000000804f01043587cf02da3fd00880000001b90452427139cd78c2cff2444be353cd58605e3e513285e528b407fae3f6173503d30a5e97c8adbc557dac2ad9a7e39c1722ebac69e668b6f2667cc1d671c83cab0cd90c6a4fae000080010000800001012b0065cd1d000000002200202c5486126c4978079a814e13715d65f36459e4d6ccaded266d0508645bafa6320105475221029da12cdb5b235692b91536afefe5c91c3ab9473d8e43b533836ab456299c88712103372b34234ed7cf9c1fea5d05d441557927be9542b162eb02e1ab2ce80224c00b52ae2206029da12cdb5b235692b91536afefe5c91c3ab9473d8e43b533836ab456299c887110d90c6a4fae0000800000008000000000220603372b34234ed7cf9c1fea5d05d441557927be9542b162eb02e1ab2ce80224c00b10d90c6a4fae0000800100008000000000002202039eff1f547a1d5f92dfa2ba7af6ac971a4bd03ba4a734b03156a256b8ad3a1ef910ede45cc500000080000000800100008000
+** Base64 String:
cHNidP8BAFICAAAAAZ38ZijCbFiZ/hvT3DOGZb/VXXraEPYiCXPfLTht7BJ2AQAAAAD/////AfA9zR0AAAAAFgAUezoAv9wU0neVwrdJAdCdpu8TNXkAAAAATwEENYfPAto/0AiAAAAAlwSLGtBEWx7IJ1UXcnyHtOTrwYogP/oPlMAVZr046QADUbdDiH7h1A3DKmBDck8tZFmztaTXPa7I+64EcvO8Q+IM2QxqT64AAIAAAACATwEENYfPAto/0AiAAAABuQRSQnE5zXjCz/JES+NTzVhgXj5RMoXlKLQH+uP2FzUD0wpel8itvFV9rCrZp+OcFyLrrGnmaLbyZnzB1nHIPKsM2QxqT64AAIABAACAAAEBKwBlzR0AAAAAIgAgLFSGEmxJeAeagU4TcV1l82RZ5NbMre0mbQUIZFuvpjIBBUdSIQKdoSzbWyNWkrkVNq/v5ckcOrlHPY5DtTODarRWKZyIcSEDNys0I07Xz5wf6l0F1EFVeSe+lUKxYusC4ass6AIkwAtSriIGAp2hLNtbI1aSuRU2r+/lyRw6uUc9jkO1M4NqtFYpnIhxENkMak+uAACAAAAAgAAAAAAiBgM3KzQjTtfPnB/qXQXUQVV5J76VQrFi6wLhqyzoAiTACxDZDGpPrgAAgAEAAIAAAAAAACICA57/H1R6HV+S36K6evaslxpL0DukpzSwMVaiVritOh75EO3kXMUAAACAAAAAgAEAAIAA
+ * Case: PSBT with unknown types in the inputs. ** Bytes in Hex:
70736274ff01003f0200000001ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000000000ffffffff010000000000000000036a010000000000000a0f0102030405060708090f0102030405060708090a0b0c0d0e0f0000
** Base64 String:
cHNidP8BAD8CAAAAAf//////////////////////////////////////////AAAAAAD/////AQAAAAAAAAAAA2oBAAAAAAAACg8BAgMEBQYHCAkPAQIDBAUGBwgJCgsMDQ4PAAA=
-- cgit v1.2.3 From d832c51e82a321e4721cb9e8cc3e69e8e20aa57f Mon Sep 17 00:00:00 2001 From: "nicolas.dorier" Date: Thu, 25 Jul 2019 23:00:16 +0900 Subject: bip174: Add test vector for global xpub --- bip-0174.mediawiki | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/bip-0174.mediawiki b/bip-0174.mediawiki index b4a6407..3999b52 100644 --- a/bip-0174.mediawiki +++ b/bip-0174.mediawiki @@ -550,6 +550,10 @@ The following are valid PSBTs: ** Bytes in Hex:
70736274ff01003f0200000001ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000000000ffffffff010000000000000000036a010000000000000a0f0102030405060708090f0102030405060708090a0b0c0d0e0f0000
** Base64 String:
cHNidP8BAD8CAAAAAf//////////////////////////////////////////AAAAAAD/////AQAAAAAAAAAAA2oBAAAAAAAACg8BAgMEBQYHCAkPAQIDBAUGBwgJCgsMDQ4PAAA=
+* Case: PSBT with `PSBT_GLOBAL_XPUB`. +** Bytes in Hex:
70736274ff01009d0100000002710ea76ab45c5cb6438e607e59cc037626981805ae9e0dfd9089012abb0be5350100000000ffffffff190994d6a8b3c8c82ccbcfb2fba4106aa06639b872a8d447465c0d42588d6d670000000000ffffffff0200e1f505000000001976a914b6bc2c0ee5655a843d79afedd0ccc3f7dd64340988ac605af405000000001600141188ef8e4ce0449eaac8fb141cbf5a1176e6a088000000004f010488b21e039e530cac800000003dbc8a5c9769f031b17e77fea1518603221a18fd18f2b9a54c6c8c1ac75cbc3502f230584b155d1c7f1cd45120a653c48d650b431b67c5b2c13f27d7142037c1691027569c503100008000000080000000800001011f00e1f5050000000016001433b982f91b28f160c920b4ab95e58ce50dda3a4a220203309680f33c7de38ea6a47cd4ecd66f1f5a49747c6ffb8808ed09039243e3ad5c47304402202d704ced830c56a909344bd742b6852dccd103e963bae92d38e75254d2bb424502202d86c437195df46c0ceda084f2a291c3da2d64070f76bf9b90b195e7ef28f77201220603309680f33c7de38ea6a47cd4ecd66f1f5a49747c6ffb8808ed09039243e3ad5c1827569c5031000080000000800000008000000000010000000001011f00e1f50500000000160014388fb944307eb77ef45197d0b0b245e079f011de220202c777161f73d0b7c72b9ee7bde650293d13f095bc7656ad1f525da5fd2e10b11047304402204cb1fb5f869c942e0e26100576125439179ae88dca8a9dc3ba08f7953988faa60220521f49ca791c27d70e273c9b14616985909361e25be274ea200d7e08827e514d01220602c777161f73d0b7c72b9ee7bde650293d13f095bc7656ad1f525da5fd2e10b1101827569c5031000080000000800000008000000000000000000000220202d20ca502ee289686d21815bd43a80637b0698e1fbcdbe4caed445f6c1a0a90ef1827569c50310000800000008000000080000000000400000000
+** Base64 String:
cHNidP8BAJ0BAAAAAnEOp2q0XFy2Q45gflnMA3YmmBgFrp4N/ZCJASq7C+U1AQAAAAD/////GQmU1qizyMgsy8+y+6QQaqBmObhyqNRHRlwNQliNbWcAAAAAAP////8CAOH1BQAAAAAZdqkUtrwsDuVlWoQ9ea/t0MzD991kNAmIrGBa9AUAAAAAFgAUEYjvjkzgRJ6qyPsUHL9aEXbmoIgAAAAATwEEiLIeA55TDKyAAAAAPbyKXJdp8DGxfnf+oVGGAyIaGP0Y8rmlTGyMGsdcvDUC8jBYSxVdHH8c1FEgplPEjWULQxtnxbLBPyfXFCA3wWkQJ1acUDEAAIAAAACAAAAAgAABAR8A4fUFAAAAABYAFDO5gvkbKPFgySC0q5XljOUN2jpKIgIDMJaA8zx9446mpHzU7NZvH1pJdHxv+4gI7QkDkkPjrVxHMEQCIC1wTO2DDFapCTRL10K2hS3M0QPpY7rpLTjnUlTSu0JFAiAthsQ3GV30bAztoITyopHD2i1kBw92v5uQsZXn7yj3cgEiBgMwloDzPH3jjqakfNTs1m8fWkl0fG/7iAjtCQOSQ+OtXBgnVpxQMQAAgAAAAIAAAACAAAAAAAEAAAAAAQEfAOH1BQAAAAAWABQ4j7lEMH63fvRRl9CwskXgefAR3iICAsd3Fh9z0LfHK57nveZQKT0T8JW8dlatH1Jdpf0uELEQRzBEAiBMsftfhpyULg4mEAV2ElQ5F5rojcqKncO6CPeVOYj6pgIgUh9JynkcJ9cOJzybFGFphZCTYeJb4nTqIA1+CIJ+UU0BIgYCx3cWH3PQt8crnue95lApPRPwlbx2Vq0fUl2l/S4QsRAYJ1acUDEAAIAAAACAAAAAgAAAAAAAAAAAAAAiAgLSDKUC7iiWhtIYFb1DqAY3sGmOH7zb5MrtRF9sGgqQ7xgnVpxQMQAAgAAAAIAAAACAAAAAAAQAAAAA
+ Fails Signer checks * Case: A Witness UTXO is provided for a non-witness input -- cgit v1.2.3 From db117ef473492c180939e694773104002a3929a4 Mon Sep 17 00:00:00 2001 From: Paul Sztorc Date: Thu, 25 Jul 2019 23:51:05 -0700 Subject: fix spacing --- bip-0301.mediawiki | 8 -------- 1 file changed, 8 deletions(-) diff --git a/bip-0301.mediawiki b/bip-0301.mediawiki index 1a6e999..3b77f58 100644 --- a/bip-0301.mediawiki +++ b/bip-0301.mediawiki @@ -1,5 +1,4 @@
-
     BIP: 301
     Layer: Consensus (soft fork)
     Title: Blind Merged Mining (Consensus layer)
@@ -11,10 +10,8 @@
     Type: Standards Track
     Created: 2019-07-23
     License: BSD-2-Clause
-
 
- ==Abstract== @@ -27,7 +24,6 @@ Our goal here, is to allow mainchain miners to trustlessly "sell" the act of fin ==Motivation== - Regular "Merged-Mining" (MM) allows miners to reuse their hashing work to secure other chains (for example, as in Namecoin). However, traditional MM has two drawbacks: # Miners must run a full node of the other chain. (This is because [while miners can effortlessly create the block] miners will not create a valid payment to themselves, unless the block that they MM is a valid one. Therefore, miners must assemble a *valid* block first, then MM it.) @@ -38,7 +34,6 @@ BMM addresses both shortcomings. ==Specification== - Note: This document uses the notation side:\* and main:\* in front of otherwise-ambiguous words (such as "block", "node", or "chain"), to distinguish the mainchain version from its sidechain counterpart. We also use "Simon" to refer to a Sidechain Full Node, and "Mary" to refer to a mainchain miner. @@ -194,8 +189,6 @@ As a soft fork, older software will continue to operate without modification. As (As a matter of fact, the only people receiving money here all happen to be miners. So there is less reason than ever to expect compatibility problems.) - - ==Deployment== This BIP will be deployed by "version bits" BIP9 with the name "blindmm" and using bit 4. @@ -217,7 +210,6 @@ Also, for interest, see an example sidechain here: https://github.com/drivechain ==References== - * http://www.drivechain.info/literature/index.html * http://www.truthcoin.info/blog/blind-merged-mining/ * https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-July/014789.html -- cgit v1.2.3 From 542c66e6ddc3d9f703114f7e645f19d572c8274e Mon Sep 17 00:00:00 2001 From: Paul Sztorc Date: Thu, 25 Jul 2019 23:57:12 -0700 Subject: Add 301 Blind Merged Mining --- README.mediawiki | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/README.mediawiki b/README.mediawiki index bcc1e3f..956dad8 100644 --- a/README.mediawiki +++ b/README.mediawiki @@ -805,6 +805,13 @@ Those proposing changes should consider that ultimately consent may rest with th | Sean Bowe, Daira Hopwood | Standard | Draft +|- +| [[bip-0301.mediawiki|301]] +| Consensus (soft fork) +| Blind Merged Mining (Consensus layer) +| Paul Sztorc, CryptAxe +| Standard +| Draft |} -- cgit v1.2.3 From 53e37a1dea527c51c9bacb02975cd3184d873742 Mon Sep 17 00:00:00 2001 From: Luke Dashjr Date: Fri, 26 Jul 2019 12:41:03 +0000 Subject: Fix preamble in BIP 301 --- bip-0301.mediawiki | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/bip-0301.mediawiki b/bip-0301.mediawiki index 3b77f58..d6056f2 100644 --- a/bip-0301.mediawiki +++ b/bip-0301.mediawiki @@ -1,15 +1,15 @@
-    BIP: 301
-    Layer: Consensus (soft fork)
-    Title: Blind Merged Mining (Consensus layer)
-    Author: Paul Sztorc 
-            CryptAxe 
-    Comments-Summary: No comments yet.
-    Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-301
-    Status: Draft
-    Type: Standards Track
-    Created: 2019-07-23
-    License: BSD-2-Clause
+  BIP: 301
+  Layer: Consensus (soft fork)
+  Title: Blind Merged Mining (Consensus layer)
+  Author: Paul Sztorc 
+          CryptAxe 
+  Comments-Summary: No comments yet.
+  Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0301
+  Status: Draft
+  Type: Standards Track
+  Created: 2019-07-23
+  License: BSD-2-Clause
 
==Abstract== -- cgit v1.2.3 From ec6adac5f86213f7e0a23f92fd662710cd285d23 Mon Sep 17 00:00:00 2001 From: Luke Dashjr Date: Fri, 26 Jul 2019 12:42:14 +0000 Subject: README: Minor spacing fix --- README.mediawiki | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.mediawiki b/README.mediawiki index ebfce02..93356f9 100644 --- a/README.mediawiki +++ b/README.mediawiki @@ -223,7 +223,7 @@ Those proposing changes should consider that ultimately consent may rest with th | Marek Palatinus | Standard | BIP number allocated -|-style="background-color: #cfffcf" +|- style="background-color: #cfffcf" | [[bip-0042.mediawiki|42]] | Consensus (soft fork) | A finite monetary supply for Bitcoin -- cgit v1.2.3 From 213fb62ceabdbcfe3576d83c3ca3d986e4b45ae9 Mon Sep 17 00:00:00 2001 From: Luke Dashjr Date: Fri, 26 Jul 2019 16:28:21 +0000 Subject: Promote BIP 75 to Final --- README.mediawiki | 4 ++-- bip-0075.mediawiki | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/README.mediawiki b/README.mediawiki index 93356f9..c8a1596 100644 --- a/README.mediawiki +++ b/README.mediawiki @@ -378,13 +378,13 @@ Those proposing changes should consider that ultimately consent may rest with th | Toby Padilla | Standard | Rejected -|- +|- style="background-color: #cfffcf" | [[bip-0075.mediawiki|75]] | Applications | Out of Band Address Exchange using Payment Protocol Encryption | Justin Newton, Matt David, Aaron Voisine, James MacWhyte | Standard -| Draft +| Final |- style="background-color: #ffffcf" | [[bip-0079.mediawiki|79]] | Applications diff --git a/bip-0075.mediawiki b/bip-0075.mediawiki index a516916..8c49645 100644 --- a/bip-0075.mediawiki +++ b/bip-0075.mediawiki @@ -8,7 +8,7 @@ James MacWhyte Comments-Summary: Recommended for implementation (one person) Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0075 - Status: Draft + Status: Final Type: Standards Track Created: 2015-11-20 License: CC-BY-4.0 -- cgit v1.2.3 From daa59903e23548d3017b926639b03dedc5b0b626 Mon Sep 17 00:00:00 2001 From: Karl-Johan Alm Date: Wed, 24 Jul 2019 13:27:48 +0900 Subject: BIP-322 update to remove ProveFunds purpose (with a note about extensibility) and to fix the serialization format --- bip-0322.mediawiki | 71 +++++++++++++++++++++++++++++++----------------------- 1 file changed, 41 insertions(+), 30 deletions(-) diff --git a/bip-0322.mediawiki b/bip-0322.mediawiki index 5191143..71959f1 100644 --- a/bip-0322.mediawiki +++ b/bip-0322.mediawiki @@ -23,7 +23,7 @@ The current message signing standard only works for P2PKH (1...) addresses. By e A new structure SignatureProof is added, which is a simple serializable scriptSig & witness container. -Two actions "Sign" and "Verify" are defined along with two *purposes* "SignMessage" and "ProveFunds". +Two actions "Sign" and "Verify" are defined along with one ''purpose'', "SignMessage", with the ability to expand in the future to add a potential "ProveFunds" purpose. === SignatureProof container === @@ -36,11 +36,7 @@ Two actions "Sign" and "Verify" are defined along with two *purposes* "SignMessa |- |Uint32||4||flags||standard flags (1-to-1 with standard flags in Bitcoin Core) |- -|VarInt||1-8||msglen||Number of bytes in message string, excluding NUL termination -|- -|Char*||[msglen]||msg||The message being signed for all subjects, excluding NUL termination -|- -|Uint8||1||entries||Number of proof entriesWhy support multiple proofs? In particular with proof of funds, it is non-trivial to check a large number of individual proofs (one per UTXO) for duplicates. Software could be written to do so, but it seems more efficient to build this check into the specification itself. +|Uint8||1||entries||number of proof entriesWhy support multiple proofs? It is non-trivial to check a large number of individual proofs for duplicates. Software could be written to do so, but it seems more efficient to build this check into the specification itself. |} The above is followed by [entries] number of signature entries: @@ -56,9 +52,9 @@ The above is followed by [entries] number of signature entries: |- |Uint8*||[scriptsiglen]||scriptsig||ScriptSig data |- -|VarInt||1-8||witlen||Number of bytes in witness data +|VarInt||1-8||witlen||Number of entries in witness stack |- -|Uint8*||[witlen]||wit||Witness +|Uint8[]*||[witlen]||wit||Witness stack, as [witlen] uint8* vectors, each one prepended with a varint of its size |} In some cases, the scriptsig or wit may be empty. If both are empty, the proof is incomplete. @@ -80,36 +76,24 @@ A verification call will return a result code according to the table below. |- |INVALID||One or more of the given proofs were invalid |- -|SPENT||One or more of the claimed UTXO:s has been spent -|- |ERROR||An error was encountered |} == Signing and Verifying == -Let there be an empty set `inputs` which is populated and tested at each call to one of the actions below. +If the challenge consists of a single address and the address is in the P2PK(H) (legacy) format, sign using the legacy format (further information below). Otherwise continue as stated below. + +Let there be an empty set inputs which is populated and tested at each call to one of the actions below. === Purpose: SignMessage === The "SignMessage" purpose generates a sighash based on a scriptPubKey and a message. It emits a VALID verification result code unless otherwise stated. -# Return INVALID if scriptPubKey already exists in `inputs` set, otherwise insert itWhy track duplicates? Because a 3-entry proof is not proving 3 inputs unless they are all distinct +# Return INVALID if scriptPubKey already exists in inputs set, otherwise insert itWhy track duplicates? Because a 3-entry proof is not proving 3 entries unless they are all distinct # Define the message pre-image as the sequence "Bitcoin Message:" concatenated with the message, encoded in UTF-8 using Normalization Form Compatibility Decomposition (NFKD) # Let sighash = sha256(sha256(scriptPubKey || pre-image)) -=== Purpose: ProveFunds === - -The "ProveFunds" purpose generates a sighash and a scriptPubKey from a transaction, an output index, and a message. For multiple simultaneous proofs, it also requires access to the ordered list of proofs. It emits a VALID verification result code unless otherwise stated. - -# Let txid be the transaction ID of the transaction, and vout be the output index corresponding to the index of the output being spent -# Return INVALID if the txid:vout pair already exists in `inputs` set, otherwise insert it -# Return SPENT if the txid/vout is not a valid UTXO according to a Bitcoin nodeSynced up or not? A normal verifier would use a synced up node. An auditor checking records from a client that were submitted in the past want to use a node that is synced up to the block corresponding to the proof, or the proof will fail, even if it may have been valid at the time of creation. -# Extract scriptPubKey from transaction output -# Define the message pre-image as the concatenation of the following components:Why not just the UTXO data? We want the verifier to be able to challenge the prover with a custom message to sign, or anyone can reuse the POF proof for a set of UTXO:s once they have seen it, and the funds have not yet been spent -#* the string "POF:" -#* the message, encoded in UTF-8 using Normalization Form Compatibility Decomposition (NFKD), including the null terminating character (i.e. write strlen(message) + 1 bytes, for a C string) -#* all transactions being proven for, as binary txid (little endian uint256) followed by index (little endian uint32), each separated by a single `0x00` byte -# Let sighash = sha256(sha256(scriptPubKey || pre-image)) +A private key may be used directly to sign a message. In this case, its P2WPKH bech32 address shall be derived, and used as the input. === Action: Sign === @@ -119,6 +103,8 @@ The "Sign" action takes as input a purpose. It returns a signature or fails. # Derive the private key privkey for the scriptPubKey; FAIL if not VALID # Generate and return a signature sig with privkey=privkey, sighash=sighash +The resulting signature proof should be encoded using base64 encoding. + === Action: Verify === The "Verify" action takes as input a standard flags value, a script sig, an optional witness, and a purpose. @@ -139,13 +125,39 @@ Note that the order of the entries in the proof must match the order of the entr * If a verification call returns ERROR or INVALID, return ERROR or INVALID immediately, ignoring as yet unverified entries * After all verifications complete, ** return INCONCLUSIVE if any verification call returned INCONCLUSIVE -** return SPENT if any verification call returned SPENT ** return INCOMPLETE if the INCOMPLETE flag is set ** return VALID +== Legacy format == + +The legacy format is restricted to the legacy P2PK(H) address format, and restricted to one single challenge (address). + +Any other input (e.g. multiple addresses, or non-P2PK(H) address format(s)) must be signed using the new format described above. + +=== Signing === + +Given the P2PK(H) address a and the message m: +# let p be the pubkey-hash contained in a +# let x be the private key associated with p +# let digest be SHA56d("Bitcoin Signed Message:\n"||m) +# create a compact signature sig (aka "recoverable ECDSA signature") using x on digest + +The resulting proof is sig, serialized using the base64 encoding. + +=== Verifying === + +Given the P2PK(H) address a, the message m, and the compact signature sig: + +# let p be the pubkey-hash contained in a +# let digest be SHA56d("Bitcoin Signed Message:\n"||m) +# attempt pubkey recovery for digest using the signature sig and store the resulting pubkey into Q +## fail verification if pubkey recovery above fails +# let q be the pubkey-hash of Q +# if p == q, the proof is valid, otherwise it is invalid + == Compatibility == -This specification is not backwards compatible with the legacy signmessage/verifymessage specification. However, legacy addresses (1...) may be used in this implementation without any problems. +This specification is backwards compatible with the legacy signmessage/verifymessage specification through the special case as described above. == Rationale == @@ -153,16 +165,15 @@ This specification is not backwards compatible with the legacy signmessage/verif == Reference implementation == -To do. +# Pull request to Bitcoin Core: https://github.com/bitcoin/bitcoin/pull/16440 == Acknowledgements == -TODO +Thanks to David Harding, Jim Posen, Kalle Rosenbaum, Pieter Wuille, and many others for their feedback on the specification. == References == # Original mailing list thread: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2018-March/015818.html -# Pull request, with comments: https://github.com/bitcoin/bips/pull/725 == Copyright == -- cgit v1.2.3 From e24e86b482e394e18803f14c7e2338aab9b7e1e2 Mon Sep 17 00:00:00 2001 From: Karl-Johan Alm Date: Mon, 29 Jul 2019 20:40:36 +0900 Subject: bip-322: add test vectors --- bip-0322.mediawiki | 44 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 44 insertions(+) diff --git a/bip-0322.mediawiki b/bip-0322.mediawiki index 71959f1..6d0fb78 100644 --- a/bip-0322.mediawiki +++ b/bip-0322.mediawiki @@ -178,3 +178,47 @@ Thanks to David Harding, Jim Posen, Kalle Rosenbaum, Pieter Wuille, and many oth == Copyright == This document is licensed under the Creative Commons CC0 1.0 Universal license. + +== Test vectors == + +* STANDARD_SCRIPT_VERIFY_FLAGS = 01ffdf (131039) +* address = 2MsnqGxX7Abtn4b379MEpkDaD3VbNKQosd8 +* message = "hello world" +* sighash = 7b66a1861b4e179e1dbab4702e26bcefeabf1cada7cccc97b6ebaec89a035d84 (sha256d("Bitcoin Message:hello world")) + +A possible proof is: + +* HEX: dfff01000117160014689bbb5d76774321c652832ea209958fa1770b330247304402204368b119399d33b9bc9beef06d713becefd3ac508dc95ff62d1859d4912960c7022063d88ddc648faed710b3f870b7a839fdc1b3bfc3c3bd065df51bbbd8c386c81c012102b4e4c6d5021576a5c0bc4535890c3f17e1ff23a94eac87beb0a5e8747c42d920 +* Base64: 3/8BAAEXFgAUaJu7XXZ3QyHGUoMuogmVj6F3CzMCRzBEAiBDaLEZOZ0zubyb7vBtcTvs79OsUI3JX/YtGFnUkSlgxwIgY9iN3GSPrtcQs/hwt6g5/cGzv8PDvQZd9Ru72MOGyBwBIQK05MbVAhV2pcC8RTWJDD8X4f8jqU6sh76wpeh0fELZIA== + +Split into components: + +{|class="wikitable" style="text-align: center;" +|- +!Type +!Length +!Name +!Value +!Comment +|- +|Uint32||4||flags||dfff0100||standard flags used in signing +|- +|Uint8||1||entries||01||1 entry +|- +|VarInt||1-8||scriptsiglen||17||23 byte scriptsig +|- +|Uint8[32]||32||scriptsig||160014689bbb5d76774321c652832ea209958fa1770b33||ScriptSig data +|- +|VarInt||1-8||witlen||02||2 entries in witness stack +|- +|VarInt||1-8||entry1len||47||71 byte entry +|- +|Uint8[71]||71||entry1||304402204368b119399d33b9bc9beef06d713becefd3ac50 +8dc95ff62d1859d4912960c7022063d88ddc648faed710b3 +f870b7a839fdc1b3bfc3c3bd065df51bbbd8c386c81c01||Witness stack item 1 +|- +|VarInt||1-8||entry2len||21||33 byte entry +|- +|Uint8[33]||33||entry2||02b4e4c6d5021576a5c0bc4535890c3f17e1ff23a94eac87 +beb0a5e8747c42d920||Witness stack item 2 +|} -- cgit v1.2.3 From 9e62a7a83859927bc01dd86e5f4df2e652d6de55 Mon Sep 17 00:00:00 2001 From: Karl-Johan Alm Date: Tue, 30 Jul 2019 00:22:42 +0900 Subject: BIP-154: change to withdrawn status --- README.mediawiki | 4 ++-- bip-0154.mediawiki | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/README.mediawiki b/README.mediawiki index c8a1596..be99e1d 100644 --- a/README.mediawiki +++ b/README.mediawiki @@ -763,13 +763,13 @@ Those proposing changes should consider that ultimately consent may rest with th | Matt Corallo | Standard | Draft -|- +|- style="background-color: #ffcfcf" | [[bip-0154.mediawiki|154]] | Peer Services | Rate Limiting via peer specified challenges | Karl-Johan Alm | Standard -| Draft +| Withdrawn |- | [[bip-0155.mediawiki|155]] | Peer Services diff --git a/bip-0154.mediawiki b/bip-0154.mediawiki index a0bf387..c1e4cdb 100644 --- a/bip-0154.mediawiki +++ b/bip-0154.mediawiki @@ -5,7 +5,7 @@ Author: Karl-Johan Alm Comments-Summary: No comments yet. Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0154 - Status: Draft + Status: Withdrawn Type: Standards Track Created: 2017-04-12 License: BSD-2-Clause -- cgit v1.2.3 From cfacb9d68901841f11e8d4bc7f8a852589c7ccba Mon Sep 17 00:00:00 2001 From: wigy Date: Tue, 30 Jul 2019 11:41:21 +0200 Subject: Applied standard terminology --- bip-0178.mediawiki | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bip-0178.mediawiki b/bip-0178.mediawiki index b26efde..5522664 100644 --- a/bip-0178.mediawiki +++ b/bip-0178.mediawiki @@ -3,7 +3,7 @@ Layer: Applications Title: Version Extended WIF Author: Karl-Johan Alm - Comments-Summary: One comment, with some Discouragement + Comments-Summary: Discouraged for implementation (one person) Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0178 Status: Draft Type: Standards Track -- cgit v1.2.3 From 91ecdbeb2620f5d1352b99a54b32b6f774893b6b Mon Sep 17 00:00:00 2001 From: Karl-Johan Alm Date: Wed, 24 Jul 2019 13:27:48 +0900 Subject: BIP-322 updates to fix verification and other fixes --- bip-0322.mediawiki | 97 +++++++++++++++++++++++++++++++++++++----------------- 1 file changed, 66 insertions(+), 31 deletions(-) diff --git a/bip-0322.mediawiki b/bip-0322.mediawiki index 6d0fb78..9448945 100644 --- a/bip-0322.mediawiki +++ b/bip-0322.mediawiki @@ -34,7 +34,7 @@ Two actions "Sign" and "Verify" are defined along with one ''purpose'', "SignMes !Name !Comment |- -|Uint32||4||flags||standard flags (1-to-1 with standard flags in Bitcoin Core) +|Uint32||4||version||BIP322 version format; must be equal to 1; if > 1, verifier must abort the verification process |- |Uint8||1||entries||number of proof entriesWhy support multiple proofs? It is non-trivial to check a large number of individual proofs for duplicates. Software could be written to do so, but it seems more efficient to build this check into the specification itself. |} @@ -70,7 +70,7 @@ A verification call will return a result code according to the table below. |- |INCOMPLETE||One or several of the given challenges had an empty proof. The prover may need some other entity to complete the proof. |- -|INCONCLUSIVE||One or several of the given proofs used unknown opcodes or the scriptPubKey had an unknown witness version, perhaps due to the verifying node being outdated. +|INCONCLUSIVE||One or several of the given proofs was consensus-valid but policy-invalid. |- |VALID||All proofs were deemed valid. |- @@ -81,7 +81,7 @@ A verification call will return a result code according to the table below. == Signing and Verifying == -If the challenge consists of a single address and the address is in the P2PK(H) (legacy) format, sign using the legacy format (further information below). Otherwise continue as stated below. +If the challenge consists of a single address and the address is in the P2PKH (legacy) format, sign using the legacy format (further information below). Otherwise continue as stated below. Let there be an empty set inputs which is populated and tested at each call to one of the actions below. @@ -90,7 +90,7 @@ Let there be an empty set inputs which is populated and tested at e The "SignMessage" purpose generates a sighash based on a scriptPubKey and a message. It emits a VALID verification result code unless otherwise stated. # Return INVALID if scriptPubKey already exists in inputs set, otherwise insert itWhy track duplicates? Because a 3-entry proof is not proving 3 entries unless they are all distinct -# Define the message pre-image as the sequence "Bitcoin Message:" concatenated with the message, encoded in UTF-8 using Normalization Form Compatibility Decomposition (NFKD) +# Define the message pre-image as the sequence "Bitcoin Signed Message:\n" concatenated with the message, encoded in UTF-8 using Normalization Form Compatibility Decomposition (NFKD) # Let sighash = sha256(sha256(scriptPubKey || pre-image)) A private key may be used directly to sign a message. In this case, its P2WPKH bech32 address shall be derived, and used as the input. @@ -111,9 +111,10 @@ The "Verify" action takes as input a standard flags value, a script sig, an opti It emits one of INCONCLUSIVE, VALID, INVALID, or ERROR. # Obtain the sighash and scriptPubKey from the purpose; pass on result code if not VALID -# If one or more of the standard flags are unknown, return INCONCLUSIVE -# Verify Script with flags=standard flags, scriptSig=script sig, scriptPubKey=scriptPubKey, witness=witness, and sighash=sighash -# Return VALID if verify succeeds, otherwise return INVALID +# Verify Script with flags=consensus flags (currently P2SH, DERSIG, NULLDUMMY, CLTV, CSV, WITNESS), scriptSig=script sig, scriptPubKey=scriptPubKey, witness=witness, and sighash=sighash +# Return INVALID if verification fails +# Verify Script with flags=standard flags (above plus STRICTENC, MINIMALDATA, etc.), scriptSig=script sig, scriptPubKey=scriptPubKey, witness=witness, and sighash=sighash +# Return VALID if verification succeeds, otherwise return INCONCLUSIVE === Multiple Proofs === @@ -130,15 +131,16 @@ Note that the order of the entries in the proof must match the order of the entr == Legacy format == -The legacy format is restricted to the legacy P2PK(H) address format, and restricted to one single challenge (address). +The legacy format is restricted to the legacy P2PKH address format, and restricted to one single challenge (address). -Any other input (e.g. multiple addresses, or non-P2PK(H) address format(s)) must be signed using the new format described above. +Any other input (e.g. multiple addresses, or non-P2PKH address format(s)) must be signed using the new format described above. === Signing === -Given the P2PK(H) address a and the message m: -# let p be the pubkey-hash contained in a -# let x be the private key associated with p +Given the P2PKH address a and the message m, and the pubkey-hash function pkh(P) = ripemd160(sha256(P)): + +# let p be the pubkey-hash pkh(P) for the pubkey P, contained in a +# let x be the private key associated with P so that pkh(xG) = p # let digest be SHA56d("Bitcoin Signed Message:\n"||m) # create a compact signature sig (aka "recoverable ECDSA signature") using x on digest @@ -146,13 +148,13 @@ The resulting proof is sig, serialized using the base64 encoding. === Verifying === -Given the P2PK(H) address a, the message m, and the compact signature sig: +Given the P2PKH address a, the message m, the compact signature sig, and the pubkey-hash function pkh(P) = ripemd160(sha256(P)): -# let p be the pubkey-hash contained in a +# let p be the pubkey-hash pkh(P) for the pubkey P, contained in a # let digest be SHA56d("Bitcoin Signed Message:\n"||m) # attempt pubkey recovery for digest using the signature sig and store the resulting pubkey into Q ## fail verification if pubkey recovery above fails -# let q be the pubkey-hash of Q +# let q be the pubkey-hash pkh(Q) for the pubkey Q # if p == q, the proof is valid, otherwise it is invalid == Compatibility == @@ -179,17 +181,50 @@ Thanks to David Harding, Jim Posen, Kalle Rosenbaum, Pieter Wuille, and many oth This document is licensed under the Creative Commons CC0 1.0 Universal license. +== Consensus and standard flags == + +Each flag is associated with some type of enforced rule (most often a soft fork). There are two sets of flags: consensus flags (which result in a block being rejected, if violated), and policy flags (which result in a transaction being accepted only if it is contained within an actual block, and rejected otherwise, if violated). The policy flags are a super-set of the consensus flags. + +BIP322 specifies that a proof that validates for both rulesets is valid, a proof that validates for consensus rules, but not for policy rules, is "inconclusive", and a proof that does not validate for consensus rules is "invalid" (regardless of policy rule validation). + +The ruleset sometimes changes. This BIP does not intend to be complete, nor does it indicate enforcement of rules, it simply lists the rules as they stand at the point of writing. + +=== Consensus rules === + +* P2SH: evaluate P2SH ([https://github.com/bitcoin/bips/blob/master/bip-0016.mediawiki BIP16]) subscripts +* DERSIG: enforce strict DER ([https://github.com/bitcoin/bips/blob/master/bip-0066.mediawiki BIP66]) compliance +* NULLDUMMY: enforce NULLDUMMY ([https://github.com/bitcoin/bips/blob/master/bip-0147.mediawiki BIP147]) +* CHECKLOCKTIMEVERIFY: enable CHECKLOCKTIMEVERIFY ([https://github.com/bitcoin/bips/blob/master/bip-0065.mediawiki BIP65]) +* CHECKSEQUENCEVERIFY: enable CHECKSEQUENCEVERIFY ([https://github.com/bitcoin/bips/blob/master/bip-0112.mediawiki BIP112]) +* WITNESS: enable WITNESS ([https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki BIP141]) + +=== Policy rules === + +All of the above, plus (subject to change): + +* STRICTENC: non-strict DER signature or undefined hashtype +* MINIMALDATA: require minimal encodings for all push operations +* DISCOURAGE_UPGRADABLE_NOPS: discourage use of NOPs reserved for upgrades +* CLEANSTACK: require that only a single stack element remains after evaluation +* MINIMALIF: Segwit script only: require the argument of OP_IF/NOTIF to be exactly 0x01 or empty vector +* NULLFAIL: signature(s) must be empty vector if a CHECK(MULTI)SIG operation failed +* LOW_S: signature with S > order/2 in a checksig operation +* DISCOURAGE_UPGRADABLE_WITNESS_PROGRAM: v1-16 witness programs are non-standard (i.e. forbidden) +* WITNESS_PUBKEYTYPE: public keys in segregated witness scripts must be compressed +* CONST_SCRIPTCODE: OP_CODESEPARATOR and FindAndDelete fail any non-segwit scripts + == Test vectors == -* STANDARD_SCRIPT_VERIFY_FLAGS = 01ffdf (131039) -* address = 2MsnqGxX7Abtn4b379MEpkDaD3VbNKQosd8 -* message = "hello world" -* sighash = 7b66a1861b4e179e1dbab4702e26bcefeabf1cada7cccc97b6ebaec89a035d84 (sha256d("Bitcoin Message:hello world")) +== Native segwit test vector == + +* address = bcrt1qe7nte4zk4ayly5tc53dtdjupgkz0lr8azx3rzz +* message = hello +* sighash = 790eef86c204f0bff969ff822121317aa34eff0215dbd30ccf031e7b2f3f0cc1 (sha256d("Bitcoin Signed Message:\n:hello")) A possible proof is: -* HEX: dfff01000117160014689bbb5d76774321c652832ea209958fa1770b330247304402204368b119399d33b9bc9beef06d713becefd3ac508dc95ff62d1859d4912960c7022063d88ddc648faed710b3f870b7a839fdc1b3bfc3c3bd065df51bbbd8c386c81c012102b4e4c6d5021576a5c0bc4535890c3f17e1ff23a94eac87beb0a5e8747c42d920 -* Base64: 3/8BAAEXFgAUaJu7XXZ3QyHGUoMuogmVj6F3CzMCRzBEAiBDaLEZOZ0zubyb7vBtcTvs79OsUI3JX/YtGFnUkSlgxwIgY9iN3GSPrtcQs/hwt6g5/cGzv8PDvQZd9Ru72MOGyBwBIQK05MbVAhV2pcC8RTWJDD8X4f8jqU6sh76wpeh0fELZIA== +* HEX: 01000000010002473044022075b4fb40421d55c55462879cb352a85eeb3af2138d3f02902c9143f12870f5f70220119c2995c1661138142f3899c1fd6d1af7e790e0e081be72db9ce7bf5b5b932901210290beccd02b73eca57467b2b6f1e47161a9b76a5e67586e7c1dee9ea6e2dcd869 +* Base64: AQAAAAEAAkcwRAIgdbT7QEIdVcVUYoecs1KoXus68hONPwKQLJFD8Shw9fcCIBGcKZXBZhE4FC84mcH9bRr355Dg4IG+ctuc579bW5MpASECkL7M0Ctz7KV0Z7K28eRxYam3al5nWG58He6epuLc2Gk= Split into components: @@ -201,24 +236,24 @@ Split into components: !Value !Comment |- -|Uint32||4||flags||dfff0100||standard flags used in signing +|Uint32||4||flags||01000000||proof format version |- |Uint8||1||entries||01||1 entry |- -|VarInt||1-8||scriptsiglen||17||23 byte scriptsig -|- -|Uint8[32]||32||scriptsig||160014689bbb5d76774321c652832ea209958fa1770b33||ScriptSig data +|VarInt||1-8||scriptsiglen||00||0 byte scriptsig |- -|VarInt||1-8||witlen||02||2 entries in witness stack +|VarInt||1-8||wit entries||02||2 witness stack entries |- |VarInt||1-8||entry1len||47||71 byte entry |- -|Uint8[71]||71||entry1||304402204368b119399d33b9bc9beef06d713becefd3ac50 -8dc95ff62d1859d4912960c7022063d88ddc648faed710b3 -f870b7a839fdc1b3bfc3c3bd065df51bbbd8c386c81c01||Witness stack item 1 +|Uint8[71]||71||entry1||3044022075b4fb40421d55c55462879cb352a85eeb3af213 +8d3f02902c9143f12870f5f70220119c2995c1661138142f +3899c1fd6d1af7e790e0e081be72db9ce7bf5b5b932901||Witness stack item 1 |- |VarInt||1-8||entry2len||21||33 byte entry |- -|Uint8[33]||33||entry2||02b4e4c6d5021576a5c0bc4535890c3f17e1ff23a94eac87 -beb0a5e8747c42d920||Witness stack item 2 +|Uint8[33]||33||entry2||0290beccd02b73eca57467b2b6f1e47161a9b76a5e67586e +7c1dee9ea6e2dcd869||Witness stack item 2 |} + +The above test vector is for a bech32 P2WPKH (native segwit) address. (Once BIP solidifies, will add test vector for other types.) -- cgit v1.2.3 From 503f35a9a066a51be29d9c81e73f72f54a403b76 Mon Sep 17 00:00:00 2001 From: Andrew Chow Date: Thu, 1 Aug 2019 14:28:37 -0400 Subject: Add Version type --- bip-0174.mediawiki | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/bip-0174.mediawiki b/bip-0174.mediawiki index b4a6407..bbc0fc6 100644 --- a/bip-0174.mediawiki +++ b/bip-0174.mediawiki @@ -124,6 +124,12 @@ The currently defined global types are as follows: ** Value: The master key fingerprint as defined by BIP 32 concatenated with the derivation path of the public key. The derivation path is represented as 32 bit unsigned integer indexes concatenated with each other. The number of 32 bit unsigned integer indexes must match the depth provided in the extended public key. *** {master key fingerprint}|{32-bit int}|...|{32-bit int} +* Type: Version Number PSBT_GLOBAL_VERSION = 0xFB +** Key: None. The key must only contain the 1 byte type. +*** {0xFB} +** Value: The 32-bit little endian unsigned integer representing the version number of this PSBT. If ommitted, the version number is 0. +*** {32-bit int} + The currently defined per-input types are defined as follows: * Type: Non-Witness UTXO PSBT_IN_NON_WITNESS_UTXO = 0x00 @@ -432,6 +438,17 @@ types will be ignored and passed-through by signers which do not know about them If one byte type fields were to ever run out, new extensions can still be added by defining multi-byte types where the first byte signals that the next byte indicates the type and so on. +===Version Numbers=== + +The Version number field exists only as a safeguard in the event that a backwards incompatible change is introduced to PSBT. +If a parser encounters a version number it does not recognize, it should exit immediately as this indicates that the PSBT will contain types that it does not know about and cannot be ignored. +Current PSBTs are Version 0. Any PSBT that does not have the version field is version 0. +It is not expected that any backwards incompatible change will be introduced to PSBT, so it is not expected that the version field will ever actually be seen. + +Updaters and combiners that need to add a version number to a PSBT should use the highest version number required. +For example, if a combiner sees two PSBTs for the same transaction, one with version 0, and the other with version 1, then it should combine them and produce a PSBT with version 1. +If an updater is updating a PSBT and needs to add a field that is only available in version 1, then it should set the PSBT version number to 1 unless a version higher than that is already specified. + ==Compatibility== This transaction format is designed so that it is unable to be properly unserialized -- cgit v1.2.3 From 20fdf77a2bcb04886b2b1d4d1b20b0135388ac4b Mon Sep 17 00:00:00 2001 From: Andrew Chow Date: Thu, 1 Aug 2019 14:36:25 -0400 Subject: Specify that types are compact size unsigned ints to allow for multi-byte types --- bip-0174.mediawiki | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/bip-0174.mediawiki b/bip-0174.mediawiki index bbc0fc6..d30b05d 100644 --- a/bip-0174.mediawiki +++ b/bip-0174.mediawiki @@ -106,8 +106,10 @@ The format of each key-value map is as follows: | Must be 0x00. |} -The first byte of each key specifies the type of the key-value pair. There are global types, -per-input types, and per-output types. +At the beginning of each key is a compact size unsigned integer representing the type. +This compact size unsigned integer must be minimally encoded, i.e. if the value can be represented using one byte, it must be represented as one byte. +For convenience, this BIP will specify types using their full serialization, so a multi-byte type will have it's full prefix and zero padding as necessary. +There are global types, per-input types, and per-output types. The currently defined global types are as follows: @@ -436,8 +438,6 @@ The Partially Signed Transaction format can be extended in the future by adding new types for key-value pairs. Backwards compatibilty will still be maintained as those new types will be ignored and passed-through by signers which do not know about them. -If one byte type fields were to ever run out, new extensions can still be added by defining multi-byte types where the first byte signals that the next byte indicates the type and so on. - ===Version Numbers=== The Version number field exists only as a safeguard in the event that a backwards incompatible change is introduced to PSBT. -- cgit v1.2.3 From 6129830a330c9c53a918c671dce24a49b3921396 Mon Sep 17 00:00:00 2001 From: Karl-Johan Alm Date: Thu, 1 Aug 2019 18:06:58 +0900 Subject: travis: add a check for common mistake mediawiki links Several BIPs have mistaken links in format [text](url) which this travis check will detect and complain about. --- .travis.yml | 1 + scripts/link-format-chk.sh | 23 +++++++++++++++++++++++ 2 files changed, 24 insertions(+) create mode 100755 scripts/link-format-chk.sh diff --git a/.travis.yml b/.travis.yml index ed99de0..e463ab6 100644 --- a/.travis.yml +++ b/.travis.yml @@ -2,6 +2,7 @@ os: linux language: generic sudo: false script: + - scripts/link-format-chk.sh - scripts/buildtable.pl >/tmp/table.mediawiki || exit 1 - diff README.mediawiki /tmp/table.mediawiki | grep '^[<>] |' >/tmp/after.diff || true - if git checkout HEAD^ && scripts/buildtable.pl >/tmp/table.mediawiki 2>/dev/null; then diff README.mediawiki /tmp/table.mediawiki | grep '^[<>] |' >/tmp/before.diff || true; newdiff=$(diff -s /tmp/before.diff /tmp/after.diff -u | grep '^+'); if [ -n "$newdiff" ]; then echo "$newdiff"; exit 1; fi; else echo 'Cannot build previous commit table for comparison'; fi diff --git a/scripts/link-format-chk.sh b/scripts/link-format-chk.sh new file mode 100755 index 0000000..86f28be --- /dev/null +++ b/scripts/link-format-chk.sh @@ -0,0 +1,23 @@ +#!/usr/bin/env bash +# +# Copyright (c) 2019 The Bitcoin Core developers +# Distributed under the MIT software license, see the accompanying +# file COPYING or http://www.opensource.org/licenses/mit-license.php. +# +# Check wrong mediawiki link format + +ECODE=0 +FILES="" +for fname in $(git diff --name-only HEAD $(git merge-base HEAD master)); do + if [[ $fname == *.mediawiki ]]; then + GRES=$(grep -n '](' $fname) + if [ "$GRES" != "" ]; then + if [ $ECODE -eq 0 ]; then + >&2 echo "Github Mediawiki format writes link as [URL text], not as [text](url):" + fi + ECODE=1 + echo "- $fname:$GRES" + fi + fi +done +exit $ECODE -- cgit v1.2.3 From f0784c6e922724c674550ee8aa4bfc72d6d32a8a Mon Sep 17 00:00:00 2001 From: Karl-Johan Alm Date: Tue, 6 Aug 2019 17:09:25 +0900 Subject: BIP322: added background to explain why BIP322 exists, and what it changes --- bip-0322.mediawiki | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/bip-0322.mediawiki b/bip-0322.mediawiki index 9448945..a4973d8 100644 --- a/bip-0322.mediawiki +++ b/bip-0322.mediawiki @@ -15,6 +15,17 @@ A standard for interoperable generic signed messages based on the Bitcoin Script format. +== Background == + +* Assume two actors, a prover P and a verifier V. +* P wants to prove that they own the private key k associated with a given address A (which in turn is derived from the pubkey kG). +* Let V generate a message M and hand this to P. +* P generates a signature S by signing the message M using k. Given S, V can prove that P has the private key associated with A. + +The astute reader will notice that the above is missing a critical part, namely the pubkey kG, without which the verifier cannot actually verify the message. The current message signing standard solves this via a cryptographic trick, wherein the signature S above is a special "recoverable signature" type. Given the message M and the signature S, it is then possible to recover the pubkey kG. The system thus derives the address for the pubkey kG, and if it does not match A, the proof is deemed invalid. + +While this is a neat trick, it unnecessarily restricts and complicates the message signing mechanism; for instance, it is currently not possible to sign a message for a P2SH address, because there is no pubkey to recover from the resulting signature. + == Motivation == The current message signing standard only works for P2PKH (1...) addresses. By extending it to use a Bitcoin Script based approach, it could be made more generic without causing a too big burden on implementers, who most likely have access to Bitcoin Script interpreters already. -- cgit v1.2.3 From b3cec02aa4170bd92f6ec61c1e6a440017c35b9c Mon Sep 17 00:00:00 2001 From: Karl-Johan Alm Date: Thu, 8 Aug 2019 01:08:12 +0900 Subject: bip-322: clarify when to return ERROR in verify action --- bip-0322.mediawiki | 2 ++ 1 file changed, 2 insertions(+) diff --git a/bip-0322.mediawiki b/bip-0322.mediawiki index a4973d8..d936e1e 100644 --- a/bip-0322.mediawiki +++ b/bip-0322.mediawiki @@ -121,6 +121,8 @@ The resulting signature proof should be encoded using base64 encoding. The "Verify" action takes as input a standard flags value, a script sig, an optional witness, and a purpose. It emits one of INCONCLUSIVE, VALID, INVALID, or ERROR. +While omitted below, ERROR is returned if an unforeseen error occurs at any point in the process. A concrete example of this is if a legacy proof is given as input to a non-legacy address; the deserialization of the proof will fail in this case, and this should result in an ERROR result. + # Obtain the sighash and scriptPubKey from the purpose; pass on result code if not VALID # Verify Script with flags=consensus flags (currently P2SH, DERSIG, NULLDUMMY, CLTV, CSV, WITNESS), scriptSig=script sig, scriptPubKey=scriptPubKey, witness=witness, and sighash=sighash # Return INVALID if verification fails -- cgit v1.2.3 From 71529345e8145b2827b878f9c72f021d540ee89e Mon Sep 17 00:00:00 2001 From: Clark Moody Date: Mon, 26 Aug 2019 17:15:18 -0500 Subject: Switch from Markdown-style code to Mediawiki --- bip-0045.mediawiki | 36 ++++++++++++++++++------------------ 1 file changed, 18 insertions(+), 18 deletions(-) diff --git a/bip-0045.mediawiki b/bip-0045.mediawiki index f4d200c..d721582 100644 --- a/bip-0045.mediawiki +++ b/bip-0045.mediawiki @@ -16,7 +16,7 @@ This BIP defines a structure for hierarchical deterministic P2SH multi-party multi-signature wallets (HDPM wallets from now on) based on the algorithm -described in BIP-0032 (BIP32 from now on) and purpose scheme described in +described in BIP-0032 (BIP32 from now on) and purpose scheme described in BIP-0043 (BIP43 from now on). This BIP is a particular application of BIP43. @@ -63,7 +63,7 @@ Hardened derivation is used at this level. The index of the party creating a P2SH multisig address. The indices can be determined independently by lexicographically sorting the purpose public keys of each cosigner. Each cosigner creates addresses on its own branch, -even though they have independent extended master public key, as explained +even though they have independent extended master public key, as explained in the "Address generation" section. Note that the master public key is not shared amongst the cosigners. Only the @@ -79,12 +79,12 @@ purpose public keys: 03f76588e06c0d688617ef365d1e58a7f1aa84daa3801380b1e7f12acc9a69cd13
-it should use `m / 45 ' / 0 / *` for -`039863fb5f07b667d9b1ca68773c6e6cdbcac0088ffba9af46f6f6acd153d44463`, -`m / 45 ' / 1 / *` for -`03a473275a750a20b7b71ebeadfec83130c014da4b53f1c4743fcf342af6589a38`, -and `m / 45 ' / 2 / *` for -`03f76588e06c0d688617ef365d1e58a7f1aa84daa3801380b1e7f12acc9a69cd13`, +it should use m / 45 ' / 0 / * for +039863fb5f07b667d9b1ca68773c6e6cdbcac0088ffba9af46f6f6acd153d44463, +m / 45 ' / 1 / * for +03a473275a750a20b7b71ebeadfec83130c014da4b53f1c4743fcf342af6589a38, +and m / 45 ' / 2 / * for +03f76588e06c0d688617ef365d1e58a7f1aa84daa3801380b1e7f12acc9a69cd13, as dictated by their lexicographical order. @@ -102,7 +102,7 @@ chain is used for addresses which are not meant to be visible outside of the wallet and is used for return transaction change. For example, if cosigner 2 wants to generate a change address, he would use -`m / 45 ' / 2 / 1 / *`, and `m / 45 ' / 2 / 0 / *` for a receive +m / 45 ' / 2 / 1 / *, and m / 45 ' / 2 / 0 / * for a receive address. Non-hardened derivation is used at this level. @@ -118,7 +118,7 @@ Non-hardened derivation is used at this level. Each party generates their own extended master keypair and shares the extended purpose' public key with the others, which is stored encrypted. Each party can generate any of the other's derived public keys, but only -his own private keys. +his own private keys. ===Address Generation Procedure=== When generating an address, each party can independently generate the N needed @@ -137,18 +137,18 @@ others using the next index, and calculate the needed script for the address. Example: Cosigner #2 wants to receive a payment to the shared wallet. His last used index on his own branch is 4. Then, the path for the next receive -address is `m/45'/2/0/5`. He uses this same path in all of the cosigners +address is m/45'/2/0/5. He uses this same path in all of the cosigners trees to generate a public key for each one, and from that he gets the new p2sh address. ====Change address case==== Again, each cosigner generates addresses only on his own branch. One of the n cosigners wants to create an outgoing payment, for which he'll need a change address. He generates a new address using the same procedure as above, but -using a separate index to track the used change addresses. +using a separate index to track the used change addresses. Example: Cosigner #5 wants to send a payment from the shared wallet, for which he'll need a change address. His last used change index on his own branch is -11. Then, the path for the next change address is `m/45'/5/1/12`. He uses +11. Then, the path for the next change address is m/45'/5/1/12. He uses this same path in all of the cosigners trees to generate a public key for each one, and from that he gets the new p2sh address. @@ -163,7 +163,7 @@ that specific address (using the same path that generated the public key in that address, but deriving the private key instead), and sign it. Once the proposal reaches m signatures, any cosigner can broadcast it to the network, becoming final. The specifics of how this proposal is structured, and the -protocol to accept or reject it, belong to another BIP, in my opinion. +protocol to accept or reject it, belong to another BIP, in my opinion. ===Address discovery=== @@ -171,8 +171,8 @@ When the master seed is imported from an external source the software should start to discover the addresses in the following manner: # for each cosigner: -# derive the cosigner's node (`m / 45' / cosigner_index`) -# for both the external and internal chains on this node (`m / 45' / cosigner_index / 0` and `m / 45' / cosigner_index / 1`): +# derive the cosigner's node (m / 45' / cosigner_index) +# for both the external and internal chains on this node (m / 45' / cosigner_index / 0 and m / 45' / cosigner_index / 1): # scan addresses of the chain; respect the gap limit described below Please note that the algorithm uses the transaction history, not address @@ -182,7 +182,7 @@ even if the earlier ones don't have transactions ===Address gap limit=== -Address gap limit is currently set to 20. If the software hits 20 unused +Address gap limit is currently set to 20. If the software hits 20 unused addresses (no transactions associated with that address) in a row, it expects there are no used addresses beyond this point and stops searching the address chain. @@ -198,7 +198,7 @@ parties. Here are some explanations about the design decisions made. The reason for using separate branches for each cosigner is we don't want two of them generating the same address and receiving simultaneous payments to it. The ideal case is that each address receives at most one payment, -requested by the corresponding cosigner. +requested by the corresponding cosigner. ==Examples== -- cgit v1.2.3 From c28773f34ee383dd9278e127ffd4236666a7284e Mon Sep 17 00:00:00 2001 From: Karl-Johan Alm Date: Wed, 28 Aug 2019 15:02:06 +0900 Subject: bip322: fix test-vectors --- bip-0322.mediawiki | 27 +++++++++++++++++++++------ 1 file changed, 21 insertions(+), 6 deletions(-) diff --git a/bip-0322.mediawiki b/bip-0322.mediawiki index d936e1e..aaf5496 100644 --- a/bip-0322.mediawiki +++ b/bip-0322.mediawiki @@ -230,14 +230,29 @@ All of the above, plus (subject to change): == Native segwit test vector == -* address = bcrt1qe7nte4zk4ayly5tc53dtdjupgkz0lr8azx3rzz -* message = hello -* sighash = 790eef86c204f0bff969ff822121317aa34eff0215dbd30ccf031e7b2f3f0cc1 (sha256d("Bitcoin Signed Message:\n:hello")) +
+address      = bcrt1qe7nte4zk4ayly5tc53dtdjupgkz0lr8azx3rzz
+scriptpubkey = 0014cfa6bcd456af49f25178a45ab6cb814584ff8cfd
+message      = hello
+preimage     = 0014cfa6bcd456af49f25178a45ab6cb814584ff8cfd426974636f696e205369
+               676e6564204d6573736167653a0a68656c6c6f
+               (scriptpubkey || "Bitcoin Signed Message:\nhello")
+sighash      = 790eef86c204f0bff969ff822121317aa34eff0215dbd30ccf031e7b2f3f0cc1
+               (sha256d(preimage), displayed in big-endian)
+
-A possible proof is: +The proof becomes: -* HEX: 01000000010002473044022075b4fb40421d55c55462879cb352a85eeb3af2138d3f02902c9143f12870f5f70220119c2995c1661138142f3899c1fd6d1af7e790e0e081be72db9ce7bf5b5b932901210290beccd02b73eca57467b2b6f1e47161a9b76a5e67586e7c1dee9ea6e2dcd869 -* Base64: AQAAAAEAAkcwRAIgdbT7QEIdVcVUYoecs1KoXus68hONPwKQLJFD8Shw9fcCIBGcKZXBZhE4FC84mcH9bRr355Dg4IG+ctuc579bW5MpASECkL7M0Ctz7KV0Z7K28eRxYam3al5nWG58He6epuLc2Gk= +
+HEX:    01000000010002473044022075b4fb40421d55c55462879cb352a85eeb3af2138d3f0290
+        2c9143f12870f5f70220119c2995c1661138142f3899c1fd6d1af7e790e0e081be72db9c
+        e7bf5b5b932901210290beccd02b73eca57467b2b6f1e47161a9b76a5e67586e7c1dee9e
+        a6e2dcd869
+
+Base64: AQAAAAEAAkcwRAIgdbT7QEIdVcVUYoecs1KoXus68hONPwKQLJFD8Shw9fcCIBGcKZXBZhE4
+        FC84mcH9bRr355Dg4IG+ctuc579bW5MpASECkL7M0Ctz7KV0Z7K28eRxYam3al5nWG58He6e
+        puLc2Gk=
+
Split into components: -- cgit v1.2.3 From b4e6b0fac257b773afd370d23f243ae22f8ab142 Mon Sep 17 00:00:00 2001 From: darosior Date: Mon, 16 Sep 2019 13:16:11 +0200 Subject: bip-0174: remove duplicated 'only' (typo) --- bip-0174.mediawiki | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bip-0174.mediawiki b/bip-0174.mediawiki index b4a6407..2a73053 100644 --- a/bip-0174.mediawiki +++ b/bip-0174.mediawiki @@ -313,7 +313,7 @@ If a Signer cannot sign a transaction, it must not add a Partial Signature. The Signer can additionally compute the addresses and values being sent, and the transaction fee, optionally showing this data to the user as a confirmation of intent and the consequences of signing the PSBT. -Signers do not need to sign for all possible input types. For example, a signer may choose to only sign only Segwit inputs. +Signers do not need to sign for all possible input types. For example, a signer may choose to only sign Segwit inputs. A single entity is likely to be both a Signer and an Updater as it can update a PSBT with necessary information prior to signing it. -- cgit v1.2.3 From 4d47fb24aaff26dfae438fbeb24685bd1e980c9b Mon Sep 17 00:00:00 2001 From: Vincenzo Palazzo Date: Wed, 18 Sep 2019 11:22:24 +0200 Subject: Update bip-0013.mediawiki --- bip-0013.mediawiki | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bip-0013.mediawiki b/bip-0013.mediawiki index 081ea2a..c5a4354 100644 --- a/bip-0013.mediawiki +++ b/bip-0013.mediawiki @@ -50,7 +50,7 @@ This proposal is not backwards compatible, but it fails gracefully-- if an older ==Reference Implementation== -See base58.cpp1/base58.h at https://github.com/bitcoin/bitcoin/src +See base58.cpp1/base58.h at https://github.com/bitcoin/bitcoin/tree/master/src ==See Also== -- cgit v1.2.3 From aeda6226dd799c02af260a3582768f839cf6316d Mon Sep 17 00:00:00 2001 From: Vincenzo Palazzo Date: Wed, 18 Sep 2019 11:23:47 +0200 Subject: Update bip-0013.mediawiki --- bip-0013.mediawiki | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bip-0013.mediawiki b/bip-0013.mediawiki index c5a4354..70be90d 100644 --- a/bip-0013.mediawiki +++ b/bip-0013.mediawiki @@ -50,7 +50,7 @@ This proposal is not backwards compatible, but it fails gracefully-- if an older ==Reference Implementation== -See base58.cpp1/base58.h at https://github.com/bitcoin/bitcoin/tree/master/src +See base58.cpp/base58.h at https://github.com/bitcoin/bitcoin/tree/master/src ==See Also== -- cgit v1.2.3 From 2eca4d32c8284952ba4cbe19a1db7a23767482b9 Mon Sep 17 00:00:00 2001 From: Luke Dashjr Date: Thu, 19 Sep 2019 22:16:51 +0000 Subject: Update BIP 102 status Draft->Rejected --- README.mediawiki | 4 ++-- bip-0102.mediawiki | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/README.mediawiki b/README.mediawiki index c8a1596..0f21815 100644 --- a/README.mediawiki +++ b/README.mediawiki @@ -455,13 +455,13 @@ Those proposing changes should consider that ultimately consent may rest with th | Gavin Andresen | Standard | Withdrawn -|- +|- style="background-color: #ffcfcf" | [[bip-0102.mediawiki|102]] | Consensus (hard fork) | Block size increase to 2MB | Jeff Garzik | Standard -| Draft +| Rejected |- | [[bip-0103.mediawiki|103]] | Consensus (hard fork) diff --git a/bip-0102.mediawiki b/bip-0102.mediawiki index ed6b4e3..5a2c91a 100644 --- a/bip-0102.mediawiki +++ b/bip-0102.mediawiki @@ -5,7 +5,7 @@ Author: Jeff Garzik Comments-Summary: No comments yet. Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0102 - Status: Draft + Status: Rejected Type: Standards Track Created: 2015-06-23
-- cgit v1.2.3 From f1aff33f46a8787d54813b25b6792d3d16902934 Mon Sep 17 00:00:00 2001 From: Kalle Rosenbaum Date: Fri, 20 Sep 2019 18:57:17 +0200 Subject: BIP174: Remove misleading sentence The sentence seems to suggest that the "master key fingerprint" can be the fingerprint of any intermediate node on the derivation path, which isn't true. --- bip-0174.mediawiki | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bip-0174.mediawiki b/bip-0174.mediawiki index 75a4df9..7deb0b1 100644 --- a/bip-0174.mediawiki +++ b/bip-0174.mediawiki @@ -205,7 +205,7 @@ determine which outputs are change outputs and verify that the change is returni * Type: BIP 32 Derivation Path PSBT_OUT_BIP32_DERIVATION = 0x02 ** Key: The public key *** {0x02}|{public key} -** Value: The master key fingerprint concatenated with the derivation path of the public key. The derivation path is represented as 32 bit unsigned integer indexes concatenated with each other. This must omit the index of the master key. Public keys are those needed to spend this output. +** Value: The master key fingerprint concatenated with the derivation path of the public key. The derivation path is represented as 32 bit unsigned integer indexes concatenated with each other. Public keys are those needed to spend this output. *** {master key fingerprint}|{32-bit int}|...|{32-bit int} The transaction format is specified as follows: -- cgit v1.2.3 From dd02ff4c07736f4e95b9f69866acdb17428cdf2e Mon Sep 17 00:00:00 2001 From: Paul Sztorc Date: Mon, 23 Apr 2018 19:13:45 -0400 Subject: edit and shorten slightly --- bip-hashrate-escrows.mediawiki | 469 ++++++++++++++++++++++++++++++++++++ bip-hashrate-escrows/appendix-1.txt | 20 ++ 2 files changed, 489 insertions(+) create mode 100644 bip-hashrate-escrows.mediawiki create mode 100644 bip-hashrate-escrows/appendix-1.txt diff --git a/bip-hashrate-escrows.mediawiki b/bip-hashrate-escrows.mediawiki new file mode 100644 index 0000000..0a61b47 --- /dev/null +++ b/bip-hashrate-escrows.mediawiki @@ -0,0 +1,469 @@ + + +
+    BIP: ????
+    Layer: Consensus (soft fork)
+    Title: Hashrate Escrows (Consensus layer)
+    Author: Paul Sztorc 
+            CryptAxe 
+    Comments-Summary: No comments yet.
+    Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-???????
+    Status: Draft
+    Type: Standards Track
+    Created: 2017-08-14
+    License: BSD-2-Clause
+    Post-History: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-May/014364.html
+
+ +==Abstract== + +A "Hashrate Escrow" is a clearer term for the concept of "locked to an SPV Proof", which is itself a restatement of the phrase "within a sidechain" as described in [https://blockstream.com/sidechains.pdf a famous Oct 2014 paper] written partially by some Blockstream co-founders. + +A Hashrate Escrow resembles a 2-of-3 multisig escrow, where the 3rd party (who will arbitrate any disputes) is a decentralized group of people: the dynamic-membership set of Bitcoin Miners. However, the 3rd party does not sign escrow-withdrawal transactions with a private key. Instead, these are "signed" by directing hashpower over them for a period of time. + +This project has [http://www.drivechain.info/ a website] which includes [http://www.drivechain.info/faq/index.html a FAQ]. + + +==Motivation== + +In practice these escrows are likely to be "asymmetric sidechains" of Bitcoin (such as [http://www.rsk.co/ Rootstock]) or "virtual chains" within Bitcoin (such as [https://github.com/blockstack/virtualchain proposed by Blockstack] in mid-2016). + +Sidechains have many potential benefits, including: + +1. Protect Bitcoin from competition from altcoins and spinoffs. Safely allow competing implementations (of *sidechains*). +2. Protect Bitcoin from hard fork campaigns. (Such campaigns represent an existential threat to Bitcoin, as well as an avenue for developer corruption.) +3. Help with review, by making it much easier for reviewers to ignore bad ideas. +4. Provide an avenue for good-but-confusing ideas to prove their value safely. + + + +==Specification== + +==== Components ==== + +Hashrate Escrows are built of two types of component: [1] new databases, and [2] new message-interpretations. + +===== 1. New Databases ===== + +* D1. "Escrow_DB" -- a database of "accounts" and their attributes. +* D2. "Withdrawal_DB" -- a database of pending withdrawals from these accounts, and their statuses. + +Please note that these structures (D1 and D2) will not literally exist anywhere in the blockchain. Instead they are constructed from messages...these messages, in contrast, *will* exist in the blockchain (with the exception of M4). + +===== 2. New Messages ===== + +* M1. "Propose New Escrow" +* M2. "ACK Escrow Proposal" +* M3. "Propose Withdrawal" +* M4. (implied) "ACK Withdrawal" +* M5. "Execute Deposit" -- a transfer of BTC from-main-to-side +* M6. "Execute Withdrawal" -- a transfer of BTC from-side-to-main + + +==== On the Resource Requirements of New Databases ==== + +The "new" databases are simply reinterpretations of data that are already contained elsewhere in the blockchain. Specifically, M1 M2 and M3 are all located in the block's coinbase txn, and M5 and M6 might be found in any regular txn. M4 is a special case and does not actually need to be included anywhere, so it is not. If you like, you can imagine that the M4s reside in an optional extension block. + +In other words, we just rearrange what is already there. Because of this, even though "new databases" are created and stored in memory, the existing bandwidth and storage limits are respected (although, see "M4" below). + + + + +=== Adding Sidechains and Tracking Them (D1, M1, M2) === + +==== D1 -- "Escrow_DB" ==== + +The table below enumerates the new database fields, their size in bytes, and their purpose. In general, an escrow designer (for example, a sidechain-designer), is free to choose any value for these. + +Note: Fields 6 through 9 have been intentionally removed. Previously, this section allowed miners to set and commit to voting/waiting periods. However, I have since standardized the periods: withdrawals expire after 6 months (26298 blocks), and they succeed if they ever achieve an ACK score of 13140 or higher. I have removed the waiting period, because anyone who adopts a policy of ignoring all withdrawals with fewer than 400 ACKs will automatically gain all of the benefits of the waiting period. The justification for this change is that it strongly implies that an attack on any one sidechain is an attack on all of them (in a sense, this change makes the "victimhood" of each sidechain "fungible"). + +{| class="wikitable" +! Field No. +! Label +! Bytes +! Type +! Description / Purpose +|- +| 1 +| Escrow Number +| 1 +| uInt +| A number assigned to the entire escrow. Used to make it easy to refer to each escrow. +|- +| 2 +| Active* +| 2 +| uInt +| This counter starts at zero when a new entry is added, and increments as a new entry is supported by "M2". The entry either reaches the max value of 4032 (at which point the counter freezes), or else the entry is removed. This enforces the requirement "a soft fork is required to add a new escrow". +|- +| 3 +| Escrow Name/Description +| 120 +| string +| A human-readable name and description of the sidechain. More than enough space to hold a 32 byte hash. Helps prevent destructive interference among sidechains (see below). +|- +| 4 +| Critical Private Key +| 32 +| hex +| This is the Double-SHA256 of the binary release of the reference software. It ties the sidechain to a specific software release, and doubles as a way of signing withdrawals (see below). +|- +| 5 +| Critical Address* +| 32 +| string +| This is derived by taking f(x) = ImportPrivateKey(HexToBase58Check(x)) of field #3. It is intentionally in the human-readable format {{although it could be 25 bytes of binary}}. +|- +| 10 +| "CTIP" -- Part 1 "TxID"* +| 32 +| hex +| The CTIP, or "Critical (TxID, Index) Pair" is a variable for keeping track of where the escrow's money is (ie, which member of the UTXO set). +|- +| 11 +| "CTIP" -- Part 2 "Index"* +| 4 +| hex +| Of the CTIP, this is second element of the pair: the Index. See #10 above. +|} + +\* Denotes a "convenience field", the entry for this field is derived from other fields, or from the blockchain-state itself. The escrow-creator does not need to provide these values in M1 (or anywhere). + +Escrow_DB requires 223 bytes [1+120+32+32+2+32+4] for each escrow in the main blockchain. Of these, 70 bytes [2+32+32+4] are merely for convenience. Therefore, a sidechain is defined (see "M1") by 153 bytes of information. + +====Notes on D1==== + +# D1 will always exist. +# D1 will always have a unique sort (first by "Escrow Number" (field #1), and second by "Active" (field #2)). There is only ever one (escrow number, Active) pair. +# D1 is updated according to M1 and M2 (below). +# If a new entry is added to D1 with an "Escrow Number" that is already in use, then this entry will either eventually be removed (because it was not supported with an M2), or it will eventually overwrite the old entry (if it *was* supported via M2). + + +====Notes on D1==== + +=====Obligations Placed on Miners===== + +Miners have always upgraded their software according to criteria that are known only to them (in other words, "whenever they want"). + +However, this soft fork imposes two new criteria upon them. First: miners should only upgrade their software, if any modification to the portfolio of sidechains [that are added/removed in the upgrade] can be expected to increase miner wealth. Trivially, this implies that miners should make sure that the upgrade doesn't overwrite (and destroy) an existing sidechain that they like! But, more seriously, it implies that miners should take an interest in what the sidechain is doing to the mainchain and other sidechains (see below). + +===== Destructive Sidechain Interference ===== + +People frequently emphasize that miners should have "as little control" as possible. It is a very safe claim to make, and a very easy sentence to write. Much harder is to determine exactly what this minimum value is, and how to achieve it. Harder still is to untie the knot of who is actually controlling what, in a decentralized, interacting system. + +Certainly, miners can not have "zero control" -- for that is the same as to just remove them from the system altogether. Some rules are enforced "on miners by nodes" (such as the infamous blocksize limit); other rules are enforced by nodes but are narrowly-controlled by miners (such as the proof-of-work itself, or the block's timestamp). Thirdly, some rules are enforced by both against each other (such as the rule against including invalid txns or double-spent txns), for mutual benefit. + +Some pause should be given, after one considers that the sidechain design goal is literally a piece of software that can do *anything*. Anything includes a great many things, many of which I demonstrate to be undesirable. Bitcoin itself does not allow "anything" -- it allows any person to transact, but, in contrast, it does not permit any person to double-spend. This is because "allowing anyone to do anything" is not viable in a world that contains undesirable interactions (what a libertarian might call "aggression") -- in the case of money, these are theft and counterfeiting. + +I have produced a comprehensive quantity of written material [1], presentations [2], etc [3] on exactly what the level of miner-control should be, and why. Specifically, I claim that **miners should be aware of the purpose of the sidechain, and they should reject sidechains which have an unclear purpose or which have a purpose that will lead to decrease in miner-wealth** (where wealth measured explicitly as: the estimated present value of the purchasing power of the blockchain's coinbase txns). I claim that this criterion is necessary because, just Original Bitcoin filters unwanted interactions among different BTC txns, so too much "Sidechain Bitcoin" filter out unwanted interactions among sidechain. + +* [1] http://www.truthcoin.info/blog/wise-contracts/ +* [2] https://www.youtube.com/watch?v=xGu0o8HH10U&index=1&list=PLw8-6ARlyVciMH79ZyLOpImsMug3LgNc4 +* [3] http://www.drivechain.info/literature/index.html + +Call it a "sidechain non-aggression principle", if you want. + +To the best of my knowledge, everyone who *has* reviewed this information as found the arguments to be acceptable. It has, also, changed a few minds (from "unacceptable" to "acceptable"). + + +===== ISSUE: "Signing" BTC Txns ===== + +Currently, we use a process which may be suboptimal. It is that we *literally sign* a txn with a globally and publicly known private key. But this is for convenience purposes -- to easily detect the sidechain's balance. The signature that is produced is not doing anything. This is probably an area of improvement. + + +( The following messages were modeled on SegWit -- https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki#commitment-structure ) + + +==== M1 -- "Propose New Sidechain" ==== + + 1-byte - OP_RETURN (0x6a) + 1-byte - Push the following 157 bytes (0x9d) + 4-byte - Commitment header (0x53707243) + 153-byte - the critical bytes mentioned above (fields #1, #3, and #4, to populate a new D1 entry) + + +==== New Block Validation Rules ==== + +# If the network detects a properly-formatted M1, it must add an entry to D1, into the very next block, with the following initial values: +## Field #5 will be calculated as per [https://en.bitcoin.it/wiki/Technical_background_of_version_1_Bitcoin_addresses version 1 Bitcoin addresses], but with a prefix of "4" instead of "1". +## Field #9 will be derived from #7 and #8 using math. +## The initial values of Fields #10, #11, and #12 are set to zero. +# Only one M1 (of any kind) can be added into a block at a time. + +==== Notes on M1 ==== + +The escrow will "reuse" the same address over and over. But notice that there is no privacy loss due to address reuse, because the address in question does not "belong" to any specific individual. Furthermore, the transactions in question could already be linked by just watching the Hashrate Escrow for deposits ...this, in fact, is exactly what a sidechain must do in order to make main-to-side transfers. + + +==== M2 -- "ACK Sidechain Proposal" ==== + + 1-byte - OP_RETURN (0x6a) + 1-byte - Push the following 29 bytes (0x1D) + 4-byte - Commitment header (0x53616343) + 25-byte - Commitment hash: RIPMD-160 of Sha256 of a given M1 (above) + +==== New Block Validation Rules ==== + +1. An entry will be removed from D1, unless that block contains an M2 supporting it (this will increment field #2), or the entry has accumulated 4032 M2s. Therefore, each entry needs to have 4032 *consecutive* M2s supporting it, else it will be removed from D1 and must start over. This requires all miners to agree to "soft fork" the sidechain into existence. This makes it easier to curate the list of active sidechains, and prevents destructive interference (see above) among contradictory sidechains. +2. Once the entry has 4032 M2s, then a miner must make a 0 btc payment to the Critical Address, thus activating the chain. This payment becomes the first "critical (TxID, index) pair" (or "CTIP"). +3. It is possible to "overwrite" a D1 slot -- this destroys the sidechain that is there, and replaces it with a new one. This is defined as requiring 6 months (26298 blocks) of M2s, instead of 4 weeks (4032). This possibility does not change the security assumptions, because we already assume that users have access to "magically true" bandwidth at a rate of 1 bit per ACK-cycle. (This is why the voting cycles should be large, 1 month Waiting, 2+ months Voting). + + + +=== Adding Withdrawal-Attempts and Tracking Their Success (D2, M3, M4) === + +==== D2 -- "Withdrawal_DB" ==== + +The table below enumerates the database fields, their size (in bytes), type and purpose. + + +{| class="wikitable" +! Field No. +! Label +! Bytes +! Type +! Description / Purpose +|- +| 1 +| Escrow Number +| 1 +| uInt +| Links the withdrawal-request to a specific escrow. +|- +| 2 +| WT^ +| 32 +| hex +| This is a "blinded transaction id" (ie, the double-Sha256 of a txn that has had two fields zeroed out, see M6) of a withdrawal-attempt. +|- +| 3 +| ACKs* +| 2 +| uInt +| The current total number of "votes", this starts at 0 and remains there throughout the waiting period. +|- +| 4 +| Age* +| 3 +| uInt +| Total duration of time, in blocks, that this WT^ has been inside of D2. +|- +| 5 +| Waiting Period* +| 2 +| uInt +| Total duration of time, in blocks, that this entry must sit idle, before it can begin to accumulate ACKs/NACKs. Pulled from D1's field #6. +|- +| 6 +| Max Age* +| 3 +| uInt +| Determined by summing (D1's field #6) and (D1's field #7). +|- +| 7 +| Threshold* +| 2 +| uInt +| Total ACKs needed, this is pulled from D1's field #9. +|- +| 8 +| Approved* +| 1 +| boolean +| True while ACKs > Threshold, False otherwise. +|} + +\* Denotes a "convenience field" (see above). + +Withdrawal_DB requires 46 bytes [1+32+2+3+2+3+2+1] per entry. Of these, 13 bytes ([2+3+2+3+2+1], all fields except #1 and #2) can be generated locally, leaving 33 critical bytes [1+32]. + +==== New Block Validation Rules for D2 ==== + +# In each block, a hash commitment to D2 must always exist (even if D2 is blank). +# D2 must always be sorted first by field #1 (Escrow Number) and second by field #4 (Age). This imposes a unique sort. +# From one block to the next, every entry's "Age" field must increase by exactly 1. +# From one block to the next, entries are only removed from D2 (in the very next block) if: +## "Age" = "MaxAge". +## If the block contains a txn who's blinded txID matches WT^. {{ This might be unnecessary, and a lot of work. }} +# In addition, there are special rules for the allowed values in the "ACKs" field (field #3). See M4 below. + +==== M3 -- "Propose Withdrawal" ==== + + 1-byte - OP_RETURN (0x6a) + 1-byte - Push the following 37 bytes (0x25) + 4-byte - Commitment header (0xD45AA943) + 33-byte - the critical bytes mentioned above (fields #1 and #2, to populate a new D2 entry) + + +==== New Block Validation Rules for M3 ==== + +# If the network detects a properly-formatted M3, it must add an entry to D2 in the very next block. The starting values of fields #3 and #4 are zero, and #5 is pulled over by extracting the relevant value from D1. +# Each block can only contain one M3 per sidechain. + + +==== M4 -- "ACK Withdrawal" ==== + +==== Very Little Info, Probably Calculable in Advance ==== + +M4 is exceptional (in comparison to the other M's) in a few ways. First, its content is not stored anywhere, only the *hash* of its *effect* is stored (in a leaf of a merkle tree who's root is inserted into a mainchain coinbase). M4 alters the contents of D2 -- the *contents* of D2 are consensus critical, but M4 (the process by which nodes reach a new valid D2) can be anything. + +In fact, M4 can also be *nothing*. In other words, it may be optional. This is precisely because, from one block to the next, we have constrained D2 such that it is only allowed to change in a few ways. Therefore, the exhaustive set of "candidate D2s" can be precomputed by full nodes in advance. + +The following sections assume a maximum of one sucessful withdrawal per sidechain at a time (see [bip-hashrate-escrows/appendix-1.txt appendix 1]). + +==== How Hard is it to Guess M4? ==== + +If there are n Escrows and m Withdrawals-per-escrow1, then there are (m+2)^n total candidates for the next D2. This is because, [per block per escrow], one of three things can happen: (1) one of the m withdrawal-candidates can be "ACK"ed (or "upvoted" or "promoted"), which automatically downvotes the others; or (2) all withdrawal-candidates can be downvoted, or finally (3) the miners can abstain from voting on the escrow's withdrawals altogether, leaving the tallies the same. + +First, for nodes which validate all sidechains (assuming these escrows are sidechains), this simplifies to 2^n -- these nodes only have to choose between the single honest choice (on one hand) or an abstention (on the other). Second, even for nodes that don't validate any sidechains, the number of candidates might be reduced from m^n to 3^n, by making a simplifying assumption: whichever withdrawal was most recently added/upvoted, is likely to be the one which is upvoted next. + +Of course, that is still O(k^n) for n sidechains, which isn't great2. If the "D2 update" cannot be guessed, it must be transmitted in some way. + +==== Giving Up and Getting M4 the Old Fashioned Way ==== + +Two examples for transmitting it are below: + +"Short Form" (Assumes there are no more than 254 active withdrawal-attempts per account) + + 4-byte - Message identifier (0x????????) + 1-byte - Version of this message + N-byte - N is the total number of active accounts ("sidechains"), each byte specifies the position of the single WT that was "upvoted". A value of 0 indicates "downvote everything", a value of 255 indicates abstention. + +"Long Form" (Makes no assumptions about anything) + + 4-byte - Message identifier (0x????????) + 1-byte - Version of this message + 1-byte - Length (in bytes) of this message; total number of withdrawal attempts; y = ceiling( sum_i(m_i +2)/8 ). Nodes should already know what length to expect, because they know the sequence of M3s and therefore the vector of WT^s. + Y-byte - stream of bits (not bytes), with a 1 indicating the position of the chosen action [downvote all, abstain, upvote1, upvote2, ...] + + +If the message is very very large, then nodes may not want to broadcast it. This opens up an "exhaustion attack"2, in which many miners create bad WT^s, vote on these randomly, and then refuse to broadcast their votes. Fortunately, even for a worst-case scenario of 200 sidechains and 1,000 withdrawal-attempts per sidechain, honest nodes can communicate a long form M4 with each other by using just 25,056 bytes per block [4+1+1+(200\*(1000+1+1)/8)]. + +Today's pre-drivechain miners can already carry out a similar attack, by creating and including txns and then not broadcasting that part of the block to anyone. This is often characterized as a [https://petertodd.org/2016/block-publication-incentives-for-miners "block publication incentive"], because in that case the prospect of exhaustively computing all possible transactions (to uncover the missing ones) is completely out of the question. + +However, message M4 is different from a withheld-txn, because M4 operates outside of the block's mandated information-processing limits (ie, outside the infamous 1 MB nonwitness blocksize limit). So we should examine the conditions under which M4 grows and shrinks, to ensure that we are not smuggling in a tremendous burden on full nodes. + +Under adversarial conditions, to lengthen a long-form M4 by one bit per block, for C blocks, the attacker must pay 312 bits (39 bytes) one time (to embed a new M3 message). The value C is the length of the sidechain's voting period, which varies but which I expect to be approximately 8,064 (and which could theoretically be as high as 65,536). Thus the attacker can burden nodes disproportionately, if (s)he wishes. + +Fortunately, the attack in question has no motivation (as far as I can tell). If the miner's goal is to trick rivals into mining on top of invalid blocks, he can already do this much more effectively with the unpublished-txn method (above). If instead he is just trying to harass nodes, then nodes may freely "downgrade" to earlier versions of the protocol, and simply ignore all drivechain-related messages. It seems that the attack could best be used in order to: make a large D2, make D2 confusing, sneak in votes for evil WT^ lurking in D2. Thus, the attack disables the transparency of the drivechain system, to some extent. The cost of the attack is forgone transaction fees, due to block space wasted on useless M3s. + +In practice, n is already capped, and miners may impose [on each other] a "soft cap" on m for their mutual protection. Thus, n and m might never get above 10 and 30, respectfully. In this case, the [Short Form, this time] M4 can never require more than 15 bytes per block, no matter what the attacker tries. + +In practice, m should always be 1 or 2, else something fishy is going on; and m can only inch up by 1 unit per block. So the system as a whole is still quite transparent, in that users are warned appropriately and well in advance. Attackers must invest upfront and they face an uphill climb, in order to eventually make things more expensive for a few others; defenders can wait-and-see if the attack looks like it will ever amount to anything before lifting a finger. + + +===== New Block Validation Rules (for D2 and, by implication, M4) ===== + +From one block to the next, D2 can only be edited in a few strict ways: + +* Entries can only be added/removed from D2 if they meet the criteria above (in M3, and implicitly M1 and M2). +* The ACK-counter of any individual entry can only change by (-1,0,+1) relative to its previous entry. +* Within a sidechain group, upvoting one withdrawal (ACK=ACK+1) requires you to downvote all other withdrawals in that group. However, the minimum ACK value is zero (and, therefore, downvotes cannot reduce it below zero). + +===== Footnotes for M4 ===== + +1 This represents the worst-case scenario is one where all the Withdrawals are spread evenly over each Sidechain. Under normal operations, there is no reason to expect the all sidechains will have the same number of withdrawals at any given time. In fact, under normal operations, the very *concept* of counting the withdrawals-per-sidechain should be a purposeless one, because there should only be *one* withdrawal at a time. Nonetheless we consider the worst case scenario here. + +2 Guessing becomes more computationally intensive in a highly adversarial situation where the "limited range" is intentionally expanded. In such a scenario, [a] there are many sidechains, and [b] miners voluntarily sacrifice their scarce block-space by creating a high number of (mutually-exclusive, and hence ultimately invalid) withdrawal attempts and putting these into coinbase transactions; and then agree to all [c] vote on these randomly (guaranteeing that all withdrawals fail, including any true withdrawals) and [d] successfully withhold their random voting strategies from nodes (even including spy-miner-nodes). Under this bizarre scenario, nodes may require computing resources which increase near-exponentially with the number of withdrawals, and it may take a long time for an ignorant node to exhaustively work out the underlying state of Withdrawal_DB. In this case, nodes may decide to temporarily stop validating such transactions (as if they had not yet upgraded to support this soft fork). + + + +=== Depositing and Withdrawing (M5, M6) === + + +Both M5 and M6 are regular Bitcoin txns. They are identified by meeting an important criteria: they select a one of the Critical TxID-index Pairs (a "CTIP") as one of their inputs. Deposits ("M5") are distinguished from withdrawals ("M6") by simply checking to see if money is "going in", or "out". In other words, we compare the BTC value of the original CTIP to that of new CTIP. If original <= new it is a deposit, if original > new then it is a withdrawal. + +The code that identifies sidechain withdrawal / deposit txns (by calculating how much value is being put into or taken out of a sidechain) can be seen here: https://github.com/drivechain-project/bitcoin/blob/mainchainBMM/src/validation.cpp#L351-L386 + +Such txns are forced (by consensus) to obey two additional criteria: + +# They must contain an output paying "to" the Critical Address [probably in TxOut0]. +# They must be accompanied by an update to this sidechain's Critical TxID-index Pair (CTIP). The new CTIP must be "this" txn itself. + +These criteria are enforced [https://github.com/drivechain-project/bitcoin/blob/mainchainBMM/src/validation.cpp#L440-L473 here] by checking that a deposit is paying back to the sidechain more than it is taking out, and completely rejecting any withdrawal from the mempool. And [https://github.com/drivechain-project/bitcoin/blob/mainchainBMM/src/validation.cpp#L1747-L1757 here] we allow for a withdrawal only once it has attained sufficient work score (ACKs). + +The purpose of this is to have all of the escrow's money (ie all of the sidechain's money) in one TxID, so that depositors immediately undo any UTXO bloat they may cause. This simplifies the withdrawal process, as there is no need to worry about cleaning up "dust deposits" (...and such cleaning can often result in headaches, for example where a withdrawal-txn is larger than 1MB in size, or else may only withdraw an arbitrarily limited amount of BTC). Notice that, unless we assume that an account will last forever, all utxos which are deposited must eventually be withdrawn by someone. Therefore, the relevant design criterion is not "efficiency" (total network cost) but rather "who should pay" (allocation of costs). + +==== M5. "Make a Deposit" -- a transfer of BTC from-main-to-side ==== + +As far as mainchain consensus is concerned, there are no additional requirements. + +However, in practice there *are* additional mainchain requirements...specified by the escrow account, (ie specified by the "sidechain" or "virtual chain"). These requirements are not part of mainchain consensus and are allowed to be anything. In other words, the sidechain is free to invent any way to credit depositor's money -- M5 is fully customizable. + +One method, is for mainchain depositors to append a zero-value OP Return to a Deposit txn, so that the sidechain knows how to credit funds. Mainchain users must upgrade their wallet software, of course, (on an individual basis) in order to become aware of and take advantage of new deposit-methods. + +===== Inconvenient Race Condition ===== + +The requirement that each hashrate escrow be linked to a single TxID does create an interesting inconvenience for depositors. If a user is slow to sign a txn after constructing it (perhaps because the user employs an air-gapped computer, etc), then the signed txn may no longer be valid. This is because the input it selects, may no longer be the Critical TxID (as "the" Critical TxID changes with each deposit). **Only one user can deposit at a time** (although many can deposit per block). As a result, the transaction must fail, and the user would need to be prompted to remake and resign the txn. If this is problem is too frustrating, users can always make main-to-side transfers using atomic cross chain swaps (or, the LN, if they already have a channel open on both chains). + +Fortunately, it is already a part of mainchain consensus that no two txns can spend the same TxID. The only new issue here is the confusion it might create for the user (hence the need for error messages and alternative deposit-methods). + + +==== M6. "Execute Withdrawal" -- a transfer of BTC from-side-to-main ==== + +We come, finally, to the critical matter: where users can take their money *out* of the escrow account, and return it to the "regular" UTXO set. As previously mentioned, this txn is one which (a) spends from a CTIP and (b) reduces the quantity of BTC in an account's CTIP. Most of the work has already been done by D1, M3, M4, and D2. Furthermore, existing Bitcoin tx-rules prevent the sidechain from ever withdrawing more money than has been placed into it. + +From there, we merely introduce two final concepts: + +# In each block, an entry in D2 is considered an "approved candidate" if the "ACKs" value is above 13140. +# A "blinded TxID" is way of hashing the txn, in which we first overwrite some parts of the txn with zeros. Specifically, the first 36 bytes of "TxIn0" (the first input, including TxOutHash and TxOutIndex), as well as the first 8 bytes of "TxOut0" (the first output). + +Blinding is necessary because we allow each sidechain only one UTXO at a time. + +of our restriction of the account to a single UTXO-member. Because of this, during the ACKing process the withdrawal-txn (which is currently being ACKed) may change in two ways: the CTIP (which changes with each deposit), and the total quantity of BTC stored in the account (which arbitrarily increases with each new deposit). In other words, a withdrawal-attempt is created via M3, but this takes place many blocks before the withdrawal is actually included via M6. During this time, a single new deposit to the account would change its CTIP and its value. So, what do we ACK? Well, we ACK a "blinded" version of the withdrawal. This blinded version is stable because the dynamic parts are always overwritten with zeros. + +While we ACK a blinded WT^, what is actually included in the blockchain ("M6") is an unblinded WT^. Since each blinded WT^ could correspond to many different unblinded WT^s, we need to impose further restrictions on those unblinded WT^s that are finally included. First, we will force the final unblinded WT^ to spend the entire sidechain balance (by forcing sum(input_values) to equal sum(output_values)). To avoid withdrawing the entire sidechain balance with every withdrawal, we will, secondly, force the unblinded WT^ to create a new output which is itself a deposit to the sidechain it withdrew from (which nodes can check using D1's CTIP field). Unfortunately, these requirements eliminate the possibility of including a transaction fee, as traditionally calculated. So, finally, to compensate for *that*, txn fees are encoded explicitly as a withdrawal to OP_TRUE (which the main:block's miner can immediately claim). + +With all of this in place, the only requirements for inclusion in a block are these: + +# "Be ACKed" -- The "blinded TxID" of this txn must be member of the "approved candidate" set in the D2 of this block. +# "Return Change to Account" -- TxOut0 must pay to the "critical account" (see D1) that corresponds to the CTIP that was selected as a TxIn. +# "Return *all* Change to Account" -- Sum of inputs must equal the sum of outputs. No traditional tx fee is possible. + +Finally, don't forget that M6 inherits the requirement (common to both M5 and M6) that the CTIP be selected as an input, and that the CTIP then be updated. In this case, we know that the critical index will be zero, so the new CTIP will be ("this TxID" (NOT blinded), 0). The TxID is NOT blinded because blinding is only for accumulating ACKs. + +As a result of these requirements, every single withdrawal-attempt will fail, unless an entry has been added to D2 and "ACKed" a sufficient number of times. + + + +==Backward compatibility== + + +As a soft fork, older software will continue to operate without modification. Non-upgraded nodes will see a number of phenomena that they don't understand -- coinbase txns with non-txn data, value accumulating in anyone-can-spend UTXOs for months at a time, and then random amounts leaving the UTXO in single, infrequent bursts. However, this phenomena doesn't affect them or the validity of the money that they receive. + +( As a nice bonus, note that the sidechains themselves inherit a resistance to hard forks. The only way to guarantee that the WT^s reported by different clients will continue to match identically, is to upgrade sidechains via soft forks of themselves. ) + + +==Deployment== + + +This BIP will be deployed by "version bits" BIP9 with the name "hrescrow" and using bit 4. + +
+// Deployment of Drivechains (BIPX, BIPY)
+consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].bit = 4;
+consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].nStartTime = 1515974401; // January 15th, 2018.
+consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].nTimeout = 1547510401; // January 15th, 2019.
+
+ +==Reference Implementation== + + +See: https://github.com/drivechain-project/bitcoin/tree/mainchainBMM + +Also, for interest, see an example sidechain here: https://github.com/drivechain-project/bitcoin/tree/sidechainBMM + + +==References== + +See http://www.drivechain.info/literature/index.html + + +==Credits== + +Thanks to everyone who contributed to the discussion, especially: ZmnSCPxj, Adam Back, Peter Todd, Dan Anderson, Sergio Demian Lerner, Chris Stewart, Matt Corallo, Sjors Provoost, Tier Nolan, Erik Aronesty, Jason Dreyzehner, Joe Miyamoto, Ben Goldhaber. + + +==Copyright== + +This BIP is licensed under the BSD 2-clause license. diff --git a/bip-hashrate-escrows/appendix-1.txt b/bip-hashrate-escrows/appendix-1.txt new file mode 100644 index 0000000..921be29 --- /dev/null +++ b/bip-hashrate-escrows/appendix-1.txt @@ -0,0 +1,20 @@ + +==== Two Withdrawals at Once ==== + +Currently, the documentation and code describe a situation where only one withdrawal can proceed at a time. + +In other words, as WT^s are proposed, only one can make progress toward the finish line. As a result, a given side-to-main transfer will always take between 3 and 6 months. + +However, it might be best to use a maximum of two transfers at once. + +If there were more simulataneous withdrawals, the worst-case transfer duration would improve. + +See image: bip-hashrate-escrows/two-groups.png?raw=true + +The worst-case withdrawal time obeys f(n)=3+(3/n) months, where n is the number of simultaneous withdrawals. + +N=2 is the most desirable choice for several reasons. First, it delievers the greatest marginal benefit (of 1.5 months). Later choices only deliver 0.5 and 0.25 marginal months. + +Second, n=2 can be implemented in a clever way: by allowing a withdrawal to freely advance, if and only if has an ACK-score of 6575 or greater, and if it also has the largest ACK score. In other words, the withdrawal that is furthest along can advance (or retreat) for free, if it has already made it at least halfway to the finish line. With this change, our new M4, is either an "abstain" for the sidechain (in which case nothing happens to any ACK scores), or else it will be in one of two cases: old_M4 + "the largest advances", or new_M4 + "the largest retreats". As a result the number of M4 possibilities (of which the next section is concerned) only increases by a factor of two (instead of exponentially). + +It is possible to troll this rule, by getting two (or even three) withdrawals to have 6575+ ACK scores, and then getting them to *tie* for first place. So, if there are any ties, the ability to "bonus move" would be disabled until all ties are broken. \ No newline at end of file -- cgit v1.2.3 From 3201b2311951652020e1d948362c24fcc06d6708 Mon Sep 17 00:00:00 2001 From: Paul Sztorc Date: Mon, 23 Apr 2018 19:15:13 -0400 Subject: typo --- bip-hashrate-escrows.mediawiki | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bip-hashrate-escrows.mediawiki b/bip-hashrate-escrows.mediawiki index 0a61b47..ae31e29 100644 --- a/bip-hashrate-escrows.mediawiki +++ b/bip-hashrate-escrows.mediawiki @@ -309,7 +309,7 @@ M4 is exceptional (in comparison to the other M's) in a few ways. First, its con In fact, M4 can also be *nothing*. In other words, it may be optional. This is precisely because, from one block to the next, we have constrained D2 such that it is only allowed to change in a few ways. Therefore, the exhaustive set of "candidate D2s" can be precomputed by full nodes in advance. -The following sections assume a maximum of one sucessful withdrawal per sidechain at a time (see [bip-hashrate-escrows/appendix-1.txt appendix 1]). +The following sections assume a maximum of one sucessful withdrawal per sidechain at a time (see [/bip-hashrate-escrows/appendix-1.txt appendix 1]). ==== How Hard is it to Guess M4? ==== -- cgit v1.2.3 From 99e57b086aa9b2bbba485386992818d4972f99d2 Mon Sep 17 00:00:00 2001 From: Paul Sztorc Date: Thu, 4 Apr 2019 16:30:43 -0700 Subject: compress We were able to dramatically shorten the BIP, by deleting superfluous explanations/justifications. Instead it just focuses on what the messages are. --- bip-hashrate-escrows.mediawiki | 335 +++++++++++------------------------------ 1 file changed, 88 insertions(+), 247 deletions(-) diff --git a/bip-hashrate-escrows.mediawiki b/bip-hashrate-escrows.mediawiki index ae31e29..650b17e 100644 --- a/bip-hashrate-escrows.mediawiki +++ b/bip-hashrate-escrows.mediawiki @@ -17,11 +17,11 @@ ==Abstract== -A "Hashrate Escrow" is a clearer term for the concept of "locked to an SPV Proof", which is itself a restatement of the phrase "within a sidechain" as described in [https://blockstream.com/sidechains.pdf a famous Oct 2014 paper] written partially by some Blockstream co-founders. +A "Hashrate Escrow" is a clearer term for the concept of "locked to an SPV Proof", which is itself a restatement of the phrase "within a sidechain" as described in [https://blockstream.com/sidechains.pdf the 2014 Blockstream whitepaper]. -A Hashrate Escrow resembles a 2-of-3 multisig escrow, where the 3rd party (who will arbitrate any disputes) is a decentralized group of people: the dynamic-membership set of Bitcoin Miners. However, the 3rd party does not sign escrow-withdrawal transactions with a private key. Instead, these are "signed" by directing hashpower over them for a period of time. +A Hashrate Escrow resembles a 2-of-3 multisig escrow, where the 3rd party (who will arbitrate any disputes) is a decentralized group of people: the dynamic-membership set of Bitcoin Miners. However, the 3rd party does not sign escrow-withdrawal transactions with a private key. Instead, these are "signed" by the accumulation of hashpower over time. -This project has [http://www.drivechain.info/ a website] which includes [http://www.drivechain.info/faq/index.html a FAQ]. +This project has [http://www.drivechain.info/ a website] which includes [http://www.drivechain.info/faq/index.html an FAQ]. ==Motivation== @@ -30,10 +30,10 @@ In practice these escrows are likely to be "asymmetric sidechains" of Bitcoin (s Sidechains have many potential benefits, including: -1. Protect Bitcoin from competition from altcoins and spinoffs. Safely allow competing implementations (of *sidechains*). -2. Protect Bitcoin from hard fork campaigns. (Such campaigns represent an existential threat to Bitcoin, as well as an avenue for developer corruption.) -3. Help with review, by making it much easier for reviewers to ignore bad ideas. -4. Provide an avenue for good-but-confusing ideas to prove their value safely. +# Protect Bitcoin from competition from altcoins and spinoffs. +# Protect Bitcoin from hard fork campaigns. (Such campaigns represent an existential threat to Bitcoin, as well as an avenue for developer corruption.) +# Help with review, by making it much easier for reviewers to ignore bad ideas. +# Provide an avenue for good-but-confusing ideas to prove their value safely. @@ -60,239 +60,144 @@ Please note that these structures (D1 and D2) will not literally exist anywhere * M6. "Execute Withdrawal" -- a transfer of BTC from-side-to-main -==== On the Resource Requirements of New Databases ==== -The "new" databases are simply reinterpretations of data that are already contained elsewhere in the blockchain. Specifically, M1 M2 and M3 are all located in the block's coinbase txn, and M5 and M6 might be found in any regular txn. M4 is a special case and does not actually need to be included anywhere, so it is not. If you like, you can imagine that the M4s reside in an optional extension block. -In other words, we just rearrange what is already there. Because of this, even though "new databases" are created and stored in memory, the existing bandwidth and storage limits are respected (although, see "M4" below). - - - - -=== Adding Sidechains and Tracking Them (D1, M1, M2) === +=== Adding Sidechains (D1, M1, M2) === ==== D1 -- "Escrow_DB" ==== The table below enumerates the new database fields, their size in bytes, and their purpose. In general, an escrow designer (for example, a sidechain-designer), is free to choose any value for these. -Note: Fields 6 through 9 have been intentionally removed. Previously, this section allowed miners to set and commit to voting/waiting periods. However, I have since standardized the periods: withdrawals expire after 6 months (26298 blocks), and they succeed if they ever achieve an ACK score of 13140 or higher. I have removed the waiting period, because anyone who adopts a policy of ignoring all withdrawals with fewer than 400 ACKs will automatically gain all of the benefits of the waiting period. The justification for this change is that it strongly implies that an attack on any one sidechain is an attack on all of them (in a sense, this change makes the "victimhood" of each sidechain "fungible"). + {| class="wikitable" ! Field No. ! Label -! Bytes ! Type ! Description / Purpose |- | 1 | Escrow Number -| 1 -| uInt +| uint8_t | A number assigned to the entire escrow. Used to make it easy to refer to each escrow. |- | 2 -| Active* -| 2 -| uInt -| This counter starts at zero when a new entry is added, and increments as a new entry is supported by "M2". The entry either reaches the max value of 4032 (at which point the counter freezes), or else the entry is removed. This enforces the requirement "a soft fork is required to add a new escrow". +| Sidechain Deposit Script Hex +| string +| The script that will be deposited to, and update the CTIP of the sidechain. |- | 3 -| Escrow Name/Description -| 120 +| Sidechain Private Key | string -| A human-readable name and description of the sidechain. More than enough space to hold a 32 byte hash. Helps prevent destructive interference among sidechains (see below). +| The private key of the sidechain deposit script. |- | 4 -| Critical Private Key -| 32 -| hex -| This is the Double-SHA256 of the binary release of the reference software. It ties the sidechain to a specific software release, and doubles as a way of signing withdrawals (see below). +| Escrow Name +| string +| A human-readable name of the sidechain. |- | 5 -| Critical Address* -| 32 +| Escrow Description | string -| This is derived by taking f(x) = ImportPrivateKey(HexToBase58Check(x)) of field #3. It is intentionally in the human-readable format {{although it could be 25 bytes of binary}}. +| A human-readable name description of the sidechain. More than enough space to hold a 32 byte hash. +|- +| 6 +| Hash ID 1 +| uint256 +| A field of 32 bytes, which could be any bytes such as a sha256 hash. +|- +| 7 +| Hash ID 2 +| uint256 +| A field of 32 bytes, which could be any bytes such as a sha256 hash. |- -| 10 -| "CTIP" -- Part 1 "TxID"* -| 32 -| hex +| 8 +| "CTIP" -- Part 1 "TxID" +| uint256 | The CTIP, or "Critical (TxID, Index) Pair" is a variable for keeping track of where the escrow's money is (ie, which member of the UTXO set). |- -| 11 -| "CTIP" -- Part 2 "Index"* -| 4 -| hex -| Of the CTIP, this is second element of the pair: the Index. See #10 above. +| 9 +| "CTIP" -- Part 2 "Index" +| int32_t +| Of the CTIP, this is second element of the pair: the Index. See #9 above. +|- |} -\* Denotes a "convenience field", the entry for this field is derived from other fields, or from the blockchain-state itself. The escrow-creator does not need to provide these values in M1 (or anywhere). - -Escrow_DB requires 223 bytes [1+120+32+32+2+32+4] for each escrow in the main blockchain. Of these, 70 bytes [2+32+32+4] are merely for convenience. Therefore, a sidechain is defined (see "M1") by 153 bytes of information. - -====Notes on D1==== - -# D1 will always exist. -# D1 will always have a unique sort (first by "Escrow Number" (field #1), and second by "Active" (field #2)). There is only ever one (escrow number, Active) pair. -# D1 is updated according to M1 and M2 (below). -# If a new entry is added to D1 with an "Escrow Number" that is already in use, then this entry will either eventually be removed (because it was not supported with an M2), or it will eventually overwrite the old entry (if it *was* supported via M2). - - -====Notes on D1==== - -=====Obligations Placed on Miners===== - -Miners have always upgraded their software according to criteria that are known only to them (in other words, "whenever they want"). - -However, this soft fork imposes two new criteria upon them. First: miners should only upgrade their software, if any modification to the portfolio of sidechains [that are added/removed in the upgrade] can be expected to increase miner wealth. Trivially, this implies that miners should make sure that the upgrade doesn't overwrite (and destroy) an existing sidechain that they like! But, more seriously, it implies that miners should take an interest in what the sidechain is doing to the mainchain and other sidechains (see below). +D1 is updated via M1 and M2. -===== Destructive Sidechain Interference ===== - -People frequently emphasize that miners should have "as little control" as possible. It is a very safe claim to make, and a very easy sentence to write. Much harder is to determine exactly what this minimum value is, and how to achieve it. Harder still is to untie the knot of who is actually controlling what, in a decentralized, interacting system. - -Certainly, miners can not have "zero control" -- for that is the same as to just remove them from the system altogether. Some rules are enforced "on miners by nodes" (such as the infamous blocksize limit); other rules are enforced by nodes but are narrowly-controlled by miners (such as the proof-of-work itself, or the block's timestamp). Thirdly, some rules are enforced by both against each other (such as the rule against including invalid txns or double-spent txns), for mutual benefit. - -Some pause should be given, after one considers that the sidechain design goal is literally a piece of software that can do *anything*. Anything includes a great many things, many of which I demonstrate to be undesirable. Bitcoin itself does not allow "anything" -- it allows any person to transact, but, in contrast, it does not permit any person to double-spend. This is because "allowing anyone to do anything" is not viable in a world that contains undesirable interactions (what a libertarian might call "aggression") -- in the case of money, these are theft and counterfeiting. - -I have produced a comprehensive quantity of written material [1], presentations [2], etc [3] on exactly what the level of miner-control should be, and why. Specifically, I claim that **miners should be aware of the purpose of the sidechain, and they should reject sidechains which have an unclear purpose or which have a purpose that will lead to decrease in miner-wealth** (where wealth measured explicitly as: the estimated present value of the purchasing power of the blockchain's coinbase txns). I claim that this criterion is necessary because, just Original Bitcoin filters unwanted interactions among different BTC txns, so too much "Sidechain Bitcoin" filter out unwanted interactions among sidechain. - -* [1] http://www.truthcoin.info/blog/wise-contracts/ -* [2] https://www.youtube.com/watch?v=xGu0o8HH10U&index=1&list=PLw8-6ARlyVciMH79ZyLOpImsMug3LgNc4 -* [3] http://www.drivechain.info/literature/index.html - -Call it a "sidechain non-aggression principle", if you want. - -To the best of my knowledge, everyone who *has* reviewed this information as found the arguments to be acceptable. It has, also, changed a few minds (from "unacceptable" to "acceptable"). - - -===== ISSUE: "Signing" BTC Txns ===== - -Currently, we use a process which may be suboptimal. It is that we *literally sign* a txn with a globally and publicly known private key. But this is for convenience purposes -- to easily detect the sidechain's balance. The signature that is produced is not doing anything. This is probably an area of improvement. - - -( The following messages were modeled on SegWit -- https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki#commitment-structure ) +( The following messages were modeled on SegWit -- see [https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki#commitment-structure here] and [https://github.com/DriveNetTESTDRIVE/DriveNet/blob/564516653c1d876429382971a011f5f6119f7eb4/src/validation.cpp#L3348-L3375 here]. ) ==== M1 -- "Propose New Sidechain" ==== 1-byte - OP_RETURN (0x6a) - 1-byte - Push the following 157 bytes (0x9d) - 4-byte - Commitment header (0x53707243) - 153-byte - the critical bytes mentioned above (fields #1, #3, and #4, to populate a new D1 entry) - - -==== New Block Validation Rules ==== - -# If the network detects a properly-formatted M1, it must add an entry to D1, into the very next block, with the following initial values: -## Field #5 will be calculated as per [https://en.bitcoin.it/wiki/Technical_background_of_version_1_Bitcoin_addresses version 1 Bitcoin addresses], but with a prefix of "4" instead of "1". -## Field #9 will be derived from #7 and #8 using math. -## The initial values of Fields #10, #11, and #12 are set to zero. -# Only one M1 (of any kind) can be added into a block at a time. - -==== Notes on M1 ==== - -The escrow will "reuse" the same address over and over. But notice that there is no privacy loss due to address reuse, because the address in question does not "belong" to any specific individual. Furthermore, the transactions in question could already be linked by just watching the Hashrate Escrow for deposits ...this, in fact, is exactly what a sidechain must do in order to make main-to-side transfers. + 4-byte - Commitment header (0xD5E0C4AF) + N-byte - The serialization of the sidechain. ==== M2 -- "ACK Sidechain Proposal" ==== 1-byte - OP_RETURN (0x6a) - 1-byte - Push the following 29 bytes (0x1D) - 4-byte - Commitment header (0x53616343) - 25-byte - Commitment hash: RIPMD-160 of Sha256 of a given M1 (above) + 4-byte - Commitment header (0xD6E1C5BF) + 32-byte - Commitment hash: sha256D hash of sidechain's serialization ==== New Block Validation Rules ==== -1. An entry will be removed from D1, unless that block contains an M2 supporting it (this will increment field #2), or the entry has accumulated 4032 M2s. Therefore, each entry needs to have 4032 *consecutive* M2s supporting it, else it will be removed from D1 and must start over. This requires all miners to agree to "soft fork" the sidechain into existence. This makes it easier to curate the list of active sidechains, and prevents destructive interference (see above) among contradictory sidechains. -2. Once the entry has 4032 M2s, then a miner must make a 0 btc payment to the Critical Address, thus activating the chain. This payment becomes the first "critical (TxID, index) pair" (or "CTIP"). -3. It is possible to "overwrite" a D1 slot -- this destroys the sidechain that is there, and replaces it with a new one. This is defined as requiring 6 months (26298 blocks) of M2s, instead of 4 weeks (4032). This possibility does not change the security assumptions, because we already assume that users have access to "magically true" bandwidth at a rate of 1 bit per ACK-cycle. (This is why the voting cycles should be large, 1 month Waiting, 2+ months Voting). +# Escrows are added in a procedure that resembles BIP 9 soft fork activation: the network must see a properly-formatted M1, followed by "acknowledgement" of the sidechain in 95% of the following 2016 blocks. +# It is possible to "overwrite" an escrow. This requires 6 months (26298 blocks) of M2s, instead of 2 weeks (XXXX). This possibility does not change the security assumptions (because we already assume that users perform extra-protocolic validation at a rate of 1 bit per 26298 blocks). -=== Adding Withdrawal-Attempts and Tracking Their Success (D2, M3, M4) === -==== D2 -- "Withdrawal_DB" ==== +=== Withdrawing from Escrows (D2, M3, M4) === -The table below enumerates the database fields, their size (in bytes), type and purpose. +==== D2 -- "Withdrawal_DB" ==== +D2 changes deterministically with respect to M3, M4, M5, and M6. {| class="wikitable" ! Field No. ! Label -! Bytes ! Type ! Description / Purpose |- | 1 | Escrow Number -| 1 -| uInt +| uint8_t | Links the withdrawal-request to a specific escrow. |- | 2 -| WT^ -| 32 -| hex +| WT^ Hash +| uint256 | This is a "blinded transaction id" (ie, the double-Sha256 of a txn that has had two fields zeroed out, see M6) of a withdrawal-attempt. |- | 3 -| ACKs* -| 2 -| uInt -| The current total number of "votes", this starts at 0 and remains there throughout the waiting period. +| ACKs (Work Score) +| uint16_t +| The current total number of ACKs (PoW) |- | 4 -| Age* -| 3 -| uInt -| Total duration of time, in blocks, that this WT^ has been inside of D2. -|- -| 5 -| Waiting Period* -| 2 -| uInt -| Total duration of time, in blocks, that this entry must sit idle, before it can begin to accumulate ACKs/NACKs. Pulled from D1's field #6. -|- -| 6 -| Max Age* -| 3 -| uInt -| Determined by summing (D1's field #6) and (D1's field #7). -|- -| 7 -| Threshold* -| 2 -| uInt -| Total ACKs needed, this is pulled from D1's field #9. -|- -| 8 -| Approved* -| 1 -| boolean -| True while ACKs > Threshold, False otherwise. +| Blocks Remaining (Age) +| uint16_t +| The number of blocks which this WT^ has remaining to accumulate ACKs |} -\* Denotes a "convenience field" (see above). - -Withdrawal_DB requires 46 bytes [1+32+2+3+2+3+2+1] per entry. Of these, 13 bytes ([2+3+2+3+2+1], all fields except #1 and #2) can be generated locally, leaving 33 critical bytes [1+32]. ==== New Block Validation Rules for D2 ==== -# In each block, a hash commitment to D2 must always exist (even if D2 is blank). -# D2 must always be sorted first by field #1 (Escrow Number) and second by field #4 (Age). This imposes a unique sort. -# From one block to the next, every entry's "Age" field must increase by exactly 1. -# From one block to the next, entries are only removed from D2 (in the very next block) if: -## "Age" = "MaxAge". -## If the block contains a txn who's blinded txID matches WT^. {{ This might be unnecessary, and a lot of work. }} -# In addition, there are special rules for the allowed values in the "ACKs" field (field #3). See M4 below. +# A hash commitment to D2 exists in each block (even if D2 is blank). +# Withdrawals in D2 are sorted first by field #1 (Escrow Number) and second by field #4 (Age). This imposes a unique sort. +# From one block to the next, "Age" fields must increase by exactly 1. +# Withdrawals are stored in D2 until they fail ("Age" = "MaxAge"), or they succeed (the blockchain contains a txn who's blinded txID matches "WT^"). + +In addition, there are special rules for the "ACKs" field (see M4 below). ==== M3 -- "Propose Withdrawal" ==== 1-byte - OP_RETURN (0x6a) - 1-byte - Push the following 37 bytes (0x25) + 1-byte - Push the following 36 bytes (0x24) 4-byte - Commitment header (0xD45AA943) - 33-byte - the critical bytes mentioned above (fields #1 and #2, to populate a new D2 entry) + 32-byte - The WT^ hash to populate a new D2 entry ==== New Block Validation Rules for M3 ==== @@ -303,126 +208,62 @@ Withdrawal_DB requires 46 bytes [1+32+2+3+2+3+2+1] per entry. Of these, 13 bytes ==== M4 -- "ACK Withdrawal" ==== -==== Very Little Info, Probably Calculable in Advance ==== - -M4 is exceptional (in comparison to the other M's) in a few ways. First, its content is not stored anywhere, only the *hash* of its *effect* is stored (in a leaf of a merkle tree who's root is inserted into a mainchain coinbase). M4 alters the contents of D2 -- the *contents* of D2 are consensus critical, but M4 (the process by which nodes reach a new valid D2) can be anything. - -In fact, M4 can also be *nothing*. In other words, it may be optional. This is precisely because, from one block to the next, we have constrained D2 such that it is only allowed to change in a few ways. Therefore, the exhaustive set of "candidate D2s" can be precomputed by full nodes in advance. - -The following sections assume a maximum of one sucessful withdrawal per sidechain at a time (see [/bip-hashrate-escrows/appendix-1.txt appendix 1]). - -==== How Hard is it to Guess M4? ==== - -If there are n Escrows and m Withdrawals-per-escrow1, then there are (m+2)^n total candidates for the next D2. This is because, [per block per escrow], one of three things can happen: (1) one of the m withdrawal-candidates can be "ACK"ed (or "upvoted" or "promoted"), which automatically downvotes the others; or (2) all withdrawal-candidates can be downvoted, or finally (3) the miners can abstain from voting on the escrow's withdrawals altogether, leaving the tallies the same. - -First, for nodes which validate all sidechains (assuming these escrows are sidechains), this simplifies to 2^n -- these nodes only have to choose between the single honest choice (on one hand) or an abstention (on the other). Second, even for nodes that don't validate any sidechains, the number of candidates might be reduced from m^n to 3^n, by making a simplifying assumption: whichever withdrawal was most recently added/upvoted, is likely to be the one which is upvoted next. - -Of course, that is still O(k^n) for n sidechains, which isn't great2. If the "D2 update" cannot be guessed, it must be transmitted in some way. - -==== Giving Up and Getting M4 the Old Fashioned Way ==== - -Two examples for transmitting it are below: +M4 is a way of describing changes to the "ACKs" column of D2. -"Short Form" (Assumes there are no more than 254 active withdrawal-attempts per account) +From one block to the next, "ACKs" can only change as follows: - 4-byte - Message identifier (0x????????) - 1-byte - Version of this message - N-byte - N is the total number of active accounts ("sidechains"), each byte specifies the position of the single WT that was "upvoted". A value of 0 indicates "downvote everything", a value of 255 indicates abstention. +* The ACK-counter of any withdrawal can only change by (-1,0,+1). +* Within a sidechain-group, upvoting one withdrawal ("+1") requires you to downvote all other withdrawals in that group. However, the minimum ACK-value is zero (and, therefore, downvotes cannot reduce it below zero). +* While only one withdrawal can be upvoted at once, they can all be unchangd at once ("abstain") and they can all be downvoted at once ("alarm"). -"Long Form" (Makes no assumptions about anything) +One option for explict transmission of M4 is: 4-byte - Message identifier (0x????????) 1-byte - Version of this message 1-byte - Length (in bytes) of this message; total number of withdrawal attempts; y = ceiling( sum_i(m_i +2)/8 ). Nodes should already know what length to expect, because they know the sequence of M3s and therefore the vector of WT^s. - Y-byte - stream of bits (not bytes), with a 1 indicating the position of the chosen action [downvote all, abstain, upvote1, upvote2, ...] - - -If the message is very very large, then nodes may not want to broadcast it. This opens up an "exhaustion attack"2, in which many miners create bad WT^s, vote on these randomly, and then refuse to broadcast their votes. Fortunately, even for a worst-case scenario of 200 sidechains and 1,000 withdrawal-attempts per sidechain, honest nodes can communicate a long form M4 with each other by using just 25,056 bytes per block [4+1+1+(200\*(1000+1+1)/8)]. - -Today's pre-drivechain miners can already carry out a similar attack, by creating and including txns and then not broadcasting that part of the block to anyone. This is often characterized as a [https://petertodd.org/2016/block-publication-incentives-for-miners "block publication incentive"], because in that case the prospect of exhaustively computing all possible transactions (to uncover the missing ones) is completely out of the question. - -However, message M4 is different from a withheld-txn, because M4 operates outside of the block's mandated information-processing limits (ie, outside the infamous 1 MB nonwitness blocksize limit). So we should examine the conditions under which M4 grows and shrinks, to ensure that we are not smuggling in a tremendous burden on full nodes. - -Under adversarial conditions, to lengthen a long-form M4 by one bit per block, for C blocks, the attacker must pay 312 bits (39 bytes) one time (to embed a new M3 message). The value C is the length of the sidechain's voting period, which varies but which I expect to be approximately 8,064 (and which could theoretically be as high as 65,536). Thus the attacker can burden nodes disproportionately, if (s)he wishes. - -Fortunately, the attack in question has no motivation (as far as I can tell). If the miner's goal is to trick rivals into mining on top of invalid blocks, he can already do this much more effectively with the unpublished-txn method (above). If instead he is just trying to harass nodes, then nodes may freely "downgrade" to earlier versions of the protocol, and simply ignore all drivechain-related messages. It seems that the attack could best be used in order to: make a large D2, make D2 confusing, sneak in votes for evil WT^ lurking in D2. Thus, the attack disables the transparency of the drivechain system, to some extent. The cost of the attack is forgone transaction fees, due to block space wasted on useless M3s. - -In practice, n is already capped, and miners may impose [on each other] a "soft cap" on m for their mutual protection. Thus, n and m might never get above 10 and 30, respectfully. In this case, the [Short Form, this time] M4 can never require more than 15 bytes per block, no matter what the attacker tries. - -In practice, m should always be 1 or 2, else something fishy is going on; and m can only inch up by 1 unit per block. So the system as a whole is still quite transparent, in that users are warned appropriately and well in advance. Attackers must invest upfront and they face an uphill climb, in order to eventually make things more expensive for a few others; defenders can wait-and-see if the attack looks like it will ever amount to anything before lifting a finger. - - -===== New Block Validation Rules (for D2 and, by implication, M4) ===== + N-byte - stream of bits (not bytes), with a 1 indicating the position of the chosen action [downvote all, abstain, upvote1, upvote2, ...] -From one block to the next, D2 can only be edited in a few strict ways: - -* Entries can only be added/removed from D2 if they meet the criteria above (in M3, and implicitly M1 and M2). -* The ACK-counter of any individual entry can only change by (-1,0,+1) relative to its previous entry. -* Within a sidechain group, upvoting one withdrawal (ACK=ACK+1) requires you to downvote all other withdrawals in that group. However, the minimum ACK value is zero (and, therefore, downvotes cannot reduce it below zero). - -===== Footnotes for M4 ===== - -1 This represents the worst-case scenario is one where all the Withdrawals are spread evenly over each Sidechain. Under normal operations, there is no reason to expect the all sidechains will have the same number of withdrawals at any given time. In fact, under normal operations, the very *concept* of counting the withdrawals-per-sidechain should be a purposeless one, because there should only be *one* withdrawal at a time. Nonetheless we consider the worst case scenario here. - -2 Guessing becomes more computationally intensive in a highly adversarial situation where the "limited range" is intentionally expanded. In such a scenario, [a] there are many sidechains, and [b] miners voluntarily sacrifice their scarce block-space by creating a high number of (mutually-exclusive, and hence ultimately invalid) withdrawal attempts and putting these into coinbase transactions; and then agree to all [c] vote on these randomly (guaranteeing that all withdrawals fail, including any true withdrawals) and [d] successfully withhold their random voting strategies from nodes (even including spy-miner-nodes). Under this bizarre scenario, nodes may require computing resources which increase near-exponentially with the number of withdrawals, and it may take a long time for an ignorant node to exhaustively work out the underlying state of Withdrawal_DB. In this case, nodes may decide to temporarily stop validating such transactions (as if they had not yet upgraded to support this soft fork). +But sometimes M4 does not need to be transmitted at all! If there are n Escrows and m Withdrawals-per-escrow, then there are (m+2)^n total candidates for the next D2. So, when m and n are low, all of the possible D2s can be trivially computed in advance. +Miners can impose a "soft limit" on m, blocking new withdrawal-attempts until previous ones expire. For a worst-case scenario of n=200 and m=1,000, honest nodes can communicate M4 with ~25 KB per block [4+1+1+(200\*(1000+1+1)/8)]. === Depositing and Withdrawing (M5, M6) === +Both M5 and M6 are regular Bitcoin txns. They are identified by meeting an important criteria: they select a one of the Critical TxID-index Pairs (a "CTIP") as one of their inputs. -Both M5 and M6 are regular Bitcoin txns. They are identified by meeting an important criteria: they select a one of the Critical TxID-index Pairs (a "CTIP") as one of their inputs. Deposits ("M5") are distinguished from withdrawals ("M6") by simply checking to see if money is "going in", or "out". In other words, we compare the BTC value of the original CTIP to that of new CTIP. If original <= new it is a deposit, if original > new then it is a withdrawal. - -The code that identifies sidechain withdrawal / deposit txns (by calculating how much value is being put into or taken out of a sidechain) can be seen here: https://github.com/drivechain-project/bitcoin/blob/mainchainBMM/src/validation.cpp#L351-L386 - -Such txns are forced (by consensus) to obey two additional criteria: +Just as these txns must select a CTIP input, they must create a new CTIP output. D1 is then updated to match only the latest CTIP output. The purpose of this is to have all of the escrow's money (ie all of the sidechain's money) in one TxID, so that depositors immediately undo any UTXO bloat they may cause. -# They must contain an output paying "to" the Critical Address [probably in TxOut0]. -# They must be accompanied by an update to this sidechain's Critical TxID-index Pair (CTIP). The new CTIP must be "this" txn itself. +Deposits ("M5") are distinguished from withdrawals ("M6") by simply checking to see if money is "going in", or "out". -These criteria are enforced [https://github.com/drivechain-project/bitcoin/blob/mainchainBMM/src/validation.cpp#L440-L473 here] by checking that a deposit is paying back to the sidechain more than it is taking out, and completely rejecting any withdrawal from the mempool. And [https://github.com/drivechain-project/bitcoin/blob/mainchainBMM/src/validation.cpp#L1747-L1757 here] we allow for a withdrawal only once it has attained sufficient work score (ACKs). +https://github.com/DriveNetTESTDRIVE/DriveNet/blob/564516653c1d876429382971a011f5f6119f7eb4/src/validation.cpp#L647-L742 -The purpose of this is to have all of the escrow's money (ie all of the sidechain's money) in one TxID, so that depositors immediately undo any UTXO bloat they may cause. This simplifies the withdrawal process, as there is no need to worry about cleaning up "dust deposits" (...and such cleaning can often result in headaches, for example where a withdrawal-txn is larger than 1MB in size, or else may only withdraw an arbitrarily limited amount of BTC). Notice that, unless we assume that an account will last forever, all utxos which are deposited must eventually be withdrawn by someone. Therefore, the relevant design criterion is not "efficiency" (total network cost) but rather "who should pay" (allocation of costs). ==== M5. "Make a Deposit" -- a transfer of BTC from-main-to-side ==== -As far as mainchain consensus is concerned, there are no additional requirements. +As far as mainchain consensus is concerned, deposits to the escrow are always valid. -However, in practice there *are* additional mainchain requirements...specified by the escrow account, (ie specified by the "sidechain" or "virtual chain"). These requirements are not part of mainchain consensus and are allowed to be anything. In other words, the sidechain is free to invent any way to credit depositor's money -- M5 is fully customizable. +However, in practice there will be additional requirements. The escrow account (ie the "sidechain") needs to know how to credit depositors. One well-known method, is for mainchain depositors to append a zero-value OP Return to a Deposit txn, so that the sidechain knows how to credit funds. Mainchain users must upgrade their wallet software, of course, (on an individual basis) in order to become aware of and take advantage of new deposit-methods. -One method, is for mainchain depositors to append a zero-value OP Return to a Deposit txn, so that the sidechain knows how to credit funds. Mainchain users must upgrade their wallet software, of course, (on an individual basis) in order to become aware of and take advantage of new deposit-methods. - -===== Inconvenient Race Condition ===== - -The requirement that each hashrate escrow be linked to a single TxID does create an interesting inconvenience for depositors. If a user is slow to sign a txn after constructing it (perhaps because the user employs an air-gapped computer, etc), then the signed txn may no longer be valid. This is because the input it selects, may no longer be the Critical TxID (as "the" Critical TxID changes with each deposit). **Only one user can deposit at a time** (although many can deposit per block). As a result, the transaction must fail, and the user would need to be prompted to remake and resign the txn. If this is problem is too frustrating, users can always make main-to-side transfers using atomic cross chain swaps (or, the LN, if they already have a channel open on both chains). - -Fortunately, it is already a part of mainchain consensus that no two txns can spend the same TxID. The only new issue here is the confusion it might create for the user (hence the need for error messages and alternative deposit-methods). ==== M6. "Execute Withdrawal" -- a transfer of BTC from-side-to-main ==== We come, finally, to the critical matter: where users can take their money *out* of the escrow account, and return it to the "regular" UTXO set. As previously mentioned, this txn is one which (a) spends from a CTIP and (b) reduces the quantity of BTC in an account's CTIP. Most of the work has already been done by D1, M3, M4, and D2. Furthermore, existing Bitcoin tx-rules prevent the sidechain from ever withdrawing more money than has been placed into it. -From there, we merely introduce two final concepts: - -# In each block, an entry in D2 is considered an "approved candidate" if the "ACKs" value is above 13140. -# A "blinded TxID" is way of hashing the txn, in which we first overwrite some parts of the txn with zeros. Specifically, the first 36 bytes of "TxIn0" (the first input, including TxOutHash and TxOutIndex), as well as the first 8 bytes of "TxOut0" (the first output). - -Blinding is necessary because we allow each sidechain only one UTXO at a time. +In each block, a withdrawal in D2 is considered "approved" if its "ACKs" value meets the threshold (13,150). -of our restriction of the account to a single UTXO-member. Because of this, during the ACKing process the withdrawal-txn (which is currently being ACKed) may change in two ways: the CTIP (which changes with each deposit), and the total quantity of BTC stored in the account (which arbitrarily increases with each new deposit). In other words, a withdrawal-attempt is created via M3, but this takes place many blocks before the withdrawal is actually included via M6. During this time, a single new deposit to the account would change its CTIP and its value. So, what do we ACK? Well, we ACK a "blinded" version of the withdrawal. This blinded version is stable because the dynamic parts are always overwritten with zeros. +Approved withdrawals give the green light to their respective "WT^". A "WT^" is 32-bytes which aspire to represent the withdrawing transtion (the txn that actually withdraws funds from the escrow). The two cannot match exactly, beacuse "WT^" is defined at onset, and the withdrawing TxID depends on the its CTIP input (which is constantly changing). -While we ACK a blinded WT^, what is actually included in the blockchain ("M6") is an unblinded WT^. Since each blinded WT^ could correspond to many different unblinded WT^s, we need to impose further restrictions on those unblinded WT^s that are finally included. First, we will force the final unblinded WT^ to spend the entire sidechain balance (by forcing sum(input_values) to equal sum(output_values)). To avoid withdrawing the entire sidechain balance with every withdrawal, we will, secondly, force the unblinded WT^ to create a new output which is itself a deposit to the sidechain it withdrew from (which nodes can check using D1's CTIP field). Unfortunately, these requirements eliminate the possibility of including a transaction fee, as traditionally calculated. So, finally, to compensate for *that*, txn fees are encoded explicitly as a withdrawal to OP_TRUE (which the main:block's miner can immediately claim). +To solve this, we define a "blinded TxID" as a way of hashing a txn, in which some bytes are first overwritten with zeros. Specifically, these bytes are the first input and the first output. -With all of this in place, the only requirements for inclusion in a block are these: +So, withdrawals must meet the following three criteria: # "Be ACKed" -- The "blinded TxID" of this txn must be member of the "approved candidate" set in the D2 of this block. # "Return Change to Account" -- TxOut0 must pay to the "critical account" (see D1) that corresponds to the CTIP that was selected as a TxIn. # "Return *all* Change to Account" -- Sum of inputs must equal the sum of outputs. No traditional tx fee is possible. -Finally, don't forget that M6 inherits the requirement (common to both M5 and M6) that the CTIP be selected as an input, and that the CTIP then be updated. In this case, we know that the critical index will be zero, so the new CTIP will be ("this TxID" (NOT blinded), 0). The TxID is NOT blinded because blinding is only for accumulating ACKs. -As a result of these requirements, every single withdrawal-attempt will fail, unless an entry has been added to D2 and "ACKed" a sufficient number of times. @@ -442,14 +283,14 @@ This BIP will be deployed by "version bits" BIP9 with the name "hrescrow" and us
 // Deployment of Drivechains (BIPX, BIPY)
 consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].bit = 4;
-consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].nStartTime = 1515974401; // January 15th, 2018.
-consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].nTimeout = 1547510401; // January 15th, 2019.
+consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].nStartTime = 1579072881; // January 15th, 2020.
+consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].nTimeout = 1610695281; // January 15th, 2021.
 
==Reference Implementation== -See: https://github.com/drivechain-project/bitcoin/tree/mainchainBMM +See: https://github.com/DriveNetTESTDRIVE/DriveNet Also, for interest, see an example sidechain here: https://github.com/drivechain-project/bitcoin/tree/sidechainBMM -- cgit v1.2.3 From c6da99018d7cffb4f997ab58d471e2bdcdbdc3e5 Mon Sep 17 00:00:00 2001 From: Paul Sztorc Date: Thu, 4 Apr 2019 16:47:04 -0700 Subject: typos --- bip-hashrate-escrows.mediawiki | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/bip-hashrate-escrows.mediawiki b/bip-hashrate-escrows.mediawiki index 650b17e..7438025 100644 --- a/bip-hashrate-escrows.mediawiki +++ b/bip-hashrate-escrows.mediawiki @@ -188,7 +188,7 @@ D2 changes deterministically with respect to M3, M4, M5, and M6. # A hash commitment to D2 exists in each block (even if D2 is blank). # Withdrawals in D2 are sorted first by field #1 (Escrow Number) and second by field #4 (Age). This imposes a unique sort. # From one block to the next, "Age" fields must increase by exactly 1. -# Withdrawals are stored in D2 until they fail ("Age" = "MaxAge"), or they succeed (the blockchain contains a txn who's blinded txID matches "WT^"). +# Withdrawals are stored in D2 until they fail ("Age" = "MaxAge"), or they succeed (the blockchain contains a txn whose blinded txID matches "WT^"). In addition, there are special rules for the "ACKs" field (see M4 below). @@ -270,7 +270,7 @@ So, withdrawals must meet the following three criteria: ==Backward compatibility== -As a soft fork, older software will continue to operate without modification. Non-upgraded nodes will see a number of phenomena that they don't understand -- coinbase txns with non-txn data, value accumulating in anyone-can-spend UTXOs for months at a time, and then random amounts leaving the UTXO in single, infrequent bursts. However, this phenomena doesn't affect them or the validity of the money that they receive. +As a soft fork, older software will continue to operate without modification. Non-upgraded nodes will see a number of phenomena that they don't understand -- coinbase txns with non-txn data, value accumulating in anyone-can-spend UTXOs for months at a time, and then random amounts leaving the UTXO in single, infrequent bursts. However, these phenomena don't affect them, or the validity of the money that they receive. ( As a nice bonus, note that the sidechains themselves inherit a resistance to hard forks. The only way to guarantee that the WT^s reported by different clients will continue to match identically, is to upgrade sidechains via soft forks of themselves. ) -- cgit v1.2.3 From a9b0bc593a49b5f2433fdc6fa08a4964e4687887 Mon Sep 17 00:00:00 2001 From: Paul Sztorc Date: Fri, 5 Apr 2019 09:59:18 -0700 Subject: spellcheck --- bip-hashrate-escrows.mediawiki | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/bip-hashrate-escrows.mediawiki b/bip-hashrate-escrows.mediawiki index 7438025..26ed32e 100644 --- a/bip-hashrate-escrows.mediawiki +++ b/bip-hashrate-escrows.mediawiki @@ -1,5 +1,3 @@ - -
     BIP: ????
     Layer: Consensus (soft fork)
@@ -144,7 +142,7 @@ D1 is updated via M1 and M2.
 ==== New Block Validation Rules ====
 
 
-# Escrows are added in a procedure that resembles BIP 9 soft fork activation: the network must see a properly-formatted M1, followed by "acknowledgement" of the sidechain in 95% of the following 2016 blocks.
+# Escrows are added in a procedure that resembles BIP 9 soft fork activation: the network must see a properly-formatted M1, followed by "acknowledgment" of the sidechain in 95% of the following 2016 blocks.
 # It is possible to "overwrite" an escrow. This requires 6 months (26298 blocks) of M2s, instead of 2 weeks (XXXX). This possibility does not change the security assumptions (because we already assume that users perform extra-protocolic validation at a rate of 1 bit per 26298 blocks).
 
 
@@ -214,9 +212,9 @@ From one block to the next, "ACKs" can only change as follows:
 
 * The ACK-counter of any withdrawal can only change by (-1,0,+1).
 * Within a sidechain-group, upvoting one withdrawal ("+1") requires you to downvote all other withdrawals in that group. However, the minimum ACK-value is zero (and, therefore, downvotes cannot reduce it below zero).
-* While only one withdrawal can be upvoted at once, they can all be unchangd at once ("abstain") and they can all be downvoted at once ("alarm").
+* While only one withdrawal can be upvoted at once, they can all be unchanged at once ("abstain") and they can all be downvoted at once ("alarm").
 
-One option for explict transmission of M4 is:
+One option for explicit transmission of M4 is:
 
     4-byte - Message identifier (0x????????)
     1-byte - Version of this message
@@ -253,7 +251,7 @@ We come, finally, to the critical matter: where users can take their money *out*
 
 In each block, a withdrawal in D2 is considered "approved" if its "ACKs" value meets the threshold (13,150).
 
-Approved withdrawals give the green light to their respective "WT^". A "WT^" is 32-bytes which aspire to represent the withdrawing transtion (the txn that actually withdraws funds from the escrow). The two cannot match exactly, beacuse "WT^" is defined at onset, and the withdrawing TxID depends on the its CTIP input (which is constantly changing).
+Approved withdrawals give the green light to their respective "WT^". A "WT^" is 32-bytes which aspire to represent the withdrawing transaction (the txn that actually withdraws funds from the escrow). The two cannot match exactly, because "WT^" is defined at onset, and the withdrawing TxID depends on the its CTIP input (which is constantly changing).
 
 To solve this, we define a "blinded TxID" as a way of hashing a txn, in which some bytes are first overwritten with zeros. Specifically, these bytes are the first input and the first output.
 
-- 
cgit v1.2.3


From ecc00805c283193525552a7be716e11710ca4d3c Mon Sep 17 00:00:00 2001
From: Paul Sztorc 
Date: Fri, 5 Apr 2019 11:03:26 -0700
Subject: clarify + specific M4 example

---
 bip-hashrate-escrows/appendix-1.txt | 43 +++++++++++++++++++++++++++++--------
 1 file changed, 34 insertions(+), 9 deletions(-)

diff --git a/bip-hashrate-escrows/appendix-1.txt b/bip-hashrate-escrows/appendix-1.txt
index 921be29..736a6c4 100644
--- a/bip-hashrate-escrows/appendix-1.txt
+++ b/bip-hashrate-escrows/appendix-1.txt
@@ -1,20 +1,45 @@
 
 ==== Two Withdrawals at Once ====
 
-Currently, the documentation and code describe a situation where only one withdrawal can proceed at a time.
+Currently, the documentation and code describe a situation where only one withdrawal can proceed at a time. As a result, one "train" (carrying everyone's withdrawals) leaves the station every 3 months, and takes 3-6 months to reach its destination.
 
-In other words, as WT^s are proposed, only one can make progress toward the finish line. As a result, a given side-to-main transfer will always take between 3 and 6 months.
+Thus, if a withdrawing-user is very unlucky, and "just misses" the train, this user must wait double-long. First, (s)he must wait for the missed-train to reach its destination. Second, (s)he must board the new train, and wait for *it* to reach its destination. Each of these steps takes 3-6 months.
 
-However, it might be best to use a maximum of two transfers at once.
+So, even when withdrawals always go as quickly as possible (3 months each), the total time varies, from 3 months (0 months waiting + 3 months travel) to 6 months (3 months waiting + 3 months travel). The average is 4.5 months.
 
-If there were more simulataneous withdrawals, the worst-case transfer duration would improve.
+To improve this, we allow for slightly different behavior if the highest-ACK-withdrawal [1st] has an ACK score >= 6575; and [2nd] is not tied with any other withdrawal.
 
-See image: bip-hashrate-escrows/two-groups.png?raw=true
+Basically: a second train can leave, if the furthest train is 50+% of the way to its destination.
 
-The worst-case withdrawal time obeys f(n)=3+(3/n) months, where n is the number of simultaneous withdrawals.
+So, previously, for m trains, M4 could be any of the following:
 
-N=2 is the most desirable choice for several reasons. First, it delievers the greatest marginal benefit (of 1.5 months). Later choices only deliver 0.5 and 0.25 marginal months.
+    abstain
+    alarm (move all trains backwards)
+    move train #1 forward (and others backwards)
+    move train #2 forward (and others backwards)
+    ...
+    move train #3 forward (and others backwards)
 
-Second, n=2 can be implemented in a clever way: by allowing a withdrawal to freely advance, if and only if has an ACK-score of 6575 or greater, and if it also has the largest ACK score. In other words, the withdrawal that is furthest along can advance (or retreat) for free, if it has already made it at least halfway to the finish line. With this change, our new M4, is either an "abstain" for the sidechain (in which case nothing happens to any ACK scores), or else it will be in one of two cases: old_M4 + "the largest advances", or new_M4 + "the largest retreats". As a result the number of M4 possibilities (of which the next section is concerned) only increases by a factor of two (instead of exponentially).
+If our new special conditions apply, we now double the (m-1) elements, to accommodate a second train:
 
-It is possible to troll this rule, by getting two (or even three) withdrawals to have 6575+ ACK scores, and then getting them to *tie* for first place. So, if there are any ties, the ability to "bonus move" would be disabled until all ties are broken.
\ No newline at end of file
+    |abstain
+    |alarm (move all trains backwards)
+
+    |advance furthest train + advance train #1 (regress all others)
+    |advance furthest train + advance train #2 (regress all others)
+    |...
+    |advance furthest train + advance train #(m-1)  (regress all others)
+     
+    |regress furthest train + advance train #1 (regress all others)
+    |regress furthest train + advance train #2 (regress all others)
+    |...
+    |regress furthest train + advance train #(m-1) (regress all others)
+    
+
+It is theoretically possible (but in practice probably impossible) to troll this rule, by getting two (or even three) withdrawals to have >6575 ACK scores, and then getting these to *tie* for first place. Then they'd both be furthest. Hence the second condition prohibiting this new behavior, if the furthest trains have any ACK-score ties.
+
+This simple change, which has almost zero impact on the security assumptions, improves the monthly total wait times drastically:
+
+    Worst-case:   6 --> 4.5
+    Average:    4.5 --> 3.75
+    Std Dev:   ~.91 --> ~.45
-- 
cgit v1.2.3


From c78766c360218df504cd56a4bb44f75f598c118b Mon Sep 17 00:00:00 2001
From: Paul Sztorc 
Date: Fri, 26 Jul 2019 16:13:20 -0700
Subject: add number 300 and update README

---
 README.mediawiki                    |   7 +
 bip-0300.mediawiki                  | 308 ++++++++++++++++++++++++++++++++++++
 bip-0300/appendix-1.txt             |  45 ++++++
 bip-0300/images.txt                 |   1 +
 bip-0300/two-groups.png             | Bin 0 -> 39695 bytes
 bip-hashrate-escrows.mediawiki      | 308 ------------------------------------
 bip-hashrate-escrows/appendix-1.txt |  45 ------
 7 files changed, 361 insertions(+), 353 deletions(-)
 create mode 100644 bip-0300.mediawiki
 create mode 100644 bip-0300/appendix-1.txt
 create mode 100644 bip-0300/images.txt
 create mode 100644 bip-0300/two-groups.png
 delete mode 100644 bip-hashrate-escrows.mediawiki
 delete mode 100644 bip-hashrate-escrows/appendix-1.txt

diff --git a/README.mediawiki b/README.mediawiki
index 0f21815..b5c6f10 100644
--- a/README.mediawiki
+++ b/README.mediawiki
@@ -869,6 +869,13 @@ Those proposing changes should consider that ultimately consent may rest with th
 | Standard
 | Draft
 |-
+| [[bip-0300.mediawiki|300]]
+| Consensus (soft fork)
+| Hashrate Escrows (Consensus layer)
+| Paul Sztorc, CryptAxe
+| Standard
+| Draft
+|-
 | [[bip-0301.mediawiki|301]]
 | Consensus (soft fork)
 | Blind Merged Mining (Consensus layer)
diff --git a/bip-0300.mediawiki b/bip-0300.mediawiki
new file mode 100644
index 0000000..1d7deb6
--- /dev/null
+++ b/bip-0300.mediawiki
@@ -0,0 +1,308 @@
+
+    BIP: 300
+    Layer: Consensus (soft fork)
+    Title: Hashrate Escrows (Consensus layer)
+    Author: Paul Sztorc 
+            CryptAxe 
+    Comments-Summary: No comments yet.
+    Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-???????
+    Status: Draft
+    Type: Standards Track
+    Created: 2017-08-14
+    License: BSD-2-Clause
+    Post-History: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-May/014364.html
+
+ +==Abstract== + +A "Hashrate Escrow" is a clearer term for the concept of "locked to an SPV Proof", which is itself a restatement of the phrase "within a sidechain" as described in [https://blockstream.com/sidechains.pdf the 2014 Blockstream whitepaper]. + +A Hashrate Escrow resembles a 2-of-3 multisig escrow, where the 3rd party (who will arbitrate any disputes) is a decentralized group of people: the dynamic-membership set of Bitcoin Miners. However, the 3rd party does not sign escrow-withdrawal transactions with a private key. Instead, these are "signed" by the accumulation of hashpower over time. + +This project has [http://www.drivechain.info/ a website] which includes [http://www.drivechain.info/faq/index.html an FAQ]. + + +==Motivation== + +In practice these escrows are likely to be "asymmetric sidechains" of Bitcoin (such as [http://www.rsk.co/ Rootstock]) or "virtual chains" within Bitcoin (such as [https://github.com/blockstack/virtualchain proposed by Blockstack] in mid-2016). + +Sidechains have many potential benefits, including: + +# Protect Bitcoin from competition from altcoins and spinoffs. +# Protect Bitcoin from hard fork campaigns. (Such campaigns represent an existential threat to Bitcoin, as well as an avenue for developer corruption.) +# Help with review, by making it much easier for reviewers to ignore bad ideas. +# Provide an avenue for good-but-confusing ideas to prove their value safely. + + + +==Specification== + +==== Components ==== + +Hashrate Escrows are built of two types of component: [1] new databases, and [2] new message-interpretations. + +===== 1. New Databases ===== + +* D1. "Escrow_DB" -- a database of "accounts" and their attributes. +* D2. "Withdrawal_DB" -- a database of pending withdrawals from these accounts, and their statuses. + +Please note that these structures (D1 and D2) will not literally exist anywhere in the blockchain. Instead they are constructed from messages...these messages, in contrast, *will* exist in the blockchain (with the exception of M4). + +===== 2. New Messages ===== + +* M1. "Propose New Escrow" +* M2. "ACK Escrow Proposal" +* M3. "Propose Withdrawal" +* M4. (implied) "ACK Withdrawal" +* M5. "Execute Deposit" -- a transfer of BTC from-main-to-side +* M6. "Execute Withdrawal" -- a transfer of BTC from-side-to-main + + + + +=== Adding Sidechains (D1, M1, M2) === + +==== D1 -- "Escrow_DB" ==== + +The table below enumerates the new database fields, their size in bytes, and their purpose. In general, an escrow designer (for example, a sidechain-designer), is free to choose any value for these. + + + +{| class="wikitable" +! Field No. +! Label +! Type +! Description / Purpose +|- +| 1 +| Escrow Number +| uint8_t +| A number assigned to the entire escrow. Used to make it easy to refer to each escrow. +|- +| 2 +| Sidechain Deposit Script Hex +| string +| The script that will be deposited to, and update the CTIP of the sidechain. +|- +| 3 +| Sidechain Private Key +| string +| The private key of the sidechain deposit script. +|- +| 4 +| Escrow Name +| string +| A human-readable name of the sidechain. +|- +| 5 +| Escrow Description +| string +| A human-readable name description of the sidechain. More than enough space to hold a 32 byte hash. +|- +| 6 +| Hash ID 1 +| uint256 +| A field of 32 bytes, which could be any bytes such as a sha256 hash. +|- +| 7 +| Hash ID 2 +| uint256 +| A field of 32 bytes, which could be any bytes such as a sha256 hash. +|- +| 8 +| "CTIP" -- Part 1 "TxID" +| uint256 +| The CTIP, or "Critical (TxID, Index) Pair" is a variable for keeping track of where the escrow's money is (ie, which member of the UTXO set). +|- +| 9 +| "CTIP" -- Part 2 "Index" +| int32_t +| Of the CTIP, this is second element of the pair: the Index. See #9 above. +|- +|} + +D1 is updated via M1 and M2. + +( The following messages were modeled on SegWit -- see [https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki#commitment-structure here] and [https://github.com/DriveNetTESTDRIVE/DriveNet/blob/564516653c1d876429382971a011f5f6119f7eb4/src/validation.cpp#L3348-L3375 here]. ) + + +==== M1 -- "Propose New Sidechain" ==== + + 1-byte - OP_RETURN (0x6a) + 4-byte - Commitment header (0xD5E0C4AF) + N-byte - The serialization of the sidechain. + + +==== M2 -- "ACK Sidechain Proposal" ==== + + 1-byte - OP_RETURN (0x6a) + 4-byte - Commitment header (0xD6E1C5BF) + 32-byte - Commitment hash: sha256D hash of sidechain's serialization + +==== New Block Validation Rules ==== + + +# Escrows are added in a procedure that resembles BIP 9 soft fork activation: the network must see a properly-formatted M1, followed by "acknowledgment" of the sidechain in 95% of the following 2016 blocks. +# It is possible to "overwrite" an escrow. This requires 6 months (26298 blocks) of M2s, instead of 2 weeks (XXXX). This possibility does not change the security assumptions (because we already assume that users perform extra-protocolic validation at a rate of 1 bit per 26298 blocks). + + + +=== Withdrawing from Escrows (D2, M3, M4) === + +==== D2 -- "Withdrawal_DB" ==== + +D2 changes deterministically with respect to M3, M4, M5, and M6. + +{| class="wikitable" +! Field No. +! Label +! Type +! Description / Purpose +|- +| 1 +| Escrow Number +| uint8_t +| Links the withdrawal-request to a specific escrow. +|- +| 2 +| WT^ Hash +| uint256 +| This is a "blinded transaction id" (ie, the double-Sha256 of a txn that has had two fields zeroed out, see M6) of a withdrawal-attempt. +|- +| 3 +| ACKs (Work Score) +| uint16_t +| The current total number of ACKs (PoW) +|- +| 4 +| Blocks Remaining (Age) +| uint16_t +| The number of blocks which this WT^ has remaining to accumulate ACKs +|} + + +==== New Block Validation Rules for D2 ==== + +# A hash commitment to D2 exists in each block (even if D2 is blank). +# Withdrawals in D2 are sorted first by field #1 (Escrow Number) and second by field #4 (Age). This imposes a unique sort. +# From one block to the next, "Age" fields must increase by exactly 1. +# Withdrawals are stored in D2 until they fail ("Age" = "MaxAge"), or they succeed (the blockchain contains a txn whose blinded txID matches "WT^"). + +In addition, there are special rules for the "ACKs" field (see M4 below). + +==== M3 -- "Propose Withdrawal" ==== + + 1-byte - OP_RETURN (0x6a) + 1-byte - Push the following 36 bytes (0x24) + 4-byte - Commitment header (0xD45AA943) + 32-byte - The WT^ hash to populate a new D2 entry + + +==== New Block Validation Rules for M3 ==== + +# If the network detects a properly-formatted M3, it must add an entry to D2 in the very next block. The starting values of fields #3 and #4 are zero, and #5 is pulled over by extracting the relevant value from D1. +# Each block can only contain one M3 per sidechain. + + +==== M4 -- "ACK Withdrawal" ==== + +M4 is a way of describing changes to the "ACKs" column of D2. + +From one block to the next, "ACKs" can only change as follows: + +* The ACK-counter of any withdrawal can only change by (-1,0,+1). +* Within a sidechain-group, upvoting one withdrawal ("+1") requires you to downvote all other withdrawals in that group. However, the minimum ACK-value is zero (and, therefore, downvotes cannot reduce it below zero). +* While only one withdrawal can be upvoted at once, they can all be unchanged at once ("abstain") and they can all be downvoted at once ("alarm"). + +One option for explicit transmission of M4 is: + + 4-byte - Message identifier (0x????????) + 1-byte - Version of this message + 1-byte - Length (in bytes) of this message; total number of withdrawal attempts; y = ceiling( sum_i(m_i +2)/8 ). Nodes should already know what length to expect, because they know the sequence of M3s and therefore the vector of WT^s. + N-byte - stream of bits (not bytes), with a 1 indicating the position of the chosen action [downvote all, abstain, upvote1, upvote2, ...] + +But sometimes M4 does not need to be transmitted at all! If there are n Escrows and m Withdrawals-per-escrow, then there are (m+2)^n total candidates for the next D2. So, when m and n are low, all of the possible D2s can be trivially computed in advance. + +Miners can impose a "soft limit" on m, blocking new withdrawal-attempts until previous ones expire. For a worst-case scenario of n=200 and m=1,000, honest nodes can communicate M4 with ~25 KB per block [4+1+1+(200\*(1000+1+1)/8)]. + + +=== Depositing and Withdrawing (M5, M6) === + +Both M5 and M6 are regular Bitcoin txns. They are identified by meeting an important criteria: they select a one of the Critical TxID-index Pairs (a "CTIP") as one of their inputs. + +Just as these txns must select a CTIP input, they must create a new CTIP output. D1 is then updated to match only the latest CTIP output. The purpose of this is to have all of the escrow's money (ie all of the sidechain's money) in one TxID, so that depositors immediately undo any UTXO bloat they may cause. + +Deposits ("M5") are distinguished from withdrawals ("M6") by simply checking to see if money is "going in", or "out". + +https://github.com/DriveNetTESTDRIVE/DriveNet/blob/564516653c1d876429382971a011f5f6119f7eb4/src/validation.cpp#L647-L742 + + +==== M5. "Make a Deposit" -- a transfer of BTC from-main-to-side ==== + +As far as mainchain consensus is concerned, deposits to the escrow are always valid. + +However, in practice there will be additional requirements. The escrow account (ie the "sidechain") needs to know how to credit depositors. One well-known method, is for mainchain depositors to append a zero-value OP Return to a Deposit txn, so that the sidechain knows how to credit funds. Mainchain users must upgrade their wallet software, of course, (on an individual basis) in order to become aware of and take advantage of new deposit-methods. + + + +==== M6. "Execute Withdrawal" -- a transfer of BTC from-side-to-main ==== + +We come, finally, to the critical matter: where users can take their money *out* of the escrow account, and return it to the "regular" UTXO set. As previously mentioned, this txn is one which (a) spends from a CTIP and (b) reduces the quantity of BTC in an account's CTIP. Most of the work has already been done by D1, M3, M4, and D2. Furthermore, existing Bitcoin tx-rules prevent the sidechain from ever withdrawing more money than has been placed into it. + +In each block, a withdrawal in D2 is considered "approved" if its "ACKs" value meets the threshold (13,150). + +Approved withdrawals give the green light to their respective "WT^". A "WT^" is 32-bytes which aspire to represent the withdrawing transaction (the txn that actually withdraws funds from the escrow). The two cannot match exactly, because "WT^" is defined at onset, and the withdrawing TxID depends on the its CTIP input (which is constantly changing). + +To solve this, we define a "blinded TxID" as a way of hashing a txn, in which some bytes are first overwritten with zeros. Specifically, these bytes are the first input and the first output. + +So, withdrawals must meet the following three criteria: + +# "Be ACKed" -- The "blinded TxID" of this txn must be member of the "approved candidate" set in the D2 of this block. +# "Return Change to Account" -- TxOut0 must pay to the "critical account" (see D1) that corresponds to the CTIP that was selected as a TxIn. +# "Return *all* Change to Account" -- Sum of inputs must equal the sum of outputs. No traditional tx fee is possible. + + + + + +==Backward compatibility== + + +As a soft fork, older software will continue to operate without modification. Non-upgraded nodes will see a number of phenomena that they don't understand -- coinbase txns with non-txn data, value accumulating in anyone-can-spend UTXOs for months at a time, and then random amounts leaving the UTXO in single, infrequent bursts. However, these phenomena don't affect them, or the validity of the money that they receive. + +( As a nice bonus, note that the sidechains themselves inherit a resistance to hard forks. The only way to guarantee that the WT^s reported by different clients will continue to match identically, is to upgrade sidechains via soft forks of themselves. ) + + +==Deployment== + + +This BIP will be deployed by "version bits" BIP9 with the name "hrescrow" and using bit 4. + +
+// Deployment of Drivechains (BIPX, BIPY)
+consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].bit = 4;
+consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].nStartTime = 1579072881; // January 15th, 2020.
+consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].nTimeout = 1610695281; // January 15th, 2021.
+
+ +==Reference Implementation== + + +See: https://github.com/DriveNetTESTDRIVE/DriveNet + +Also, for interest, see an example sidechain here: https://github.com/drivechain-project/bitcoin/tree/sidechainBMM + + +==References== + +See http://www.drivechain.info/literature/index.html + + +==Credits== + +Thanks to everyone who contributed to the discussion, especially: ZmnSCPxj, Adam Back, Peter Todd, Dan Anderson, Sergio Demian Lerner, Chris Stewart, Matt Corallo, Sjors Provoost, Tier Nolan, Erik Aronesty, Jason Dreyzehner, Joe Miyamoto, Ben Goldhaber. + + +==Copyright== + +This BIP is licensed under the BSD 2-clause license. diff --git a/bip-0300/appendix-1.txt b/bip-0300/appendix-1.txt new file mode 100644 index 0000000..736a6c4 --- /dev/null +++ b/bip-0300/appendix-1.txt @@ -0,0 +1,45 @@ + +==== Two Withdrawals at Once ==== + +Currently, the documentation and code describe a situation where only one withdrawal can proceed at a time. As a result, one "train" (carrying everyone's withdrawals) leaves the station every 3 months, and takes 3-6 months to reach its destination. + +Thus, if a withdrawing-user is very unlucky, and "just misses" the train, this user must wait double-long. First, (s)he must wait for the missed-train to reach its destination. Second, (s)he must board the new train, and wait for *it* to reach its destination. Each of these steps takes 3-6 months. + +So, even when withdrawals always go as quickly as possible (3 months each), the total time varies, from 3 months (0 months waiting + 3 months travel) to 6 months (3 months waiting + 3 months travel). The average is 4.5 months. + +To improve this, we allow for slightly different behavior if the highest-ACK-withdrawal [1st] has an ACK score >= 6575; and [2nd] is not tied with any other withdrawal. + +Basically: a second train can leave, if the furthest train is 50+% of the way to its destination. + +So, previously, for m trains, M4 could be any of the following: + + abstain + alarm (move all trains backwards) + move train #1 forward (and others backwards) + move train #2 forward (and others backwards) + ... + move train #3 forward (and others backwards) + +If our new special conditions apply, we now double the (m-1) elements, to accommodate a second train: + + |abstain + |alarm (move all trains backwards) + + |advance furthest train + advance train #1 (regress all others) + |advance furthest train + advance train #2 (regress all others) + |... + |advance furthest train + advance train #(m-1) (regress all others) + + |regress furthest train + advance train #1 (regress all others) + |regress furthest train + advance train #2 (regress all others) + |... + |regress furthest train + advance train #(m-1) (regress all others) + + +It is theoretically possible (but in practice probably impossible) to troll this rule, by getting two (or even three) withdrawals to have >6575 ACK scores, and then getting these to *tie* for first place. Then they'd both be furthest. Hence the second condition prohibiting this new behavior, if the furthest trains have any ACK-score ties. + +This simple change, which has almost zero impact on the security assumptions, improves the monthly total wait times drastically: + + Worst-case: 6 --> 4.5 + Average: 4.5 --> 3.75 + Std Dev: ~.91 --> ~.45 diff --git a/bip-0300/images.txt b/bip-0300/images.txt new file mode 100644 index 0000000..2fbbf63 --- /dev/null +++ b/bip-0300/images.txt @@ -0,0 +1 @@ +Images used as reference in the documentation. diff --git a/bip-0300/two-groups.png b/bip-0300/two-groups.png new file mode 100644 index 0000000..c8a3ffa Binary files /dev/null and b/bip-0300/two-groups.png differ diff --git a/bip-hashrate-escrows.mediawiki b/bip-hashrate-escrows.mediawiki deleted file mode 100644 index 26ed32e..0000000 --- a/bip-hashrate-escrows.mediawiki +++ /dev/null @@ -1,308 +0,0 @@ -
-    BIP: ????
-    Layer: Consensus (soft fork)
-    Title: Hashrate Escrows (Consensus layer)
-    Author: Paul Sztorc 
-            CryptAxe 
-    Comments-Summary: No comments yet.
-    Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-???????
-    Status: Draft
-    Type: Standards Track
-    Created: 2017-08-14
-    License: BSD-2-Clause
-    Post-History: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-May/014364.html
-
- -==Abstract== - -A "Hashrate Escrow" is a clearer term for the concept of "locked to an SPV Proof", which is itself a restatement of the phrase "within a sidechain" as described in [https://blockstream.com/sidechains.pdf the 2014 Blockstream whitepaper]. - -A Hashrate Escrow resembles a 2-of-3 multisig escrow, where the 3rd party (who will arbitrate any disputes) is a decentralized group of people: the dynamic-membership set of Bitcoin Miners. However, the 3rd party does not sign escrow-withdrawal transactions with a private key. Instead, these are "signed" by the accumulation of hashpower over time. - -This project has [http://www.drivechain.info/ a website] which includes [http://www.drivechain.info/faq/index.html an FAQ]. - - -==Motivation== - -In practice these escrows are likely to be "asymmetric sidechains" of Bitcoin (such as [http://www.rsk.co/ Rootstock]) or "virtual chains" within Bitcoin (such as [https://github.com/blockstack/virtualchain proposed by Blockstack] in mid-2016). - -Sidechains have many potential benefits, including: - -# Protect Bitcoin from competition from altcoins and spinoffs. -# Protect Bitcoin from hard fork campaigns. (Such campaigns represent an existential threat to Bitcoin, as well as an avenue for developer corruption.) -# Help with review, by making it much easier for reviewers to ignore bad ideas. -# Provide an avenue for good-but-confusing ideas to prove their value safely. - - - -==Specification== - -==== Components ==== - -Hashrate Escrows are built of two types of component: [1] new databases, and [2] new message-interpretations. - -===== 1. New Databases ===== - -* D1. "Escrow_DB" -- a database of "accounts" and their attributes. -* D2. "Withdrawal_DB" -- a database of pending withdrawals from these accounts, and their statuses. - -Please note that these structures (D1 and D2) will not literally exist anywhere in the blockchain. Instead they are constructed from messages...these messages, in contrast, *will* exist in the blockchain (with the exception of M4). - -===== 2. New Messages ===== - -* M1. "Propose New Escrow" -* M2. "ACK Escrow Proposal" -* M3. "Propose Withdrawal" -* M4. (implied) "ACK Withdrawal" -* M5. "Execute Deposit" -- a transfer of BTC from-main-to-side -* M6. "Execute Withdrawal" -- a transfer of BTC from-side-to-main - - - - -=== Adding Sidechains (D1, M1, M2) === - -==== D1 -- "Escrow_DB" ==== - -The table below enumerates the new database fields, their size in bytes, and their purpose. In general, an escrow designer (for example, a sidechain-designer), is free to choose any value for these. - - - -{| class="wikitable" -! Field No. -! Label -! Type -! Description / Purpose -|- -| 1 -| Escrow Number -| uint8_t -| A number assigned to the entire escrow. Used to make it easy to refer to each escrow. -|- -| 2 -| Sidechain Deposit Script Hex -| string -| The script that will be deposited to, and update the CTIP of the sidechain. -|- -| 3 -| Sidechain Private Key -| string -| The private key of the sidechain deposit script. -|- -| 4 -| Escrow Name -| string -| A human-readable name of the sidechain. -|- -| 5 -| Escrow Description -| string -| A human-readable name description of the sidechain. More than enough space to hold a 32 byte hash. -|- -| 6 -| Hash ID 1 -| uint256 -| A field of 32 bytes, which could be any bytes such as a sha256 hash. -|- -| 7 -| Hash ID 2 -| uint256 -| A field of 32 bytes, which could be any bytes such as a sha256 hash. -|- -| 8 -| "CTIP" -- Part 1 "TxID" -| uint256 -| The CTIP, or "Critical (TxID, Index) Pair" is a variable for keeping track of where the escrow's money is (ie, which member of the UTXO set). -|- -| 9 -| "CTIP" -- Part 2 "Index" -| int32_t -| Of the CTIP, this is second element of the pair: the Index. See #9 above. -|- -|} - -D1 is updated via M1 and M2. - -( The following messages were modeled on SegWit -- see [https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki#commitment-structure here] and [https://github.com/DriveNetTESTDRIVE/DriveNet/blob/564516653c1d876429382971a011f5f6119f7eb4/src/validation.cpp#L3348-L3375 here]. ) - - -==== M1 -- "Propose New Sidechain" ==== - - 1-byte - OP_RETURN (0x6a) - 4-byte - Commitment header (0xD5E0C4AF) - N-byte - The serialization of the sidechain. - - -==== M2 -- "ACK Sidechain Proposal" ==== - - 1-byte - OP_RETURN (0x6a) - 4-byte - Commitment header (0xD6E1C5BF) - 32-byte - Commitment hash: sha256D hash of sidechain's serialization - -==== New Block Validation Rules ==== - - -# Escrows are added in a procedure that resembles BIP 9 soft fork activation: the network must see a properly-formatted M1, followed by "acknowledgment" of the sidechain in 95% of the following 2016 blocks. -# It is possible to "overwrite" an escrow. This requires 6 months (26298 blocks) of M2s, instead of 2 weeks (XXXX). This possibility does not change the security assumptions (because we already assume that users perform extra-protocolic validation at a rate of 1 bit per 26298 blocks). - - - -=== Withdrawing from Escrows (D2, M3, M4) === - -==== D2 -- "Withdrawal_DB" ==== - -D2 changes deterministically with respect to M3, M4, M5, and M6. - -{| class="wikitable" -! Field No. -! Label -! Type -! Description / Purpose -|- -| 1 -| Escrow Number -| uint8_t -| Links the withdrawal-request to a specific escrow. -|- -| 2 -| WT^ Hash -| uint256 -| This is a "blinded transaction id" (ie, the double-Sha256 of a txn that has had two fields zeroed out, see M6) of a withdrawal-attempt. -|- -| 3 -| ACKs (Work Score) -| uint16_t -| The current total number of ACKs (PoW) -|- -| 4 -| Blocks Remaining (Age) -| uint16_t -| The number of blocks which this WT^ has remaining to accumulate ACKs -|} - - -==== New Block Validation Rules for D2 ==== - -# A hash commitment to D2 exists in each block (even if D2 is blank). -# Withdrawals in D2 are sorted first by field #1 (Escrow Number) and second by field #4 (Age). This imposes a unique sort. -# From one block to the next, "Age" fields must increase by exactly 1. -# Withdrawals are stored in D2 until they fail ("Age" = "MaxAge"), or they succeed (the blockchain contains a txn whose blinded txID matches "WT^"). - -In addition, there are special rules for the "ACKs" field (see M4 below). - -==== M3 -- "Propose Withdrawal" ==== - - 1-byte - OP_RETURN (0x6a) - 1-byte - Push the following 36 bytes (0x24) - 4-byte - Commitment header (0xD45AA943) - 32-byte - The WT^ hash to populate a new D2 entry - - -==== New Block Validation Rules for M3 ==== - -# If the network detects a properly-formatted M3, it must add an entry to D2 in the very next block. The starting values of fields #3 and #4 are zero, and #5 is pulled over by extracting the relevant value from D1. -# Each block can only contain one M3 per sidechain. - - -==== M4 -- "ACK Withdrawal" ==== - -M4 is a way of describing changes to the "ACKs" column of D2. - -From one block to the next, "ACKs" can only change as follows: - -* The ACK-counter of any withdrawal can only change by (-1,0,+1). -* Within a sidechain-group, upvoting one withdrawal ("+1") requires you to downvote all other withdrawals in that group. However, the minimum ACK-value is zero (and, therefore, downvotes cannot reduce it below zero). -* While only one withdrawal can be upvoted at once, they can all be unchanged at once ("abstain") and they can all be downvoted at once ("alarm"). - -One option for explicit transmission of M4 is: - - 4-byte - Message identifier (0x????????) - 1-byte - Version of this message - 1-byte - Length (in bytes) of this message; total number of withdrawal attempts; y = ceiling( sum_i(m_i +2)/8 ). Nodes should already know what length to expect, because they know the sequence of M3s and therefore the vector of WT^s. - N-byte - stream of bits (not bytes), with a 1 indicating the position of the chosen action [downvote all, abstain, upvote1, upvote2, ...] - -But sometimes M4 does not need to be transmitted at all! If there are n Escrows and m Withdrawals-per-escrow, then there are (m+2)^n total candidates for the next D2. So, when m and n are low, all of the possible D2s can be trivially computed in advance. - -Miners can impose a "soft limit" on m, blocking new withdrawal-attempts until previous ones expire. For a worst-case scenario of n=200 and m=1,000, honest nodes can communicate M4 with ~25 KB per block [4+1+1+(200\*(1000+1+1)/8)]. - - -=== Depositing and Withdrawing (M5, M6) === - -Both M5 and M6 are regular Bitcoin txns. They are identified by meeting an important criteria: they select a one of the Critical TxID-index Pairs (a "CTIP") as one of their inputs. - -Just as these txns must select a CTIP input, they must create a new CTIP output. D1 is then updated to match only the latest CTIP output. The purpose of this is to have all of the escrow's money (ie all of the sidechain's money) in one TxID, so that depositors immediately undo any UTXO bloat they may cause. - -Deposits ("M5") are distinguished from withdrawals ("M6") by simply checking to see if money is "going in", or "out". - -https://github.com/DriveNetTESTDRIVE/DriveNet/blob/564516653c1d876429382971a011f5f6119f7eb4/src/validation.cpp#L647-L742 - - -==== M5. "Make a Deposit" -- a transfer of BTC from-main-to-side ==== - -As far as mainchain consensus is concerned, deposits to the escrow are always valid. - -However, in practice there will be additional requirements. The escrow account (ie the "sidechain") needs to know how to credit depositors. One well-known method, is for mainchain depositors to append a zero-value OP Return to a Deposit txn, so that the sidechain knows how to credit funds. Mainchain users must upgrade their wallet software, of course, (on an individual basis) in order to become aware of and take advantage of new deposit-methods. - - - -==== M6. "Execute Withdrawal" -- a transfer of BTC from-side-to-main ==== - -We come, finally, to the critical matter: where users can take their money *out* of the escrow account, and return it to the "regular" UTXO set. As previously mentioned, this txn is one which (a) spends from a CTIP and (b) reduces the quantity of BTC in an account's CTIP. Most of the work has already been done by D1, M3, M4, and D2. Furthermore, existing Bitcoin tx-rules prevent the sidechain from ever withdrawing more money than has been placed into it. - -In each block, a withdrawal in D2 is considered "approved" if its "ACKs" value meets the threshold (13,150). - -Approved withdrawals give the green light to their respective "WT^". A "WT^" is 32-bytes which aspire to represent the withdrawing transaction (the txn that actually withdraws funds from the escrow). The two cannot match exactly, because "WT^" is defined at onset, and the withdrawing TxID depends on the its CTIP input (which is constantly changing). - -To solve this, we define a "blinded TxID" as a way of hashing a txn, in which some bytes are first overwritten with zeros. Specifically, these bytes are the first input and the first output. - -So, withdrawals must meet the following three criteria: - -# "Be ACKed" -- The "blinded TxID" of this txn must be member of the "approved candidate" set in the D2 of this block. -# "Return Change to Account" -- TxOut0 must pay to the "critical account" (see D1) that corresponds to the CTIP that was selected as a TxIn. -# "Return *all* Change to Account" -- Sum of inputs must equal the sum of outputs. No traditional tx fee is possible. - - - - - -==Backward compatibility== - - -As a soft fork, older software will continue to operate without modification. Non-upgraded nodes will see a number of phenomena that they don't understand -- coinbase txns with non-txn data, value accumulating in anyone-can-spend UTXOs for months at a time, and then random amounts leaving the UTXO in single, infrequent bursts. However, these phenomena don't affect them, or the validity of the money that they receive. - -( As a nice bonus, note that the sidechains themselves inherit a resistance to hard forks. The only way to guarantee that the WT^s reported by different clients will continue to match identically, is to upgrade sidechains via soft forks of themselves. ) - - -==Deployment== - - -This BIP will be deployed by "version bits" BIP9 with the name "hrescrow" and using bit 4. - -
-// Deployment of Drivechains (BIPX, BIPY)
-consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].bit = 4;
-consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].nStartTime = 1579072881; // January 15th, 2020.
-consensus.vDeployments[Consensus::DEPLOYMENT_DRIVECHAINS].nTimeout = 1610695281; // January 15th, 2021.
-
- -==Reference Implementation== - - -See: https://github.com/DriveNetTESTDRIVE/DriveNet - -Also, for interest, see an example sidechain here: https://github.com/drivechain-project/bitcoin/tree/sidechainBMM - - -==References== - -See http://www.drivechain.info/literature/index.html - - -==Credits== - -Thanks to everyone who contributed to the discussion, especially: ZmnSCPxj, Adam Back, Peter Todd, Dan Anderson, Sergio Demian Lerner, Chris Stewart, Matt Corallo, Sjors Provoost, Tier Nolan, Erik Aronesty, Jason Dreyzehner, Joe Miyamoto, Ben Goldhaber. - - -==Copyright== - -This BIP is licensed under the BSD 2-clause license. diff --git a/bip-hashrate-escrows/appendix-1.txt b/bip-hashrate-escrows/appendix-1.txt deleted file mode 100644 index 736a6c4..0000000 --- a/bip-hashrate-escrows/appendix-1.txt +++ /dev/null @@ -1,45 +0,0 @@ - -==== Two Withdrawals at Once ==== - -Currently, the documentation and code describe a situation where only one withdrawal can proceed at a time. As a result, one "train" (carrying everyone's withdrawals) leaves the station every 3 months, and takes 3-6 months to reach its destination. - -Thus, if a withdrawing-user is very unlucky, and "just misses" the train, this user must wait double-long. First, (s)he must wait for the missed-train to reach its destination. Second, (s)he must board the new train, and wait for *it* to reach its destination. Each of these steps takes 3-6 months. - -So, even when withdrawals always go as quickly as possible (3 months each), the total time varies, from 3 months (0 months waiting + 3 months travel) to 6 months (3 months waiting + 3 months travel). The average is 4.5 months. - -To improve this, we allow for slightly different behavior if the highest-ACK-withdrawal [1st] has an ACK score >= 6575; and [2nd] is not tied with any other withdrawal. - -Basically: a second train can leave, if the furthest train is 50+% of the way to its destination. - -So, previously, for m trains, M4 could be any of the following: - - abstain - alarm (move all trains backwards) - move train #1 forward (and others backwards) - move train #2 forward (and others backwards) - ... - move train #3 forward (and others backwards) - -If our new special conditions apply, we now double the (m-1) elements, to accommodate a second train: - - |abstain - |alarm (move all trains backwards) - - |advance furthest train + advance train #1 (regress all others) - |advance furthest train + advance train #2 (regress all others) - |... - |advance furthest train + advance train #(m-1) (regress all others) - - |regress furthest train + advance train #1 (regress all others) - |regress furthest train + advance train #2 (regress all others) - |... - |regress furthest train + advance train #(m-1) (regress all others) - - -It is theoretically possible (but in practice probably impossible) to troll this rule, by getting two (or even three) withdrawals to have >6575 ACK scores, and then getting these to *tie* for first place. Then they'd both be furthest. Hence the second condition prohibiting this new behavior, if the furthest trains have any ACK-score ties. - -This simple change, which has almost zero impact on the security assumptions, improves the monthly total wait times drastically: - - Worst-case: 6 --> 4.5 - Average: 4.5 --> 3.75 - Std Dev: ~.91 --> ~.45 -- cgit v1.2.3 From 34f0fe5b2af4f7c82fbe6ed0706bbb3feef94acf Mon Sep 17 00:00:00 2001 From: Luke Dashjr Date: Fri, 20 Sep 2019 17:50:14 +0000 Subject: BIP 300: Fix preamble --- bip-0300.mediawiki | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/bip-0300.mediawiki b/bip-0300.mediawiki index 1d7deb6..08f8994 100644 --- a/bip-0300.mediawiki +++ b/bip-0300.mediawiki @@ -1,16 +1,16 @@
-    BIP: 300
-    Layer: Consensus (soft fork)
-    Title: Hashrate Escrows (Consensus layer)
-    Author: Paul Sztorc 
-            CryptAxe 
-    Comments-Summary: No comments yet.
-    Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-???????
-    Status: Draft
-    Type: Standards Track
-    Created: 2017-08-14
-    License: BSD-2-Clause
-    Post-History: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-May/014364.html
+  BIP: 300
+  Layer: Consensus (soft fork)
+  Title: Hashrate Escrows (Consensus layer)
+  Author: Paul Sztorc 
+          CryptAxe 
+  Comments-Summary: No comments yet.
+  Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0300
+  Status: Draft
+  Type: Standards Track
+  Created: 2017-08-14
+  License: BSD-2-Clause
+  Post-History: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-May/014364.html
 
==Abstract== -- cgit v1.2.3 From af134f361fd4e0527b57aa7089c1154d20dce6ce Mon Sep 17 00:00:00 2001 From: MarcoFalke Date: Tue, 24 Sep 2019 15:05:03 -0400 Subject: Update bip-0158.mediawiki Fix: * Render issue in `` tag (c.f. https://en.bitcoin.it/wiki/BIP_0158#Contents) * Remove remnants of the second filter type --- bip-0158.mediawiki | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/bip-0158.mediawiki b/bip-0158.mediawiki index ad46da6..ce4a4af 100644 --- a/bip-0158.mediawiki +++ b/bip-0158.mediawiki @@ -19,7 +19,7 @@ This BIP describes a structure for compact filters on block data, for use in the BIP 157 light client protocolbip-0157.mediawiki. The filter construction proposed is an alternative to Bloom filters, as used in BIP 37, that minimizes filter size by using Golomb-Rice coding for compression. This -document specifies two initial types of filters based on this construction that +document specifies one initial filter type based on this construction that enables basic wallets and applications with more advanced smart contracts. == Motivation == @@ -284,7 +284,7 @@ We exclude all outputs that start with OP_RETURN in order to allow filters to easily be committed to in the future via a soft-fork. A likely area for future commitments is an additional OP_RETURN output in the coinbase transaction similar to the current witness commitment -https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki. By +https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki. By excluding all OP_RETURN outputs we avoid a circular dependency between the commitment, and the item being committed to. @@ -322,7 +322,7 @@ This BIP allocates a new service bit: |- | NODE_COMPACT_FILTERS | style="white-space: nowrap;" | 1 << 6 -| If enabled, the node MUST respond to all BIP 157 messages for filter types 0x00 and 0x01 +| If enabled, the node MUST respond to all BIP 157 messages for filter type 0x00 |} == Compatibility == -- cgit v1.2.3 From 773a7fe993d8557bc9bdb39d7682c7f6b0c754c9 Mon Sep 17 00:00:00 2001 From: Luke Dashjr Date: Tue, 24 Sep 2019 19:31:13 +0000 Subject: README: Fix BIP 100 fields --- README.mediawiki | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.mediawiki b/README.mediawiki index aa6c50e..a55a776 100644 --- a/README.mediawiki +++ b/README.mediawiki @@ -416,8 +416,8 @@ Those proposing changes should consider that ultimately consent may rest with th |- | [[bip-0100.mediawiki|100]] | Consensus (hard fork) -| Dynamic block size by miner vote -| Jeff Garzik, Tom Harding, Dagur Valberg +| Dynamic maximum block size by miner vote +| Jeff Garzik, Tom Harding, Dagur Valberg Johannsson | Standard | Draft |- style="background-color: #ffcfcf" -- cgit v1.2.3 From b1b248fc6ade930c4c716eb9fed1b48be6796356 Mon Sep 17 00:00:00 2001 From: Luke Dashjr Date: Tue, 24 Sep 2019 19:33:03 +0000 Subject: Update BIP 100 status Draft->Rejected --- README.mediawiki | 4 ++-- bip-0100.mediawiki | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/README.mediawiki b/README.mediawiki index a469b22..de98f05 100644 --- a/README.mediawiki +++ b/README.mediawiki @@ -448,13 +448,13 @@ Those proposing changes should consider that ultimately consent may rest with th | Jorge TimΓ³n | Informational | Draft -|- +|- style="background-color: #ffcfcf" | [[bip-0100.mediawiki|100]] | Consensus (hard fork) | Dynamic maximum block size by miner vote | Jeff Garzik, Tom Harding, Dagur Valberg Johannsson | Standard -| Draft +| Rejected |- style="background-color: #ffcfcf" | [[bip-0101.mediawiki|101]] | Consensus (hard fork) diff --git a/bip-0100.mediawiki b/bip-0100.mediawiki index 6b322d0..aaf6beb 100644 --- a/bip-0100.mediawiki +++ b/bip-0100.mediawiki @@ -7,7 +7,7 @@ Dagur Valberg Johannsson Comments-Summary: No comments yet. Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0100 - Status: Draft + Status: Rejected Type: Standards Track Created: 2015-06-11 License: BSD-2-Clause -- cgit v1.2.3 From 311c085cab2929754c79dd6e330bf888ec000e73 Mon Sep 17 00:00:00 2001 From: Gleb Naumenko Date: Wed, 25 Sep 2019 14:11:03 +0300 Subject: bip for erlay messages --- bip-reconcil.mediawiki | 296 +++++++++++++++++++++++++++++++++++ bip-reconcil/bisection.png | Bin 0 -> 60787 bytes bip-reconcil/minisketch.py | 157 +++++++++++++++++++ bip-reconcil/recon_scheme_merged.png | Bin 0 -> 113169 bytes 4 files changed, 453 insertions(+) create mode 100644 bip-reconcil.mediawiki create mode 100644 bip-reconcil/bisection.png create mode 100755 bip-reconcil/minisketch.py create mode 100644 bip-reconcil/recon_scheme_merged.png diff --git a/bip-reconcil.mediawiki b/bip-reconcil.mediawiki new file mode 100644 index 0000000..be34c55 --- /dev/null +++ b/bip-reconcil.mediawiki @@ -0,0 +1,296 @@ +
+  BIP: ???
+  Layer: Peer Services
+  Title: Transaction announcements reconciliation
+  Author: Gleb Naumenko , Pieter Wuille 
+  Comments-Summary: ???
+  Comments-URI: ???
+  Status: Draft
+  Type: Standards Track
+  Created: 2010-00-00
+  License: PD
+
+ +==Abstract== + +This document specifies a P2P protocol extension for reconciliation of transaction announcements between 2 nodes, which is a building block for efficient transaction relay protocols (e.g., [https://arxiv.org/pdf/1905.10518.pdf Erlay]). This is a step towards increasing the connectivity of the network for almost no bandwidth cost. + +==Motivation== + +Currently in the Bitcoin network, every 32-byte transaction ID is announced in at least one direction between every pair of connected peers, via INV messages. This results in high cost of announcing transactions: ''O(nodes * connections_per_node)''. + +A reconciliation-based protocol which uses the technique suggested in this document can have better scaling properties than INV-based flooding. + +Increasing the connectivity of the network makes the network more robust to partitioning attacks; thus, improving the bandwidth scaling of transaction relay to ''O(nodes)'' (and without a high constant overhead) would allow us to improve the security of the network by increasing connectivity. It would also reduce the bandwidth required to run a Bitcoin node and potentially enable more users to run full nodes. + +===Erlay=== + +[https://arxiv.org/pdf/1905.10518.pdf Erlay] is an example of a high-level transaction relay protocol which employs set reconciliation for bandwidth efficiency. + +Erlay uses both flooding (announcing using INV messages to all peers) and reconciliation to announce transactions. +Flooding is expensive, so Erlay seeks to use it sparingly and in strategic locations - only well-connected publicly reachable nodes flood transactions to other publicly reachable nodes via outbound connections. +Since every unreachable node is directly connected to several reachable nodes, this policy ensures that a transaction is quickly propagated to be within one hop from most of the nodes in the network. + +All transactions not propagated through flooding are propagated through efficient set reconciliation. +To do this, every node keeps a reconciliation set for each peer, in which transactions are placed which would have been announced using INV messages absent this protocol. Every 2 seconds every node chooses a peer from its outbound connections in a predetermined order to reconcile with, resulting in both sides learning the transactions known to the other side. After every reconciliation round, the corresponding reconciliation set is cleared. +A more detailed description of a set reconciliation round and other implementation details can be found in the paper. + +Erlay allows us to: +* save 40% of the bandwidth consumed by a node, given typical network connectivity as of July 2019. +* achieve similar latency +* increase network connectivity for almost no bandwidth or latency cost +* improves privacy as a side-effect + +This document proposes a P2P-layer extension which is required to enable efficient reconciliation-based protocols (like Erlay) for transaction relay. + +==Specification== + +===New data structures=== + +Several new data structures are introduced to the P2P protocol first, to aid with efficient transaction relay. + +====32-bit short transaction IDs==== + +During reconciliation, significantly abbreviated transaction IDs are used of just 32 bits in size. To prevent attackers from constructing sets of transactions that cause network-wide collisions, the short ID computation is salted on a per-link basis using 64 bits of entropy contributed by both communication partners. + +Short IDs are computed as follows: +* Let ''salt1'' and ''salt2'' be the entropy contributed by both sides; see the "sendrecon" message further for details how they are exchanged. +* Sort the two salts such that ''salt1 ≤ salt2'' (which side sent what doesn't matter). +* Compute ''h = SHA256("Tx Relay Salting" || salt1 || salt2)'', where the two salts are encoded in 64-bit little-endian byte order. +* Let ''k0'' be the 64-bit integer obtained by interpreting the first 8 bytes of ''h'' in little-endian byte order. +* Let ''k1'' be the 64-bit integer obtained by interpreting the second 8 bytes of ''h'' in little-endian byte order. +* Let ''s = SipHash-2-4((k0,k1),wtxid)'', where ''wtxid'' is the transaction hash including witness data as defined by BIP141. +* The short ID is equal to ''1 + (s mod 0xFFFFFFFF)''. + +This results in approximately uniformly distributed IDs in the range ''[1..0xFFFFFFFF]'', which is a requirement for using them as elements in 32-bit sketches. See the next paragraph for details. + +====Short transaction ID sketches==== + +Reconciliation-based relay uses [https://www.cs.bu.edu/~reyzin/code/fuzzy.html PinSketch] BCH-based secure sketches as introduced by the [https://www.cs.bu.edu/~reyzin/fuzzy.html Fuzzy Extractors paper]. They are a form of set checksums with the following properties: +* Sketches have a predetermined capacity, and when the number of elements in the set does not exceed the capacity, it is always possible to recover the entire set from the sketch by decoding the sketch. A sketch of nonzero b-bit elements with capacity c can be stored in bc bits. +* A sketch of the [https://en.wikipedia.org/wiki/Symmetric_difference symmetric difference] between the two sets (i.e., all elements that occur in one but not both input sets), can be obtained by combining the sketches of those sets. + +The sketches used here consists of elements of the [https://en.wikipedia.org/wiki/Finite_field finite field] ''GF(232)''. Specifically, we represent finite field elements as polynomials in ''x'' over ''GF(2)'' modulo ''x327 + x3 + x2 + 1''. To map integers to finite field elements, simply treat each bit ''i'' (with value ''2i'') in the integer as the coefficient of ''xi'' in the polynomial representation. For example the integer ''101 = 26 + 25 + 22 + 1'' is mapped to field element ''x6 + x5 + x2 + 1''. These field elements can be added and multiplied together, but the specifics of that are out of scope for this document. + +A short ID sketch with capacity ''c'' consists of a sequence of ''c'' field elements. The first is the sum of all short IDs in the set, the second is the sum of the 3rd powers of all short IDs, the third is the sum of the 5th powers etc., up to the last element with is the sum of the ''(2c-1)''th powers. These elements are then encoded as 32-bit integers in little endian byte order, resulting in a ''4c''-byte serialization. + +The following Python 3.2+ code implements the creation of sketches:
+FIELD_BITS = 32
+FIELD_MODULUS = (1 << FIELD_BITS) + 0b10001101
+
+def mul2(x):
+    """Compute 2*x in GF(2^FIELD_BITS)"""
+    return (x << 1) ^ (FIELD_MODULUS if x.bit_length() >= FIELD_BITS else 0)
+
+def mul(x, y):
+    """Compute x*y in GF(2^FIELD_BITS)"""
+    ret = 0
+    for bit in [(x >> i) & 1 for i in range(x.bit_length())]:
+        ret, y = ret ^ bit * y, mul2(y)
+    return ret
+
+def create_sketch(shortids, capacity):
+    """Compute the bytes of a sketch for given shortids and given capacity."""
+    odd_sums = [0 for _ in range(capacity)]
+    for shortid in shortids:
+        squared = mul(shortid, shortid)
+        for i in range(capacity):
+            odd_sums[i] ^= shortid
+            shortid = mul(shortid, squared)
+    return b''.join(elem.to_bytes(4, 'little') for elem in odd_sums)
+
+ +The [https://github.com/sipa/minisketch/ minisketch] library implements the construction, merging, and decoding of these sketches efficiently. + +====Truncated transaction IDs==== + +For announcing and relaying transaction outside of reconciliation, we need an unambiguous, unsalted way to refer to transactions to deduplicate transaction requests. As we're introducing a new scheme anyway, this is a good opportunity to switch to wtxid-based requests rather than txid-based ones. While using full 256-bit wtxids is possible, this is overkill as they contribute significantly to the total bandwidth as well. Instead, we truncate the wtxid to just their first 128 bits. These are referred to as truncated IDs. + +===Intended Protocol Flow=== + +Set reconciliation primarily consists of the transmission and decoding of a reconciliation set sketch upon request. + +[[File:bip-reconcil/recon_scheme_merged.png|framed|center|Set reconciliation protocol flow]] + +====Bisection==== + +If a node is unable to reconstruct the set difference from the received sketch, the node then makes an additional reconciliation request, similar to the initial one, but this request is applied to only a fraction of possible transactions (e.g., in the range 0x0–0x8). Because of the linearity of sketches, a sketch of a subset of transactions would allow the node to compute a sketch for the remainder, which saves bandwidth. + +[[File:bip-reconcil/bisection.png|framed|300px|center|Bisection]] + +===New messages=== +Several new protocol messages are added: sendrecon, reqreconcil, sketch, reqbisec, reconcildiff, invtx, gettx. This section describes their serialization, contents, and semantics. + +In what follows, all integers are serialized in little-endian byte order. Boolean values are encoded as a single byte that must be 0 or 1 exactly. Arrays are serialized with the CompactSize prefix that encodes their length, as is common in other P2P messages. + +====sendrecon==== +The sendrecon message announces support for the reconciliation protocol. It is expected to be only sent once, and ignored by nodes that don't support it. + +Its payload consists of: +{|class="wikitable" +! Data type !! Name !! Description +|- +| bool || sender || Indicates whether the sender will send "reqreconcil" message +|- +| bool || responder || Indicates whether the sender will respond to "reqreconcil" messages. +|- +| uint32 || version || Must be exactly 1 currently. +|- +| uint64 || salt || The salt used in the short transaction ID computation. +|} + +"reqreconcil" messages can only be sent if the sender has sent a "sendrecon" message with sender=true, and the receiver has sent a "sendrecon" message with responder=true. + +====reqreconcil==== +The reqreconcil message initiates a reconciliation round. + +{|class="wikitable" +! Data type !! Name !! Description +|- +| uint16 || set_size || Size of the sender's reconciliation set, used to estimate set difference. +|- +| uint8 || q || Coefficient used to estimate set difference. +|} + +Upon receipt of a "reqreconcil" message, the receiver: +* Constructs and sends a "sketch" message (see below), with a sketch of capacity computed as ``|set_size - local_set_size| + q * (set_size + local_set_size) + c'', where ''local_set_size'' represents size of the receiver's reconciliation set. +* Makes a snapshot of their current reconciliation set, and clears the set itself. The snapshot is kept until a "reconcildiff" message is received by the node. + +It is suggested to use ''c=1'' to avoid sending empty sketches and reduce the overhead caused by under-estimations. + +Intuitively, ''q'' represents the discrepancy in sets: the closer the sets are, the lower optimal ''q'' is. +As suggested by Erlay, ''q'' should be derived as an optimal ''q'' value for the previous reconciliation with a given peer, once the actual set sizes and set difference are known. Alternatively, ''q=0.1'' should be used as a default value. +For example, if in previous round ''set_size=30'' and ''local_set_size=20'', and the *actual* difference was ''4'', then a node should compute ''q'' as following: +''q=(|30-20| - 1) / (30+20)=0.18'' +The derivation of ''q'' can be changed according to the version of the protocol. + +No new "reqreconcil" message can be sent until a "reconcildiff" message is sent. + +====sketch==== +The sketch message is used to communicate a sketch required to perform set reconciliation. + +{|class="wikitable" +! Data type !! Name !! Description +|- +| byte[] || skdata || The sketch of the sender's reconciliation snapshot +|} + +Upon receipt of a "sketch" message, a node computes the set difference by combining the receiver sketch with a sketch computed locally for a corresponding reconciliation set. If this is the 2nd time for this round a "sketch" message was received, the bisection approach is used, and by combining the new sketch with the previous one, two difference sketches are obtained, one for the first half and one for the second half of the short id range. The receiving node then tries to decode this sketch (or sketches), and based on the result: +* If decoding fails, a "reconcildiff" message is sent with the failure flag set (success=false). If this was the first "sketch" in the round, a "reqbisec" message may be sent instead. +* If decoding succeeds, a "reconcildiff" message is sent with the truncated IDs of all locally known transactions that appear in the decode result, and the short IDs of the unrecognized ones. + +The receiver also makes snapshot of their current reconciliation set, and clears the set itself. The snapshot is kept until a "reconcildiff" message is sent by the node. + +====reqbisec==== +The reqbisec message is used to signal that set reconciliation has failed and an extra sketch is needed to find set difference. + +It has an empty payload. + +Upon receipt of a "reqbisec" message, a node responds to it with a "sketch" message, which contains a sketch of a subset of corresponding reconciliation set snapshot (stored when "reqreconcil" message for the current round was processed) (values in range ''[0..(2^31)]''). + +====reconcildiff==== +The reconcildiff message is used to announce transactions which are found to be missing during set reconciliation on the sender's side. + +{|class="wikitable" +! Data type !! Name !! Description +|- +| uint8 || success || Indicates whether sender of the message succeeded at set difference decoding. +|- +| uint32[] || ask_shortids || The short IDs that the sender did not have. +|} + +Upon receipt a "reconcildiff" message with ''success=1'', a node sends a "invtx" message for the transactions requested by 32-bit IDs (first vector) containing their 128-bit truncated IDs (with parent transactions occuring before their dependencies), and can request announced transactions (second vector) it does not have via a "gettx" message. +Otherwise if ''success=0'', receiver should request bisection via ''reqbisec'' (if failure happened for the first time). +If failure happened for the second time, receiver should announce the transactions from the reconciliation set via an "invtx" message, excluding the transactions announced from the sender. + +The snapshot of the corresponding reconciliation set is cleared by the sender and the receiver of the message. + +The sender should also send their own "invtx" message along with the reconcildiff message to announce transactions which are missing on the receiver's side. + +====invtx==== +The invtx message is used to announce transactions (both along with reconcildiff message and as a response to the reconcildiff message). It is the truncated ID analogue of "inv" (which cannot be used because it has 256-bit elements). + +{|class="wikitable" +! Data type !! Name !! Description +|- +| uint128[] || inv_truncids || The truncated IDs of transactions the sender believes the receiver does not have. +|} + +Upon receipt a "invtx" message, a node requests announced transactions it does not have. +The snapshot of the corresponding reconciliation set is cleared by the sender of the message. + +====gettx==== +The gettx message is used to request transactions by 128-bit truncated IDs. It is the truncated ID analogue of "getdata". + +{|class="wikitable" +! Data type !! Name !! Description +|- +| uint128[] || ask_truncids || The truncated IDs of transactions the sender wants the full transaction data for. +|} + +Upon receipt a "gettx" message, a node sends "tx" messages for the requested transactions. + +==Local state== + +This BIP suggests a stateful protocol and it requires storing several variables at every node to operate properly. + +====Reconciliation sets==== +Every node stores sets of 128-bit truncated IDs per every peer, representing the transactions which would have been sent according to the regular flooding protocol. +Incoming transactions are added to sets when those transactions are received (if they satisfy the policies such as minimum fee set by a peer). +A reconciliation set is moved to the corresponding set snapshot after the transmission of the initial sketch. + +====Reconciliation set snapshot==== +After the transmitting of the initial sketch (either sending or receiving of reconcildiff message), every node should store the snapshot of the current reconciliation set, and clear the set. +This is important to make bisection more stable during the reconciliation round (bisection should be applied to the snapshot). +The snapshot is also used to efficiently lookup the transactions requested by short ID. +The snapshot is cleared after the end of the reconciliation round (sending or receiving of the reconcildiff message). + +====q-coefficient==== +The q value should be stored to make efficient difference estimation. It is shared across peers and changed after every reconciliation. +q-coefficient represents the discrepancy in sets: the closer the sets are, the lower optimal ''q'' is. +In future implementations, q could vary across different peers or become static. + + +==Backward compatibility== + +Older clients remain fully compatible and interoperable after this change. + +Clients which do not implement this protocol remain fully compatible after this change using existing protocols, because transaction announcement reconciliation is used only for peers that negotiate support for it. + +==Rationale== + +====Why using PinSketch for set reconciliation?==== + +To the best of our knowledge, PinSketch is more bandwidth efficient than IBLT, especially for the small differences in sets we expect to operate over. +PinSketch is as bandwidth efficient as CPISync, but PinSketch has quadratic decoding complexity, while CPISync have cubic decoding complexity. This makes PinSketch significantly faster. + +====Why using 32-bit short transaction IDs?==== + +To use Minisketch in practice, transaction IDs should be shortened (ideally, not more than 64 bits per element). +Small number of bits per transaction also allows to save extra bandwidth and make operations over sketches faster. +According to our estimates, 32 bits provides low collision rate in a non-adversarial model (which is enabled by using independent salts per-link). + +====Why using 128-bit short IDs?==== + +To avoid problems caused by the delays in the network, our protocol requires extra round of announcing unsalted transaction IDs. [https://arxiv.org/pdf/1905.10518.pdf Erlay] protocol on top of this work also requires announcing unsalted transaction IDs for flooding. +Both of these measures allow to deduplicate transaction announcements across the peers. +However, using full 256-bit IDs to uniquely identify transactions seems to be an overkill. +128 is the highest power of 2 which provides good enough collision-resistance in an adversarial model, and trivially saves a significant portion of the bandwidth related to these announcements. + +====Why using bisection instead of extending the sketch?==== + +Unlike extended sketches, bisection does not require operating over sketches of higher order. +This allows to avoid the high computational cost caused by quadratic decoding complexity. + +==Implementation== + +TODO + +==Acknowledgments== + +A large fraction of this proposal was done during designing Erlay with Gregory Maxwell, Sasha Fedorova and Ivan Beschastnikh. +We would like to thank Ben Woosley for contributions to the high-level description of the idea. + +==Copyright== + +This document is placed in the public domain. diff --git a/bip-reconcil/bisection.png b/bip-reconcil/bisection.png new file mode 100644 index 0000000..70f37e8 Binary files /dev/null and b/bip-reconcil/bisection.png differ diff --git a/bip-reconcil/minisketch.py b/bip-reconcil/minisketch.py new file mode 100755 index 0000000..f64286f --- /dev/null +++ b/bip-reconcil/minisketch.py @@ -0,0 +1,157 @@ +#!/usr/bin/env python3 + +######## ENCODING and DECODING ######## + +FIELD_BITS = 32 +FIELD_MODULUS = (1 << FIELD_BITS) + 0b10001101 + +def mul2(x): + """Compute 2*x in GF(2^FIELD_BITS)""" + return (x << 1) ^ (FIELD_MODULUS if x.bit_length() >= FIELD_BITS else 0) + +def mul(x, y): + """Compute x*y in GF(2^FIELD_BITS)""" + ret = 0 + for bit in [(x >> i) & 1 for i in range(x.bit_length())]: + ret ^= bit * y + y = mul2(y) + return ret + +######## ENCODING only ######## + +def sketch(shortids, capacity): + """Compute the bytes of a sketch for given shortids and given capacity.""" + odd_sums = [0 for _ in range(capacity)] + for shortid in shortids: + squared = mul(shortid, shortid) + for i in range(capacity): + odd_sums[i] ^= shortid + shortid = mul(shortid, squared) + return b''.join(elem.to_bytes(4, 'little') for elem in odd_sums) + +######## DECODING only ######## + +import random + +def inv(x): + """Compute 1/x in GF(2^FIELD_BITS)""" + t = x + for i in range(FIELD_BITS - 2): + t = mul(mul(t, t), x) + return mul(t, t) + + +def berlekamp_massey(s): + """Given a sequence of LFSR outputs, find the coefficients of the LFSR.""" + C, B, L, m, b = [1], [1], 0, 1, 1 + for n in range(len(s)): + d = s[n] + for i in range(1, L + 1): + d ^= mul(C[i], s[n - i]) + if d == 0: + m += 1 + else: + T = list(C) + while len(C) <= len(B) + m: + C += [0] + t = mul(d, inv(b)) + for i in range(len(B)): + C[i + m] ^= mul(t, B[i]) + if 2 * L <= n: + L, B, b, m = n + 1 - L, T, d, 1 + else: + m += 1 + return C[0:L + 1] + +def poly_monic(p): + """Return the monic multiple of p, or 0 if the input is 0.""" + if len(p) == 0: + return [] + i = inv(p[-1]) + return [mul(v, i) for v in p] + +def poly_divmod(m, p): + """Compute the polynomial quotient p/m, and replace p with p mod m.""" + assert(len(m) > 0 and m[-1] == 1) + div = [0 for _ in range(len(p) - len(m) + 1)] + while len(p) >= len(m): + div[len(p) - len(m)] = p[-1] + for i in range(len(m)): + p[len(p) - len(m) + i] ^= mul(p[-1], m[i]) + assert(p[-1] == 0) + p.pop() + while (len(p) > 0 and p[-1] == 0): + p.pop() + return div + +def poly_gcd(a, b): + """Compute the GCD of a and b (destroys the inputs).""" + if len(a) < len(b): + a, b = b, a + while len(b): + if len(b) == 1: + return [1] + b = poly_monic(b) + poly_divmod(b, a) + a, b = b, a + return a + +def poly_sqr(p): + """Compute the coefficients of the square of polynomial with coefficients p.""" + return [0 if i & 1 else mul(p[i // 2], p[i // 2]) for i in range(2 * len(p))] + +def poly_trace(m, a): + """Compute the coefficients of the trace polynomial of (a*x) mod m.""" + out = [0, a] + for i in range(FIELD_BITS - 1): + out = poly_sqr(out) + while len(out) < 2: + out += [0] + out[1] = a + poly_divmod(m, out) + return out + +def find_roots_inner(p, a): + """Recursive helper function for find_roots (destroys p). a is randomizer.""" + # p must be monic + assert(len(p) > 0 and p[-1] == 1) + # Deal with degree 0 and degree 1 inputs + if len(p) == 1: + return [] + elif len(p) == 2: + return [p[0]] + # Otherwise, split p in left*right using paramater a_vals[0]. + t = poly_monic(poly_trace(p, a)) + left = poly_gcd(list(p), t) + right = poly_divmod(list(left), p) + # Invoke recursion with the remaining a_vals. + ret_right = find_roots_inner(right, mul2(a)) + ret_left = find_roots_inner(left, mul2(a)) + # Concatenate roots + return ret_left + ret_right + +def find_roots(p): + """Find the roots of polynomial with coefficients p.""" + # Compute x^(2^FIELD_BITS)+x mod p in a roundabout way. + t = poly_trace(p, 1) + t2 = poly_sqr(t) + for i in range(len(t)): + t2[i] ^= t[i] + poly_divmod(p, t2) + # If distinct from 0, p is not fully factorizable into non-repeating roots. + if len(t2): + return None + # Invoke the recursive splitting algorithm + return find_roots_inner(list(p), random.randrange(1, 2**32-1)) + +def decode(sketch): + """Recover the shortids from a sketch.""" + odd_sums = [int.from_bytes(sketch[i*4:(i+1)*4], 'little') for i in range(len(sketch) // 4)] + sums = [] + for i in range(len(odd_sums) * 2): + if i & 1: + sums.append(mul(sums[(i-1)//2], sums[(i-1)//2])) + else: + sums.append(odd_sums[(i+1)//2]) + return find_roots(list(reversed(berlekamp_massey(sums)))) + diff --git a/bip-reconcil/recon_scheme_merged.png b/bip-reconcil/recon_scheme_merged.png new file mode 100644 index 0000000..11d1559 Binary files /dev/null and b/bip-reconcil/recon_scheme_merged.png differ -- cgit v1.2.3 From be33606bfe2b9b51a1cf6bd489a1b565b373212b Mon Sep 17 00:00:00 2001 From: MarcoFalke Date: Wed, 25 Sep 2019 08:33:20 -0400 Subject: bip-reconcil: Switch PD "license" to CC0 See https://github.com/bitcoin/bips/blob/master/bip-0002.mediawiki#recommended-licenses --- bip-reconcil.mediawiki | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/bip-reconcil.mediawiki b/bip-reconcil.mediawiki index be34c55..54dc9f0 100644 --- a/bip-reconcil.mediawiki +++ b/bip-reconcil.mediawiki @@ -8,7 +8,7 @@ Status: Draft Type: Standards Track Created: 2010-00-00 - License: PD + License: CC0-1.0
==Abstract== @@ -293,4 +293,4 @@ We would like to thank Ben Woosley for contributions to the high-level descripti ==Copyright== -This document is placed in the public domain. +This document is licensed under the Creative Commons CC0 1.0 Universal license. -- cgit v1.2.3 From 630052355d203bb718f2e2cc8e0e1c6b709d0a68 Mon Sep 17 00:00:00 2001 From: MarcoFalke Date: Wed, 25 Sep 2019 15:47:11 -0400 Subject: Update bip-reconcil.mediawiki Fix mediawiki syntax for italic text --- bip-reconcil.mediawiki | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bip-reconcil.mediawiki b/bip-reconcil.mediawiki index 54dc9f0..e4454ce 100644 --- a/bip-reconcil.mediawiki +++ b/bip-reconcil.mediawiki @@ -153,7 +153,7 @@ The reqreconcil message initiates a reconciliation round. |} Upon receipt of a "reqreconcil" message, the receiver: -* Constructs and sends a "sketch" message (see below), with a sketch of capacity computed as ``|set_size - local_set_size| + q * (set_size + local_set_size) + c'', where ''local_set_size'' represents size of the receiver's reconciliation set. +* Constructs and sends a "sketch" message (see below), with a sketch of capacity computed as ''|set_size - local_set_size| + q * (set_size + local_set_size) + c'', where ''local_set_size'' represents size of the receiver's reconciliation set. * Makes a snapshot of their current reconciliation set, and clears the set itself. The snapshot is kept until a "reconcildiff" message is received by the node. It is suggested to use ''c=1'' to avoid sending empty sketches and reduce the overhead caused by under-estimations. -- cgit v1.2.3 From 0ee067c70e23aafa8ba433179adabf37181ab13e Mon Sep 17 00:00:00 2001 From: Gleb Naumenko Date: Thu, 26 Sep 2019 07:59:06 +0300 Subject: Acknowledge suhas' contributions --- bip-reconcil.mediawiki | 1 + 1 file changed, 1 insertion(+) diff --git a/bip-reconcil.mediawiki b/bip-reconcil.mediawiki index 54dc9f0..cf53776 100644 --- a/bip-reconcil.mediawiki +++ b/bip-reconcil.mediawiki @@ -289,6 +289,7 @@ TODO ==Acknowledgments== A large fraction of this proposal was done during designing Erlay with Gregory Maxwell, Sasha Fedorova and Ivan Beschastnikh. +We would like to thank Suhas Daftuar for contributions to the design and BIP structure. We would like to thank Ben Woosley for contributions to the high-level description of the idea. ==Copyright== -- cgit v1.2.3 From 32af098957cce3845c02d199483adf4e3948eb57 Mon Sep 17 00:00:00 2001 From: Gleb Naumenko Date: Mon, 30 Sep 2019 11:49:54 +0300 Subject: minor fixes Co-authored-by: Rusty Russel --- bip-reconcil.mediawiki | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/bip-reconcil.mediawiki b/bip-reconcil.mediawiki index cf53776..61d3968 100644 --- a/bip-reconcil.mediawiki +++ b/bip-reconcil.mediawiki @@ -134,7 +134,7 @@ Its payload consists of: |- | bool || responder || Indicates whether the sender will respond to "reqreconcil" messages. |- -| uint32 || version || Must be exactly 1 currently. +| uint32 || version || Sender must set this to 1 currently, otherwise receiver should ignore the message. |- | uint64 || salt || The salt used in the short transaction ID computation. |} @@ -149,7 +149,7 @@ The reqreconcil message initiates a reconciliation round. |- | uint16 || set_size || Size of the sender's reconciliation set, used to estimate set difference. |- -| uint8 || q || Coefficient used to estimate set difference. +| uint8 || q || Coefficient used to estimate set difference. Multiplied by PRECISION=2^6 and rounded up by the sender and divided by PRECISION by the receiver. |} Upon receipt of a "reqreconcil" message, the receiver: @@ -235,7 +235,7 @@ Upon receipt a "gettx" message, a node sends "tx" messages for the requested tra This BIP suggests a stateful protocol and it requires storing several variables at every node to operate properly. ====Reconciliation sets==== -Every node stores sets of 128-bit truncated IDs per every peer, representing the transactions which would have been sent according to the regular flooding protocol. +Every node stores a set of 128-bit truncated IDs for every peer which supports transaction reconciliation, representing the transactions which would have been sent according to the regular flooding protocol. Incoming transactions are added to sets when those transactions are received (if they satisfy the policies such as minimum fee set by a peer). A reconciliation set is moved to the corresponding set snapshot after the transmission of the initial sketch. @@ -261,7 +261,7 @@ Clients which do not implement this protocol remain fully compatible after this ====Why using PinSketch for set reconciliation?==== -To the best of our knowledge, PinSketch is more bandwidth efficient than IBLT, especially for the small differences in sets we expect to operate over. +PinSketch is more bandwidth efficient than IBLT, especially for the small differences in sets we expect to operate over. PinSketch is as bandwidth efficient as CPISync, but PinSketch has quadratic decoding complexity, while CPISync have cubic decoding complexity. This makes PinSketch significantly faster. ====Why using 32-bit short transaction IDs?==== -- cgit v1.2.3 From bc3a81e69819364af5c76f73762da2c95d6069ab Mon Sep 17 00:00:00 2001 From: Andrew Chow Date: Wed, 2 Oct 2019 15:09:51 -0400 Subject: Specify proprietary use type --- bip-0174.mediawiki | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/bip-0174.mediawiki b/bip-0174.mediawiki index d30b05d..8c9f010 100644 --- a/bip-0174.mediawiki +++ b/bip-0174.mediawiki @@ -132,6 +132,12 @@ The currently defined global types are as follows: ** Value: The 32-bit little endian unsigned integer representing the version number of this PSBT. If ommitted, the version number is 0. *** {32-bit int} +* Type: Version Number PSBT_GLOBAL_PROPRIETARY = 0xFC +** Key: Variable length identifier prefix, followed by a subtype, followed by the key data itself. +*** {0xFC}||{subtype}|{key data} +** Value: Any value data as defined by the proprietary type user. +*** + The currently defined per-input types are defined as follows: * Type: Non-Witness UTXO PSBT_IN_NON_WITNESS_UTXO = 0x00 @@ -194,6 +200,12 @@ The currently defined per-input types are defined as follows: ** Value: The UTF-8 encoded commitment message string for the proof-of-reserves. See [[bip-0127.mediawiki|BIP 127]] for more information. *** {porCommitment} +* Type: Version Number PSBT_INPUT_PROPRIETARY = 0xFC +** Key: Variable length identifier prefix, followed by a subtype, followed by the key data itself. +*** {0xFC}||{subtype}|{key data} +** Value: Any value data as defined by the proprietary type user. +*** + The currently defined per-output '''Why do we need per-output data?''' Per-output data allows signers to verify that the outputs are going to the intended recipient. The output data can also be use by signers to determine which outputs are change outputs and verify that the change is returning to the correct place. types are defined as follows: @@ -216,6 +228,12 @@ determine which outputs are change outputs and verify that the change is returni ** Value: The master key fingerprint concatenated with the derivation path of the public key. The derivation path is represented as 32 bit unsigned integer indexes concatenated with each other. This must omit the index of the master key. Public keys are those needed to spend this output. *** {master key fingerprint}|{32-bit int}|...|{32-bit int} +* Type: Version Number PSBT_OUTPUT_PROPRIETARY = 0xFC +** Key: Variable length identifier prefix, followed by a subtype, followed by the key data itself. +*** {0xFC}||{subtype}|{key data} +** Value: Any value data as defined by the proprietary type user. +*** + The transaction format is specified as follows: @@ -291,6 +309,23 @@ whichever value it wishes.'''Why can the values be arbitrarily chosen?''' W valid or invalid. If the values are invalid, a signer would simply produce an invalid signature and the final transaction itself would be invalid. If the values are valid, then it does not matter which is chosen as either way the transaction is still valid. +===Proprietary Use Type=== + +For all global, per-input, and per-output maps, the types 0xFC is reserved for proprietary use. +The proprietary use type requires keys that follow the type with a variable length string identifer, then a subtype. + +The identifier can be any variable length string that software can use to identify whether the particular data in the proprietary type can be used by it. +It can also be the empty string and just be a single 0x00 byte although this is not recommended. + +The subtype is defined by the proprietary type user and can mean whatever they want it to mean. +The subtype must also be a compact size unsigned integer in the same form as the normal types. +The key data and value data are defined by the proprietary type user. + +The proprietary use types is for private use by individuals and organizations who wish to use PSBT in their processes. +It is useful when there are additional data that they need attached to a PSBT but such data are not useful or available for the general public. +The proprietary use type is not to be used by any public specification and there is no expectation that any publicly available software be able to understand any specific meanings of it and the subtypes. +This type must be used for internal processes only. + ==Responsibilities== Using the transaction format involves many different responsibilities. Multiple responsibilities can be handled by a single entity, but each responsibility is specialized in what it should be capable of doing. -- cgit v1.2.3 From d2277115ce8194ee3d92ae02ddaea60034c06d39 Mon Sep 17 00:00:00 2001 From: MarcoFalke Date: Thu, 10 Oct 2019 17:03:48 -0400 Subject: travis: Remove unused sudo:false Builds in sudo-disabled docker containers are no longer available as of last year and all builds happen on sudo enabled vms. https://blog.travis-ci.com/2018-11-19-required-linux-infrastructure-migration#timeline---its-happening-fast --- .travis.yml | 1 - 1 file changed, 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index e463ab6..70d339a 100644 --- a/.travis.yml +++ b/.travis.yml @@ -1,6 +1,5 @@ os: linux language: generic -sudo: false script: - scripts/link-format-chk.sh - scripts/buildtable.pl >/tmp/table.mediawiki || exit 1 -- cgit v1.2.3 From 09d3d83d7e6d59208939b4432c48f4da851060e8 Mon Sep 17 00:00:00 2001 From: MarcoFalke Date: Thu, 10 Oct 2019 16:53:11 -0400 Subject: BIP 103: Withdrawn --- README.mediawiki | 4 ++-- bip-0103.mediawiki | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/README.mediawiki b/README.mediawiki index de98f05..8bb77e4 100644 --- a/README.mediawiki +++ b/README.mediawiki @@ -469,13 +469,13 @@ Those proposing changes should consider that ultimately consent may rest with th | Jeff Garzik | Standard | Rejected -|- +|- style="background-color: #ffcfcf" | [[bip-0103.mediawiki|103]] | Consensus (hard fork) | Block size following technological growth | Pieter Wuille | Standard -| Draft +| Withdrawn |- | [[bip-0104.mediawiki|104]] | Consensus (hard fork) diff --git a/bip-0103.mediawiki b/bip-0103.mediawiki index bc06000..3a8bab5 100644 --- a/bip-0103.mediawiki +++ b/bip-0103.mediawiki @@ -5,7 +5,7 @@ Author: Pieter Wuille Comments-Summary: No comments yet. Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0103 - Status: Draft + Status: Withdrawn Type: Standards Track Created: 2015-07-21 License: BSD-2-Clause -- cgit v1.2.3 From 898559fd055689e30c7df524da9489ef46159fd6 Mon Sep 17 00:00:00 2001 From: Olaoluwa Osuntokun Date: Thu, 24 Oct 2019 19:27:12 -0700 Subject: BIP-0157: increase max getcfilters request to 1k blocks In this commit, we effectively revert #699 by allow clients to request filter for up to 1k consecutive blocks. Testing in the field has shown that applications are able to reduce perceived latency from syncing to full functionality after an app has been offline for several days by batching requests for filters. A value of 100 would mean each additional day behind adds an additional round trip, resulting in 10s of seconds of lag after just a few days of being offline. A value of ~1k allows implementations to catch up with roughly a week's worth of filters in a single round trip. --- bip-0157.mediawiki | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bip-0157.mediawiki b/bip-0157.mediawiki index 4a47706..bb864aa 100644 --- a/bip-0157.mediawiki +++ b/bip-0157.mediawiki @@ -168,7 +168,7 @@ fields: # Nodes SHOULD NOT send getcfilters unless the peer has signaled support for this filter type. Nodes receiving getcfilters with an unsupported filter type SHOULD NOT respond. # StopHash MUST be known to belong to a block accepted by the receiving peer. This is the case if the peer had previously sent a headers or inv message with that block or any descendents. A node that receives getcfilters with an unknown StopHash SHOULD NOT respond. -# The height of the block with hash StopHash MUST be greater than or equal to StartHeight, and the difference MUST be strictly less than 100. +# The height of the block with hash StopHash MUST be greater than or equal to StartHeight, and the difference MUST be strictly less than 1000. # The receiving node MUST respond to valid requests by sending one cfilter message for each block in the requested range, sequentially in order by block height. ==== cfilter ==== -- cgit v1.2.3 From aae7384c46407ac6b63598956b4d1fba27e23c88 Mon Sep 17 00:00:00 2001 From: User Date: Mon, 4 Nov 2019 13:24:24 -0500 Subject: Assigned a number, separated lines for authors, added License-Code field. --- bip-0330/bisection.png | Bin 0 -> 60787 bytes bip-0330/minisketch.py | 157 ++++++++++++++++++ bip-0330/recon_scheme_merged.png | Bin 0 -> 113169 bytes bip-330.mediawiki | 299 +++++++++++++++++++++++++++++++++++ bip-reconcil.mediawiki | 297 ---------------------------------- bip-reconcil/bisection.png | Bin 60787 -> 0 bytes bip-reconcil/minisketch.py | 157 ------------------ bip-reconcil/recon_scheme_merged.png | Bin 113169 -> 0 bytes 8 files changed, 456 insertions(+), 454 deletions(-) create mode 100644 bip-0330/bisection.png create mode 100755 bip-0330/minisketch.py create mode 100644 bip-0330/recon_scheme_merged.png create mode 100644 bip-330.mediawiki delete mode 100644 bip-reconcil.mediawiki delete mode 100644 bip-reconcil/bisection.png delete mode 100755 bip-reconcil/minisketch.py delete mode 100644 bip-reconcil/recon_scheme_merged.png diff --git a/bip-0330/bisection.png b/bip-0330/bisection.png new file mode 100644 index 0000000..70f37e8 Binary files /dev/null and b/bip-0330/bisection.png differ diff --git a/bip-0330/minisketch.py b/bip-0330/minisketch.py new file mode 100755 index 0000000..f64286f --- /dev/null +++ b/bip-0330/minisketch.py @@ -0,0 +1,157 @@ +#!/usr/bin/env python3 + +######## ENCODING and DECODING ######## + +FIELD_BITS = 32 +FIELD_MODULUS = (1 << FIELD_BITS) + 0b10001101 + +def mul2(x): + """Compute 2*x in GF(2^FIELD_BITS)""" + return (x << 1) ^ (FIELD_MODULUS if x.bit_length() >= FIELD_BITS else 0) + +def mul(x, y): + """Compute x*y in GF(2^FIELD_BITS)""" + ret = 0 + for bit in [(x >> i) & 1 for i in range(x.bit_length())]: + ret ^= bit * y + y = mul2(y) + return ret + +######## ENCODING only ######## + +def sketch(shortids, capacity): + """Compute the bytes of a sketch for given shortids and given capacity.""" + odd_sums = [0 for _ in range(capacity)] + for shortid in shortids: + squared = mul(shortid, shortid) + for i in range(capacity): + odd_sums[i] ^= shortid + shortid = mul(shortid, squared) + return b''.join(elem.to_bytes(4, 'little') for elem in odd_sums) + +######## DECODING only ######## + +import random + +def inv(x): + """Compute 1/x in GF(2^FIELD_BITS)""" + t = x + for i in range(FIELD_BITS - 2): + t = mul(mul(t, t), x) + return mul(t, t) + + +def berlekamp_massey(s): + """Given a sequence of LFSR outputs, find the coefficients of the LFSR.""" + C, B, L, m, b = [1], [1], 0, 1, 1 + for n in range(len(s)): + d = s[n] + for i in range(1, L + 1): + d ^= mul(C[i], s[n - i]) + if d == 0: + m += 1 + else: + T = list(C) + while len(C) <= len(B) + m: + C += [0] + t = mul(d, inv(b)) + for i in range(len(B)): + C[i + m] ^= mul(t, B[i]) + if 2 * L <= n: + L, B, b, m = n + 1 - L, T, d, 1 + else: + m += 1 + return C[0:L + 1] + +def poly_monic(p): + """Return the monic multiple of p, or 0 if the input is 0.""" + if len(p) == 0: + return [] + i = inv(p[-1]) + return [mul(v, i) for v in p] + +def poly_divmod(m, p): + """Compute the polynomial quotient p/m, and replace p with p mod m.""" + assert(len(m) > 0 and m[-1] == 1) + div = [0 for _ in range(len(p) - len(m) + 1)] + while len(p) >= len(m): + div[len(p) - len(m)] = p[-1] + for i in range(len(m)): + p[len(p) - len(m) + i] ^= mul(p[-1], m[i]) + assert(p[-1] == 0) + p.pop() + while (len(p) > 0 and p[-1] == 0): + p.pop() + return div + +def poly_gcd(a, b): + """Compute the GCD of a and b (destroys the inputs).""" + if len(a) < len(b): + a, b = b, a + while len(b): + if len(b) == 1: + return [1] + b = poly_monic(b) + poly_divmod(b, a) + a, b = b, a + return a + +def poly_sqr(p): + """Compute the coefficients of the square of polynomial with coefficients p.""" + return [0 if i & 1 else mul(p[i // 2], p[i // 2]) for i in range(2 * len(p))] + +def poly_trace(m, a): + """Compute the coefficients of the trace polynomial of (a*x) mod m.""" + out = [0, a] + for i in range(FIELD_BITS - 1): + out = poly_sqr(out) + while len(out) < 2: + out += [0] + out[1] = a + poly_divmod(m, out) + return out + +def find_roots_inner(p, a): + """Recursive helper function for find_roots (destroys p). a is randomizer.""" + # p must be monic + assert(len(p) > 0 and p[-1] == 1) + # Deal with degree 0 and degree 1 inputs + if len(p) == 1: + return [] + elif len(p) == 2: + return [p[0]] + # Otherwise, split p in left*right using paramater a_vals[0]. + t = poly_monic(poly_trace(p, a)) + left = poly_gcd(list(p), t) + right = poly_divmod(list(left), p) + # Invoke recursion with the remaining a_vals. + ret_right = find_roots_inner(right, mul2(a)) + ret_left = find_roots_inner(left, mul2(a)) + # Concatenate roots + return ret_left + ret_right + +def find_roots(p): + """Find the roots of polynomial with coefficients p.""" + # Compute x^(2^FIELD_BITS)+x mod p in a roundabout way. + t = poly_trace(p, 1) + t2 = poly_sqr(t) + for i in range(len(t)): + t2[i] ^= t[i] + poly_divmod(p, t2) + # If distinct from 0, p is not fully factorizable into non-repeating roots. + if len(t2): + return None + # Invoke the recursive splitting algorithm + return find_roots_inner(list(p), random.randrange(1, 2**32-1)) + +def decode(sketch): + """Recover the shortids from a sketch.""" + odd_sums = [int.from_bytes(sketch[i*4:(i+1)*4], 'little') for i in range(len(sketch) // 4)] + sums = [] + for i in range(len(odd_sums) * 2): + if i & 1: + sums.append(mul(sums[(i-1)//2], sums[(i-1)//2])) + else: + sums.append(odd_sums[(i+1)//2]) + return find_roots(list(reversed(berlekamp_massey(sums)))) + diff --git a/bip-0330/recon_scheme_merged.png b/bip-0330/recon_scheme_merged.png new file mode 100644 index 0000000..11d1559 Binary files /dev/null and b/bip-0330/recon_scheme_merged.png differ diff --git a/bip-330.mediawiki b/bip-330.mediawiki new file mode 100644 index 0000000..83b3558 --- /dev/null +++ b/bip-330.mediawiki @@ -0,0 +1,299 @@ +
+  BIP: 330
+  Layer: Peer Services
+  Title: Transaction announcements reconciliation
+  Author: Gleb Naumenko 
+          Pieter Wuille 
+  Comments-Summary: ???
+  Comments-URI: ???
+  Status: Draft
+  Type: Standards Track
+  Created: 2010-00-00
+  License: CC0-1.0
+  License-Code: MIT
+
+ +==Abstract== + +This document specifies a P2P protocol extension for reconciliation of transaction announcements between 2 nodes, which is a building block for efficient transaction relay protocols (e.g., [https://arxiv.org/pdf/1905.10518.pdf Erlay]). This is a step towards increasing the connectivity of the network for almost no bandwidth cost. + +==Motivation== + +Currently in the Bitcoin network, every 32-byte transaction ID is announced in at least one direction between every pair of connected peers, via INV messages. This results in high cost of announcing transactions: ''O(nodes * connections_per_node)''. + +A reconciliation-based protocol which uses the technique suggested in this document can have better scaling properties than INV-based flooding. + +Increasing the connectivity of the network makes the network more robust to partitioning attacks; thus, improving the bandwidth scaling of transaction relay to ''O(nodes)'' (and without a high constant overhead) would allow us to improve the security of the network by increasing connectivity. It would also reduce the bandwidth required to run a Bitcoin node and potentially enable more users to run full nodes. + +===Erlay=== + +[https://arxiv.org/pdf/1905.10518.pdf Erlay] is an example of a high-level transaction relay protocol which employs set reconciliation for bandwidth efficiency. + +Erlay uses both flooding (announcing using INV messages to all peers) and reconciliation to announce transactions. +Flooding is expensive, so Erlay seeks to use it sparingly and in strategic locations - only well-connected publicly reachable nodes flood transactions to other publicly reachable nodes via outbound connections. +Since every unreachable node is directly connected to several reachable nodes, this policy ensures that a transaction is quickly propagated to be within one hop from most of the nodes in the network. + +All transactions not propagated through flooding are propagated through efficient set reconciliation. +To do this, every node keeps a reconciliation set for each peer, in which transactions are placed which would have been announced using INV messages absent this protocol. Every 2 seconds every node chooses a peer from its outbound connections in a predetermined order to reconcile with, resulting in both sides learning the transactions known to the other side. After every reconciliation round, the corresponding reconciliation set is cleared. +A more detailed description of a set reconciliation round and other implementation details can be found in the paper. + +Erlay allows us to: +* save 40% of the bandwidth consumed by a node, given typical network connectivity as of July 2019. +* achieve similar latency +* increase network connectivity for almost no bandwidth or latency cost +* improves privacy as a side-effect + +This document proposes a P2P-layer extension which is required to enable efficient reconciliation-based protocols (like Erlay) for transaction relay. + +==Specification== + +===New data structures=== + +Several new data structures are introduced to the P2P protocol first, to aid with efficient transaction relay. + +====32-bit short transaction IDs==== + +During reconciliation, significantly abbreviated transaction IDs are used of just 32 bits in size. To prevent attackers from constructing sets of transactions that cause network-wide collisions, the short ID computation is salted on a per-link basis using 64 bits of entropy contributed by both communication partners. + +Short IDs are computed as follows: +* Let ''salt1'' and ''salt2'' be the entropy contributed by both sides; see the "sendrecon" message further for details how they are exchanged. +* Sort the two salts such that ''salt1 ≤ salt2'' (which side sent what doesn't matter). +* Compute ''h = SHA256("Tx Relay Salting" || salt1 || salt2)'', where the two salts are encoded in 64-bit little-endian byte order. +* Let ''k0'' be the 64-bit integer obtained by interpreting the first 8 bytes of ''h'' in little-endian byte order. +* Let ''k1'' be the 64-bit integer obtained by interpreting the second 8 bytes of ''h'' in little-endian byte order. +* Let ''s = SipHash-2-4((k0,k1),wtxid)'', where ''wtxid'' is the transaction hash including witness data as defined by BIP141. +* The short ID is equal to ''1 + (s mod 0xFFFFFFFF)''. + +This results in approximately uniformly distributed IDs in the range ''[1..0xFFFFFFFF]'', which is a requirement for using them as elements in 32-bit sketches. See the next paragraph for details. + +====Short transaction ID sketches==== + +Reconciliation-based relay uses [https://www.cs.bu.edu/~reyzin/code/fuzzy.html PinSketch] BCH-based secure sketches as introduced by the [https://www.cs.bu.edu/~reyzin/fuzzy.html Fuzzy Extractors paper]. They are a form of set checksums with the following properties: +* Sketches have a predetermined capacity, and when the number of elements in the set does not exceed the capacity, it is always possible to recover the entire set from the sketch by decoding the sketch. A sketch of nonzero b-bit elements with capacity c can be stored in bc bits. +* A sketch of the [https://en.wikipedia.org/wiki/Symmetric_difference symmetric difference] between the two sets (i.e., all elements that occur in one but not both input sets), can be obtained by combining the sketches of those sets. + +The sketches used here consists of elements of the [https://en.wikipedia.org/wiki/Finite_field finite field] ''GF(232)''. Specifically, we represent finite field elements as polynomials in ''x'' over ''GF(2)'' modulo ''x327 + x3 + x2 + 1''. To map integers to finite field elements, simply treat each bit ''i'' (with value ''2i'') in the integer as the coefficient of ''xi'' in the polynomial representation. For example the integer ''101 = 26 + 25 + 22 + 1'' is mapped to field element ''x6 + x5 + x2 + 1''. These field elements can be added and multiplied together, but the specifics of that are out of scope for this document. + +A short ID sketch with capacity ''c'' consists of a sequence of ''c'' field elements. The first is the sum of all short IDs in the set, the second is the sum of the 3rd powers of all short IDs, the third is the sum of the 5th powers etc., up to the last element with is the sum of the ''(2c-1)''th powers. These elements are then encoded as 32-bit integers in little endian byte order, resulting in a ''4c''-byte serialization. + +The following Python 3.2+ code implements the creation of sketches:
+FIELD_BITS = 32
+FIELD_MODULUS = (1 << FIELD_BITS) + 0b10001101
+
+def mul2(x):
+    """Compute 2*x in GF(2^FIELD_BITS)"""
+    return (x << 1) ^ (FIELD_MODULUS if x.bit_length() >= FIELD_BITS else 0)
+
+def mul(x, y):
+    """Compute x*y in GF(2^FIELD_BITS)"""
+    ret = 0
+    for bit in [(x >> i) & 1 for i in range(x.bit_length())]:
+        ret, y = ret ^ bit * y, mul2(y)
+    return ret
+
+def create_sketch(shortids, capacity):
+    """Compute the bytes of a sketch for given shortids and given capacity."""
+    odd_sums = [0 for _ in range(capacity)]
+    for shortid in shortids:
+        squared = mul(shortid, shortid)
+        for i in range(capacity):
+            odd_sums[i] ^= shortid
+            shortid = mul(shortid, squared)
+    return b''.join(elem.to_bytes(4, 'little') for elem in odd_sums)
+
+ +The [https://github.com/sipa/minisketch/ minisketch] library implements the construction, merging, and decoding of these sketches efficiently. + +====Truncated transaction IDs==== + +For announcing and relaying transaction outside of reconciliation, we need an unambiguous, unsalted way to refer to transactions to deduplicate transaction requests. As we're introducing a new scheme anyway, this is a good opportunity to switch to wtxid-based requests rather than txid-based ones. While using full 256-bit wtxids is possible, this is overkill as they contribute significantly to the total bandwidth as well. Instead, we truncate the wtxid to just their first 128 bits. These are referred to as truncated IDs. + +===Intended Protocol Flow=== + +Set reconciliation primarily consists of the transmission and decoding of a reconciliation set sketch upon request. + +[[File:bip-330/recon_scheme_merged.png|framed|center|Set reconciliation protocol flow]] + +====Bisection==== + +If a node is unable to reconstruct the set difference from the received sketch, the node then makes an additional reconciliation request, similar to the initial one, but this request is applied to only a fraction of possible transactions (e.g., in the range 0x0–0x8). Because of the linearity of sketches, a sketch of a subset of transactions would allow the node to compute a sketch for the remainder, which saves bandwidth. + +[[File:bip-330/bisection.png|framed|300px|center|Bisection]] + +===New messages=== +Several new protocol messages are added: sendrecon, reqreconcil, sketch, reqbisec, reconcildiff, invtx, gettx. This section describes their serialization, contents, and semantics. + +In what follows, all integers are serialized in little-endian byte order. Boolean values are encoded as a single byte that must be 0 or 1 exactly. Arrays are serialized with the CompactSize prefix that encodes their length, as is common in other P2P messages. + +====sendrecon==== +The sendrecon message announces support for the reconciliation protocol. It is expected to be only sent once, and ignored by nodes that don't support it. + +Its payload consists of: +{|class="wikitable" +! Data type !! Name !! Description +|- +| bool || sender || Indicates whether the sender will send "reqreconcil" message +|- +| bool || responder || Indicates whether the sender will respond to "reqreconcil" messages. +|- +| uint32 || version || Sender must set this to 1 currently, otherwise receiver should ignore the message. +|- +| uint64 || salt || The salt used in the short transaction ID computation. +|} + +"reqreconcil" messages can only be sent if the sender has sent a "sendrecon" message with sender=true, and the receiver has sent a "sendrecon" message with responder=true. + +====reqreconcil==== +The reqreconcil message initiates a reconciliation round. + +{|class="wikitable" +! Data type !! Name !! Description +|- +| uint16 || set_size || Size of the sender's reconciliation set, used to estimate set difference. +|- +| uint8 || q || Coefficient used to estimate set difference. Multiplied by PRECISION=2^6 and rounded up by the sender and divided by PRECISION by the receiver. +|} + +Upon receipt of a "reqreconcil" message, the receiver: +* Constructs and sends a "sketch" message (see below), with a sketch of capacity computed as ``|set_size - local_set_size| + q * (set_size + local_set_size) + c'', where ''local_set_size'' represents size of the receiver's reconciliation set. +* Makes a snapshot of their current reconciliation set, and clears the set itself. The snapshot is kept until a "reconcildiff" message is received by the node. + +It is suggested to use ''c=1'' to avoid sending empty sketches and reduce the overhead caused by under-estimations. + +Intuitively, ''q'' represents the discrepancy in sets: the closer the sets are, the lower optimal ''q'' is. +As suggested by Erlay, ''q'' should be derived as an optimal ''q'' value for the previous reconciliation with a given peer, once the actual set sizes and set difference are known. Alternatively, ''q=0.1'' should be used as a default value. +For example, if in previous round ''set_size=30'' and ''local_set_size=20'', and the *actual* difference was ''4'', then a node should compute ''q'' as following: +''q=(|30-20| - 1) / (30+20)=0.18'' +The derivation of ''q'' can be changed according to the version of the protocol. + +No new "reqreconcil" message can be sent until a "reconcildiff" message is sent. + +====sketch==== +The sketch message is used to communicate a sketch required to perform set reconciliation. + +{|class="wikitable" +! Data type !! Name !! Description +|- +| byte[] || skdata || The sketch of the sender's reconciliation snapshot +|} + +Upon receipt of a "sketch" message, a node computes the set difference by combining the receiver sketch with a sketch computed locally for a corresponding reconciliation set. If this is the 2nd time for this round a "sketch" message was received, the bisection approach is used, and by combining the new sketch with the previous one, two difference sketches are obtained, one for the first half and one for the second half of the short id range. The receiving node then tries to decode this sketch (or sketches), and based on the result: +* If decoding fails, a "reconcildiff" message is sent with the failure flag set (success=false). If this was the first "sketch" in the round, a "reqbisec" message may be sent instead. +* If decoding succeeds, a "reconcildiff" message is sent with the truncated IDs of all locally known transactions that appear in the decode result, and the short IDs of the unrecognized ones. + +The receiver also makes snapshot of their current reconciliation set, and clears the set itself. The snapshot is kept until a "reconcildiff" message is sent by the node. + +====reqbisec==== +The reqbisec message is used to signal that set reconciliation has failed and an extra sketch is needed to find set difference. + +It has an empty payload. + +Upon receipt of a "reqbisec" message, a node responds to it with a "sketch" message, which contains a sketch of a subset of corresponding reconciliation set snapshot (stored when "reqreconcil" message for the current round was processed) (values in range ''[0..(2^31)]''). + +====reconcildiff==== +The reconcildiff message is used to announce transactions which are found to be missing during set reconciliation on the sender's side. + +{|class="wikitable" +! Data type !! Name !! Description +|- +| uint8 || success || Indicates whether sender of the message succeeded at set difference decoding. +|- +| uint32[] || ask_shortids || The short IDs that the sender did not have. +|} + +Upon receipt a "reconcildiff" message with ''success=1'', a node sends a "invtx" message for the transactions requested by 32-bit IDs (first vector) containing their 128-bit truncated IDs (with parent transactions occuring before their dependencies), and can request announced transactions (second vector) it does not have via a "gettx" message. +Otherwise if ''success=0'', receiver should request bisection via ''reqbisec'' (if failure happened for the first time). +If failure happened for the second time, receiver should announce the transactions from the reconciliation set via an "invtx" message, excluding the transactions announced from the sender. + +The snapshot of the corresponding reconciliation set is cleared by the sender and the receiver of the message. + +The sender should also send their own "invtx" message along with the reconcildiff message to announce transactions which are missing on the receiver's side. + +====invtx==== +The invtx message is used to announce transactions (both along with reconcildiff message and as a response to the reconcildiff message). It is the truncated ID analogue of "inv" (which cannot be used because it has 256-bit elements). + +{|class="wikitable" +! Data type !! Name !! Description +|- +| uint128[] || inv_truncids || The truncated IDs of transactions the sender believes the receiver does not have. +|} + +Upon receipt a "invtx" message, a node requests announced transactions it does not have. +The snapshot of the corresponding reconciliation set is cleared by the sender of the message. + +====gettx==== +The gettx message is used to request transactions by 128-bit truncated IDs. It is the truncated ID analogue of "getdata". + +{|class="wikitable" +! Data type !! Name !! Description +|- +| uint128[] || ask_truncids || The truncated IDs of transactions the sender wants the full transaction data for. +|} + +Upon receipt a "gettx" message, a node sends "tx" messages for the requested transactions. + +==Local state== + +This BIP suggests a stateful protocol and it requires storing several variables at every node to operate properly. + +====Reconciliation sets==== +Every node stores a set of 128-bit truncated IDs for every peer which supports transaction reconciliation, representing the transactions which would have been sent according to the regular flooding protocol. +Incoming transactions are added to sets when those transactions are received (if they satisfy the policies such as minimum fee set by a peer). +A reconciliation set is moved to the corresponding set snapshot after the transmission of the initial sketch. + +====Reconciliation set snapshot==== +After the transmitting of the initial sketch (either sending or receiving of reconcildiff message), every node should store the snapshot of the current reconciliation set, and clear the set. +This is important to make bisection more stable during the reconciliation round (bisection should be applied to the snapshot). +The snapshot is also used to efficiently lookup the transactions requested by short ID. +The snapshot is cleared after the end of the reconciliation round (sending or receiving of the reconcildiff message). + +====q-coefficient==== +The q value should be stored to make efficient difference estimation. It is shared across peers and changed after every reconciliation. +q-coefficient represents the discrepancy in sets: the closer the sets are, the lower optimal ''q'' is. +In future implementations, q could vary across different peers or become static. + + +==Backward compatibility== + +Older clients remain fully compatible and interoperable after this change. + +Clients which do not implement this protocol remain fully compatible after this change using existing protocols, because transaction announcement reconciliation is used only for peers that negotiate support for it. + +==Rationale== + +====Why using PinSketch for set reconciliation?==== + +PinSketch is more bandwidth efficient than IBLT, especially for the small differences in sets we expect to operate over. +PinSketch is as bandwidth efficient as CPISync, but PinSketch has quadratic decoding complexity, while CPISync have cubic decoding complexity. This makes PinSketch significantly faster. + +====Why using 32-bit short transaction IDs?==== + +To use Minisketch in practice, transaction IDs should be shortened (ideally, not more than 64 bits per element). +Small number of bits per transaction also allows to save extra bandwidth and make operations over sketches faster. +According to our estimates, 32 bits provides low collision rate in a non-adversarial model (which is enabled by using independent salts per-link). + +====Why using 128-bit short IDs?==== + +To avoid problems caused by the delays in the network, our protocol requires extra round of announcing unsalted transaction IDs. [https://arxiv.org/pdf/1905.10518.pdf Erlay] protocol on top of this work also requires announcing unsalted transaction IDs for flooding. +Both of these measures allow to deduplicate transaction announcements across the peers. +However, using full 256-bit IDs to uniquely identify transactions seems to be an overkill. +128 is the highest power of 2 which provides good enough collision-resistance in an adversarial model, and trivially saves a significant portion of the bandwidth related to these announcements. + +====Why using bisection instead of extending the sketch?==== + +Unlike extended sketches, bisection does not require operating over sketches of higher order. +This allows to avoid the high computational cost caused by quadratic decoding complexity. + +==Implementation== + +TODO + +==Acknowledgments== + +A large fraction of this proposal was done during designing Erlay with Gregory Maxwell, Sasha Fedorova and Ivan Beschastnikh. +We would like to thank Suhas Daftuar for contributions to the design and BIP structure. +We would like to thank Ben Woosley for contributions to the high-level description of the idea. + +==Copyright== + +This document is licensed under the Creative Commons CC0 1.0 Universal license. diff --git a/bip-reconcil.mediawiki b/bip-reconcil.mediawiki deleted file mode 100644 index 61d3968..0000000 --- a/bip-reconcil.mediawiki +++ /dev/null @@ -1,297 +0,0 @@ -
-  BIP: ???
-  Layer: Peer Services
-  Title: Transaction announcements reconciliation
-  Author: Gleb Naumenko , Pieter Wuille 
-  Comments-Summary: ???
-  Comments-URI: ???
-  Status: Draft
-  Type: Standards Track
-  Created: 2010-00-00
-  License: CC0-1.0
-
- -==Abstract== - -This document specifies a P2P protocol extension for reconciliation of transaction announcements between 2 nodes, which is a building block for efficient transaction relay protocols (e.g., [https://arxiv.org/pdf/1905.10518.pdf Erlay]). This is a step towards increasing the connectivity of the network for almost no bandwidth cost. - -==Motivation== - -Currently in the Bitcoin network, every 32-byte transaction ID is announced in at least one direction between every pair of connected peers, via INV messages. This results in high cost of announcing transactions: ''O(nodes * connections_per_node)''. - -A reconciliation-based protocol which uses the technique suggested in this document can have better scaling properties than INV-based flooding. - -Increasing the connectivity of the network makes the network more robust to partitioning attacks; thus, improving the bandwidth scaling of transaction relay to ''O(nodes)'' (and without a high constant overhead) would allow us to improve the security of the network by increasing connectivity. It would also reduce the bandwidth required to run a Bitcoin node and potentially enable more users to run full nodes. - -===Erlay=== - -[https://arxiv.org/pdf/1905.10518.pdf Erlay] is an example of a high-level transaction relay protocol which employs set reconciliation for bandwidth efficiency. - -Erlay uses both flooding (announcing using INV messages to all peers) and reconciliation to announce transactions. -Flooding is expensive, so Erlay seeks to use it sparingly and in strategic locations - only well-connected publicly reachable nodes flood transactions to other publicly reachable nodes via outbound connections. -Since every unreachable node is directly connected to several reachable nodes, this policy ensures that a transaction is quickly propagated to be within one hop from most of the nodes in the network. - -All transactions not propagated through flooding are propagated through efficient set reconciliation. -To do this, every node keeps a reconciliation set for each peer, in which transactions are placed which would have been announced using INV messages absent this protocol. Every 2 seconds every node chooses a peer from its outbound connections in a predetermined order to reconcile with, resulting in both sides learning the transactions known to the other side. After every reconciliation round, the corresponding reconciliation set is cleared. -A more detailed description of a set reconciliation round and other implementation details can be found in the paper. - -Erlay allows us to: -* save 40% of the bandwidth consumed by a node, given typical network connectivity as of July 2019. -* achieve similar latency -* increase network connectivity for almost no bandwidth or latency cost -* improves privacy as a side-effect - -This document proposes a P2P-layer extension which is required to enable efficient reconciliation-based protocols (like Erlay) for transaction relay. - -==Specification== - -===New data structures=== - -Several new data structures are introduced to the P2P protocol first, to aid with efficient transaction relay. - -====32-bit short transaction IDs==== - -During reconciliation, significantly abbreviated transaction IDs are used of just 32 bits in size. To prevent attackers from constructing sets of transactions that cause network-wide collisions, the short ID computation is salted on a per-link basis using 64 bits of entropy contributed by both communication partners. - -Short IDs are computed as follows: -* Let ''salt1'' and ''salt2'' be the entropy contributed by both sides; see the "sendrecon" message further for details how they are exchanged. -* Sort the two salts such that ''salt1 ≤ salt2'' (which side sent what doesn't matter). -* Compute ''h = SHA256("Tx Relay Salting" || salt1 || salt2)'', where the two salts are encoded in 64-bit little-endian byte order. -* Let ''k0'' be the 64-bit integer obtained by interpreting the first 8 bytes of ''h'' in little-endian byte order. -* Let ''k1'' be the 64-bit integer obtained by interpreting the second 8 bytes of ''h'' in little-endian byte order. -* Let ''s = SipHash-2-4((k0,k1),wtxid)'', where ''wtxid'' is the transaction hash including witness data as defined by BIP141. -* The short ID is equal to ''1 + (s mod 0xFFFFFFFF)''. - -This results in approximately uniformly distributed IDs in the range ''[1..0xFFFFFFFF]'', which is a requirement for using them as elements in 32-bit sketches. See the next paragraph for details. - -====Short transaction ID sketches==== - -Reconciliation-based relay uses [https://www.cs.bu.edu/~reyzin/code/fuzzy.html PinSketch] BCH-based secure sketches as introduced by the [https://www.cs.bu.edu/~reyzin/fuzzy.html Fuzzy Extractors paper]. They are a form of set checksums with the following properties: -* Sketches have a predetermined capacity, and when the number of elements in the set does not exceed the capacity, it is always possible to recover the entire set from the sketch by decoding the sketch. A sketch of nonzero b-bit elements with capacity c can be stored in bc bits. -* A sketch of the [https://en.wikipedia.org/wiki/Symmetric_difference symmetric difference] between the two sets (i.e., all elements that occur in one but not both input sets), can be obtained by combining the sketches of those sets. - -The sketches used here consists of elements of the [https://en.wikipedia.org/wiki/Finite_field finite field] ''GF(232)''. Specifically, we represent finite field elements as polynomials in ''x'' over ''GF(2)'' modulo ''x327 + x3 + x2 + 1''. To map integers to finite field elements, simply treat each bit ''i'' (with value ''2i'') in the integer as the coefficient of ''xi'' in the polynomial representation. For example the integer ''101 = 26 + 25 + 22 + 1'' is mapped to field element ''x6 + x5 + x2 + 1''. These field elements can be added and multiplied together, but the specifics of that are out of scope for this document. - -A short ID sketch with capacity ''c'' consists of a sequence of ''c'' field elements. The first is the sum of all short IDs in the set, the second is the sum of the 3rd powers of all short IDs, the third is the sum of the 5th powers etc., up to the last element with is the sum of the ''(2c-1)''th powers. These elements are then encoded as 32-bit integers in little endian byte order, resulting in a ''4c''-byte serialization. - -The following Python 3.2+ code implements the creation of sketches:
-FIELD_BITS = 32
-FIELD_MODULUS = (1 << FIELD_BITS) + 0b10001101
-
-def mul2(x):
-    """Compute 2*x in GF(2^FIELD_BITS)"""
-    return (x << 1) ^ (FIELD_MODULUS if x.bit_length() >= FIELD_BITS else 0)
-
-def mul(x, y):
-    """Compute x*y in GF(2^FIELD_BITS)"""
-    ret = 0
-    for bit in [(x >> i) & 1 for i in range(x.bit_length())]:
-        ret, y = ret ^ bit * y, mul2(y)
-    return ret
-
-def create_sketch(shortids, capacity):
-    """Compute the bytes of a sketch for given shortids and given capacity."""
-    odd_sums = [0 for _ in range(capacity)]
-    for shortid in shortids:
-        squared = mul(shortid, shortid)
-        for i in range(capacity):
-            odd_sums[i] ^= shortid
-            shortid = mul(shortid, squared)
-    return b''.join(elem.to_bytes(4, 'little') for elem in odd_sums)
-
- -The [https://github.com/sipa/minisketch/ minisketch] library implements the construction, merging, and decoding of these sketches efficiently. - -====Truncated transaction IDs==== - -For announcing and relaying transaction outside of reconciliation, we need an unambiguous, unsalted way to refer to transactions to deduplicate transaction requests. As we're introducing a new scheme anyway, this is a good opportunity to switch to wtxid-based requests rather than txid-based ones. While using full 256-bit wtxids is possible, this is overkill as they contribute significantly to the total bandwidth as well. Instead, we truncate the wtxid to just their first 128 bits. These are referred to as truncated IDs. - -===Intended Protocol Flow=== - -Set reconciliation primarily consists of the transmission and decoding of a reconciliation set sketch upon request. - -[[File:bip-reconcil/recon_scheme_merged.png|framed|center|Set reconciliation protocol flow]] - -====Bisection==== - -If a node is unable to reconstruct the set difference from the received sketch, the node then makes an additional reconciliation request, similar to the initial one, but this request is applied to only a fraction of possible transactions (e.g., in the range 0x0–0x8). Because of the linearity of sketches, a sketch of a subset of transactions would allow the node to compute a sketch for the remainder, which saves bandwidth. - -[[File:bip-reconcil/bisection.png|framed|300px|center|Bisection]] - -===New messages=== -Several new protocol messages are added: sendrecon, reqreconcil, sketch, reqbisec, reconcildiff, invtx, gettx. This section describes their serialization, contents, and semantics. - -In what follows, all integers are serialized in little-endian byte order. Boolean values are encoded as a single byte that must be 0 or 1 exactly. Arrays are serialized with the CompactSize prefix that encodes their length, as is common in other P2P messages. - -====sendrecon==== -The sendrecon message announces support for the reconciliation protocol. It is expected to be only sent once, and ignored by nodes that don't support it. - -Its payload consists of: -{|class="wikitable" -! Data type !! Name !! Description -|- -| bool || sender || Indicates whether the sender will send "reqreconcil" message -|- -| bool || responder || Indicates whether the sender will respond to "reqreconcil" messages. -|- -| uint32 || version || Sender must set this to 1 currently, otherwise receiver should ignore the message. -|- -| uint64 || salt || The salt used in the short transaction ID computation. -|} - -"reqreconcil" messages can only be sent if the sender has sent a "sendrecon" message with sender=true, and the receiver has sent a "sendrecon" message with responder=true. - -====reqreconcil==== -The reqreconcil message initiates a reconciliation round. - -{|class="wikitable" -! Data type !! Name !! Description -|- -| uint16 || set_size || Size of the sender's reconciliation set, used to estimate set difference. -|- -| uint8 || q || Coefficient used to estimate set difference. Multiplied by PRECISION=2^6 and rounded up by the sender and divided by PRECISION by the receiver. -|} - -Upon receipt of a "reqreconcil" message, the receiver: -* Constructs and sends a "sketch" message (see below), with a sketch of capacity computed as ``|set_size - local_set_size| + q * (set_size + local_set_size) + c'', where ''local_set_size'' represents size of the receiver's reconciliation set. -* Makes a snapshot of their current reconciliation set, and clears the set itself. The snapshot is kept until a "reconcildiff" message is received by the node. - -It is suggested to use ''c=1'' to avoid sending empty sketches and reduce the overhead caused by under-estimations. - -Intuitively, ''q'' represents the discrepancy in sets: the closer the sets are, the lower optimal ''q'' is. -As suggested by Erlay, ''q'' should be derived as an optimal ''q'' value for the previous reconciliation with a given peer, once the actual set sizes and set difference are known. Alternatively, ''q=0.1'' should be used as a default value. -For example, if in previous round ''set_size=30'' and ''local_set_size=20'', and the *actual* difference was ''4'', then a node should compute ''q'' as following: -''q=(|30-20| - 1) / (30+20)=0.18'' -The derivation of ''q'' can be changed according to the version of the protocol. - -No new "reqreconcil" message can be sent until a "reconcildiff" message is sent. - -====sketch==== -The sketch message is used to communicate a sketch required to perform set reconciliation. - -{|class="wikitable" -! Data type !! Name !! Description -|- -| byte[] || skdata || The sketch of the sender's reconciliation snapshot -|} - -Upon receipt of a "sketch" message, a node computes the set difference by combining the receiver sketch with a sketch computed locally for a corresponding reconciliation set. If this is the 2nd time for this round a "sketch" message was received, the bisection approach is used, and by combining the new sketch with the previous one, two difference sketches are obtained, one for the first half and one for the second half of the short id range. The receiving node then tries to decode this sketch (or sketches), and based on the result: -* If decoding fails, a "reconcildiff" message is sent with the failure flag set (success=false). If this was the first "sketch" in the round, a "reqbisec" message may be sent instead. -* If decoding succeeds, a "reconcildiff" message is sent with the truncated IDs of all locally known transactions that appear in the decode result, and the short IDs of the unrecognized ones. - -The receiver also makes snapshot of their current reconciliation set, and clears the set itself. The snapshot is kept until a "reconcildiff" message is sent by the node. - -====reqbisec==== -The reqbisec message is used to signal that set reconciliation has failed and an extra sketch is needed to find set difference. - -It has an empty payload. - -Upon receipt of a "reqbisec" message, a node responds to it with a "sketch" message, which contains a sketch of a subset of corresponding reconciliation set snapshot (stored when "reqreconcil" message for the current round was processed) (values in range ''[0..(2^31)]''). - -====reconcildiff==== -The reconcildiff message is used to announce transactions which are found to be missing during set reconciliation on the sender's side. - -{|class="wikitable" -! Data type !! Name !! Description -|- -| uint8 || success || Indicates whether sender of the message succeeded at set difference decoding. -|- -| uint32[] || ask_shortids || The short IDs that the sender did not have. -|} - -Upon receipt a "reconcildiff" message with ''success=1'', a node sends a "invtx" message for the transactions requested by 32-bit IDs (first vector) containing their 128-bit truncated IDs (with parent transactions occuring before their dependencies), and can request announced transactions (second vector) it does not have via a "gettx" message. -Otherwise if ''success=0'', receiver should request bisection via ''reqbisec'' (if failure happened for the first time). -If failure happened for the second time, receiver should announce the transactions from the reconciliation set via an "invtx" message, excluding the transactions announced from the sender. - -The snapshot of the corresponding reconciliation set is cleared by the sender and the receiver of the message. - -The sender should also send their own "invtx" message along with the reconcildiff message to announce transactions which are missing on the receiver's side. - -====invtx==== -The invtx message is used to announce transactions (both along with reconcildiff message and as a response to the reconcildiff message). It is the truncated ID analogue of "inv" (which cannot be used because it has 256-bit elements). - -{|class="wikitable" -! Data type !! Name !! Description -|- -| uint128[] || inv_truncids || The truncated IDs of transactions the sender believes the receiver does not have. -|} - -Upon receipt a "invtx" message, a node requests announced transactions it does not have. -The snapshot of the corresponding reconciliation set is cleared by the sender of the message. - -====gettx==== -The gettx message is used to request transactions by 128-bit truncated IDs. It is the truncated ID analogue of "getdata". - -{|class="wikitable" -! Data type !! Name !! Description -|- -| uint128[] || ask_truncids || The truncated IDs of transactions the sender wants the full transaction data for. -|} - -Upon receipt a "gettx" message, a node sends "tx" messages for the requested transactions. - -==Local state== - -This BIP suggests a stateful protocol and it requires storing several variables at every node to operate properly. - -====Reconciliation sets==== -Every node stores a set of 128-bit truncated IDs for every peer which supports transaction reconciliation, representing the transactions which would have been sent according to the regular flooding protocol. -Incoming transactions are added to sets when those transactions are received (if they satisfy the policies such as minimum fee set by a peer). -A reconciliation set is moved to the corresponding set snapshot after the transmission of the initial sketch. - -====Reconciliation set snapshot==== -After the transmitting of the initial sketch (either sending or receiving of reconcildiff message), every node should store the snapshot of the current reconciliation set, and clear the set. -This is important to make bisection more stable during the reconciliation round (bisection should be applied to the snapshot). -The snapshot is also used to efficiently lookup the transactions requested by short ID. -The snapshot is cleared after the end of the reconciliation round (sending or receiving of the reconcildiff message). - -====q-coefficient==== -The q value should be stored to make efficient difference estimation. It is shared across peers and changed after every reconciliation. -q-coefficient represents the discrepancy in sets: the closer the sets are, the lower optimal ''q'' is. -In future implementations, q could vary across different peers or become static. - - -==Backward compatibility== - -Older clients remain fully compatible and interoperable after this change. - -Clients which do not implement this protocol remain fully compatible after this change using existing protocols, because transaction announcement reconciliation is used only for peers that negotiate support for it. - -==Rationale== - -====Why using PinSketch for set reconciliation?==== - -PinSketch is more bandwidth efficient than IBLT, especially for the small differences in sets we expect to operate over. -PinSketch is as bandwidth efficient as CPISync, but PinSketch has quadratic decoding complexity, while CPISync have cubic decoding complexity. This makes PinSketch significantly faster. - -====Why using 32-bit short transaction IDs?==== - -To use Minisketch in practice, transaction IDs should be shortened (ideally, not more than 64 bits per element). -Small number of bits per transaction also allows to save extra bandwidth and make operations over sketches faster. -According to our estimates, 32 bits provides low collision rate in a non-adversarial model (which is enabled by using independent salts per-link). - -====Why using 128-bit short IDs?==== - -To avoid problems caused by the delays in the network, our protocol requires extra round of announcing unsalted transaction IDs. [https://arxiv.org/pdf/1905.10518.pdf Erlay] protocol on top of this work also requires announcing unsalted transaction IDs for flooding. -Both of these measures allow to deduplicate transaction announcements across the peers. -However, using full 256-bit IDs to uniquely identify transactions seems to be an overkill. -128 is the highest power of 2 which provides good enough collision-resistance in an adversarial model, and trivially saves a significant portion of the bandwidth related to these announcements. - -====Why using bisection instead of extending the sketch?==== - -Unlike extended sketches, bisection does not require operating over sketches of higher order. -This allows to avoid the high computational cost caused by quadratic decoding complexity. - -==Implementation== - -TODO - -==Acknowledgments== - -A large fraction of this proposal was done during designing Erlay with Gregory Maxwell, Sasha Fedorova and Ivan Beschastnikh. -We would like to thank Suhas Daftuar for contributions to the design and BIP structure. -We would like to thank Ben Woosley for contributions to the high-level description of the idea. - -==Copyright== - -This document is licensed under the Creative Commons CC0 1.0 Universal license. diff --git a/bip-reconcil/bisection.png b/bip-reconcil/bisection.png deleted file mode 100644 index 70f37e8..0000000 Binary files a/bip-reconcil/bisection.png and /dev/null differ diff --git a/bip-reconcil/minisketch.py b/bip-reconcil/minisketch.py deleted file mode 100755 index f64286f..0000000 --- a/bip-reconcil/minisketch.py +++ /dev/null @@ -1,157 +0,0 @@ -#!/usr/bin/env python3 - -######## ENCODING and DECODING ######## - -FIELD_BITS = 32 -FIELD_MODULUS = (1 << FIELD_BITS) + 0b10001101 - -def mul2(x): - """Compute 2*x in GF(2^FIELD_BITS)""" - return (x << 1) ^ (FIELD_MODULUS if x.bit_length() >= FIELD_BITS else 0) - -def mul(x, y): - """Compute x*y in GF(2^FIELD_BITS)""" - ret = 0 - for bit in [(x >> i) & 1 for i in range(x.bit_length())]: - ret ^= bit * y - y = mul2(y) - return ret - -######## ENCODING only ######## - -def sketch(shortids, capacity): - """Compute the bytes of a sketch for given shortids and given capacity.""" - odd_sums = [0 for _ in range(capacity)] - for shortid in shortids: - squared = mul(shortid, shortid) - for i in range(capacity): - odd_sums[i] ^= shortid - shortid = mul(shortid, squared) - return b''.join(elem.to_bytes(4, 'little') for elem in odd_sums) - -######## DECODING only ######## - -import random - -def inv(x): - """Compute 1/x in GF(2^FIELD_BITS)""" - t = x - for i in range(FIELD_BITS - 2): - t = mul(mul(t, t), x) - return mul(t, t) - - -def berlekamp_massey(s): - """Given a sequence of LFSR outputs, find the coefficients of the LFSR.""" - C, B, L, m, b = [1], [1], 0, 1, 1 - for n in range(len(s)): - d = s[n] - for i in range(1, L + 1): - d ^= mul(C[i], s[n - i]) - if d == 0: - m += 1 - else: - T = list(C) - while len(C) <= len(B) + m: - C += [0] - t = mul(d, inv(b)) - for i in range(len(B)): - C[i + m] ^= mul(t, B[i]) - if 2 * L <= n: - L, B, b, m = n + 1 - L, T, d, 1 - else: - m += 1 - return C[0:L + 1] - -def poly_monic(p): - """Return the monic multiple of p, or 0 if the input is 0.""" - if len(p) == 0: - return [] - i = inv(p[-1]) - return [mul(v, i) for v in p] - -def poly_divmod(m, p): - """Compute the polynomial quotient p/m, and replace p with p mod m.""" - assert(len(m) > 0 and m[-1] == 1) - div = [0 for _ in range(len(p) - len(m) + 1)] - while len(p) >= len(m): - div[len(p) - len(m)] = p[-1] - for i in range(len(m)): - p[len(p) - len(m) + i] ^= mul(p[-1], m[i]) - assert(p[-1] == 0) - p.pop() - while (len(p) > 0 and p[-1] == 0): - p.pop() - return div - -def poly_gcd(a, b): - """Compute the GCD of a and b (destroys the inputs).""" - if len(a) < len(b): - a, b = b, a - while len(b): - if len(b) == 1: - return [1] - b = poly_monic(b) - poly_divmod(b, a) - a, b = b, a - return a - -def poly_sqr(p): - """Compute the coefficients of the square of polynomial with coefficients p.""" - return [0 if i & 1 else mul(p[i // 2], p[i // 2]) for i in range(2 * len(p))] - -def poly_trace(m, a): - """Compute the coefficients of the trace polynomial of (a*x) mod m.""" - out = [0, a] - for i in range(FIELD_BITS - 1): - out = poly_sqr(out) - while len(out) < 2: - out += [0] - out[1] = a - poly_divmod(m, out) - return out - -def find_roots_inner(p, a): - """Recursive helper function for find_roots (destroys p). a is randomizer.""" - # p must be monic - assert(len(p) > 0 and p[-1] == 1) - # Deal with degree 0 and degree 1 inputs - if len(p) == 1: - return [] - elif len(p) == 2: - return [p[0]] - # Otherwise, split p in left*right using paramater a_vals[0]. - t = poly_monic(poly_trace(p, a)) - left = poly_gcd(list(p), t) - right = poly_divmod(list(left), p) - # Invoke recursion with the remaining a_vals. - ret_right = find_roots_inner(right, mul2(a)) - ret_left = find_roots_inner(left, mul2(a)) - # Concatenate roots - return ret_left + ret_right - -def find_roots(p): - """Find the roots of polynomial with coefficients p.""" - # Compute x^(2^FIELD_BITS)+x mod p in a roundabout way. - t = poly_trace(p, 1) - t2 = poly_sqr(t) - for i in range(len(t)): - t2[i] ^= t[i] - poly_divmod(p, t2) - # If distinct from 0, p is not fully factorizable into non-repeating roots. - if len(t2): - return None - # Invoke the recursive splitting algorithm - return find_roots_inner(list(p), random.randrange(1, 2**32-1)) - -def decode(sketch): - """Recover the shortids from a sketch.""" - odd_sums = [int.from_bytes(sketch[i*4:(i+1)*4], 'little') for i in range(len(sketch) // 4)] - sums = [] - for i in range(len(odd_sums) * 2): - if i & 1: - sums.append(mul(sums[(i-1)//2], sums[(i-1)//2])) - else: - sums.append(odd_sums[(i+1)//2]) - return find_roots(list(reversed(berlekamp_massey(sums)))) - diff --git a/bip-reconcil/recon_scheme_merged.png b/bip-reconcil/recon_scheme_merged.png deleted file mode 100644 index 11d1559..0000000 Binary files a/bip-reconcil/recon_scheme_merged.png and /dev/null differ -- cgit v1.2.3 From 2e7dab87ef6822d2ed81df0eaa8e4fa9a665e86c Mon Sep 17 00:00:00 2001 From: User Date: Mon, 4 Nov 2019 13:30:08 -0500 Subject: added missing leading zero --- bip-330.mediawiki | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/bip-330.mediawiki b/bip-330.mediawiki index 36dbdd4..197f94e 100644 --- a/bip-330.mediawiki +++ b/bip-330.mediawiki @@ -112,13 +112,13 @@ For announcing and relaying transaction outside of reconciliation, we need an un Set reconciliation primarily consists of the transmission and decoding of a reconciliation set sketch upon request. -[[File:bip-330/recon_scheme_merged.png|framed|center|Set reconciliation protocol flow]] +[[File:bip-0330/recon_scheme_merged.png|framed|center|Set reconciliation protocol flow]] ====Bisection==== If a node is unable to reconstruct the set difference from the received sketch, the node then makes an additional reconciliation request, similar to the initial one, but this request is applied to only a fraction of possible transactions (e.g., in the range 0x0–0x8). Because of the linearity of sketches, a sketch of a subset of transactions would allow the node to compute a sketch for the remainder, which saves bandwidth. -[[File:bip-330/bisection.png|framed|300px|center|Bisection]] +[[File:bip-0330/bisection.png|framed|300px|center|Bisection]] ===New messages=== Several new protocol messages are added: sendrecon, reqreconcil, sketch, reqbisec, reconcildiff, invtx, gettx. This section describes their serialization, contents, and semantics. -- cgit v1.2.3 From 7f9ad3ebe53290b22d3a809d3ef1566d12fbceab Mon Sep 17 00:00:00 2001 From: User Date: Mon, 4 Nov 2019 13:39:01 -0500 Subject: add trailing zero to the file name --- bip-0330.mediawiki | 299 +++++++++++++++++++++++++++++++++++++++++++++++++++++ bip-330.mediawiki | 299 ----------------------------------------------------- 2 files changed, 299 insertions(+), 299 deletions(-) create mode 100644 bip-0330.mediawiki delete mode 100644 bip-330.mediawiki diff --git a/bip-0330.mediawiki b/bip-0330.mediawiki new file mode 100644 index 0000000..197f94e --- /dev/null +++ b/bip-0330.mediawiki @@ -0,0 +1,299 @@ +
+  BIP: 330
+  Layer: Peer Services
+  Title: Transaction announcements reconciliation
+  Author: Gleb Naumenko 
+          Pieter Wuille 
+  Comments-Summary: ???
+  Comments-URI: ???
+  Status: Draft
+  Type: Standards Track
+  Created: 2010-00-00
+  License: CC0-1.0
+  License-Code: MIT
+
+ +==Abstract== + +This document specifies a P2P protocol extension for reconciliation of transaction announcements between 2 nodes, which is a building block for efficient transaction relay protocols (e.g., [https://arxiv.org/pdf/1905.10518.pdf Erlay]). This is a step towards increasing the connectivity of the network for almost no bandwidth cost. + +==Motivation== + +Currently in the Bitcoin network, every 32-byte transaction ID is announced in at least one direction between every pair of connected peers, via INV messages. This results in high cost of announcing transactions: ''O(nodes * connections_per_node)''. + +A reconciliation-based protocol which uses the technique suggested in this document can have better scaling properties than INV-based flooding. + +Increasing the connectivity of the network makes the network more robust to partitioning attacks; thus, improving the bandwidth scaling of transaction relay to ''O(nodes)'' (and without a high constant overhead) would allow us to improve the security of the network by increasing connectivity. It would also reduce the bandwidth required to run a Bitcoin node and potentially enable more users to run full nodes. + +===Erlay=== + +[https://arxiv.org/pdf/1905.10518.pdf Erlay] is an example of a high-level transaction relay protocol which employs set reconciliation for bandwidth efficiency. + +Erlay uses both flooding (announcing using INV messages to all peers) and reconciliation to announce transactions. +Flooding is expensive, so Erlay seeks to use it sparingly and in strategic locations - only well-connected publicly reachable nodes flood transactions to other publicly reachable nodes via outbound connections. +Since every unreachable node is directly connected to several reachable nodes, this policy ensures that a transaction is quickly propagated to be within one hop from most of the nodes in the network. + +All transactions not propagated through flooding are propagated through efficient set reconciliation. +To do this, every node keeps a reconciliation set for each peer, in which transactions are placed which would have been announced using INV messages absent this protocol. Every 2 seconds every node chooses a peer from its outbound connections in a predetermined order to reconcile with, resulting in both sides learning the transactions known to the other side. After every reconciliation round, the corresponding reconciliation set is cleared. +A more detailed description of a set reconciliation round and other implementation details can be found in the paper. + +Erlay allows us to: +* save 40% of the bandwidth consumed by a node, given typical network connectivity as of July 2019. +* achieve similar latency +* increase network connectivity for almost no bandwidth or latency cost +* improves privacy as a side-effect + +This document proposes a P2P-layer extension which is required to enable efficient reconciliation-based protocols (like Erlay) for transaction relay. + +==Specification== + +===New data structures=== + +Several new data structures are introduced to the P2P protocol first, to aid with efficient transaction relay. + +====32-bit short transaction IDs==== + +During reconciliation, significantly abbreviated transaction IDs are used of just 32 bits in size. To prevent attackers from constructing sets of transactions that cause network-wide collisions, the short ID computation is salted on a per-link basis using 64 bits of entropy contributed by both communication partners. + +Short IDs are computed as follows: +* Let ''salt1'' and ''salt2'' be the entropy contributed by both sides; see the "sendrecon" message further for details how they are exchanged. +* Sort the two salts such that ''salt1 ≤ salt2'' (which side sent what doesn't matter). +* Compute ''h = SHA256("Tx Relay Salting" || salt1 || salt2)'', where the two salts are encoded in 64-bit little-endian byte order. +* Let ''k0'' be the 64-bit integer obtained by interpreting the first 8 bytes of ''h'' in little-endian byte order. +* Let ''k1'' be the 64-bit integer obtained by interpreting the second 8 bytes of ''h'' in little-endian byte order. +* Let ''s = SipHash-2-4((k0,k1),wtxid)'', where ''wtxid'' is the transaction hash including witness data as defined by BIP141. +* The short ID is equal to ''1 + (s mod 0xFFFFFFFF)''. + +This results in approximately uniformly distributed IDs in the range ''[1..0xFFFFFFFF]'', which is a requirement for using them as elements in 32-bit sketches. See the next paragraph for details. + +====Short transaction ID sketches==== + +Reconciliation-based relay uses [https://www.cs.bu.edu/~reyzin/code/fuzzy.html PinSketch] BCH-based secure sketches as introduced by the [https://www.cs.bu.edu/~reyzin/fuzzy.html Fuzzy Extractors paper]. They are a form of set checksums with the following properties: +* Sketches have a predetermined capacity, and when the number of elements in the set does not exceed the capacity, it is always possible to recover the entire set from the sketch by decoding the sketch. A sketch of nonzero b-bit elements with capacity c can be stored in bc bits. +* A sketch of the [https://en.wikipedia.org/wiki/Symmetric_difference symmetric difference] between the two sets (i.e., all elements that occur in one but not both input sets), can be obtained by combining the sketches of those sets. + +The sketches used here consists of elements of the [https://en.wikipedia.org/wiki/Finite_field finite field] ''GF(232)''. Specifically, we represent finite field elements as polynomials in ''x'' over ''GF(2)'' modulo ''x327 + x3 + x2 + 1''. To map integers to finite field elements, simply treat each bit ''i'' (with value ''2i'') in the integer as the coefficient of ''xi'' in the polynomial representation. For example the integer ''101 = 26 + 25 + 22 + 1'' is mapped to field element ''x6 + x5 + x2 + 1''. These field elements can be added and multiplied together, but the specifics of that are out of scope for this document. + +A short ID sketch with capacity ''c'' consists of a sequence of ''c'' field elements. The first is the sum of all short IDs in the set, the second is the sum of the 3rd powers of all short IDs, the third is the sum of the 5th powers etc., up to the last element with is the sum of the ''(2c-1)''th powers. These elements are then encoded as 32-bit integers in little endian byte order, resulting in a ''4c''-byte serialization. + +The following Python 3.2+ code implements the creation of sketches:
+FIELD_BITS = 32
+FIELD_MODULUS = (1 << FIELD_BITS) + 0b10001101
+
+def mul2(x):
+    """Compute 2*x in GF(2^FIELD_BITS)"""
+    return (x << 1) ^ (FIELD_MODULUS if x.bit_length() >= FIELD_BITS else 0)
+
+def mul(x, y):
+    """Compute x*y in GF(2^FIELD_BITS)"""
+    ret = 0
+    for bit in [(x >> i) & 1 for i in range(x.bit_length())]:
+        ret, y = ret ^ bit * y, mul2(y)
+    return ret
+
+def create_sketch(shortids, capacity):
+    """Compute the bytes of a sketch for given shortids and given capacity."""
+    odd_sums = [0 for _ in range(capacity)]
+    for shortid in shortids:
+        squared = mul(shortid, shortid)
+        for i in range(capacity):
+            odd_sums[i] ^= shortid
+            shortid = mul(shortid, squared)
+    return b''.join(elem.to_bytes(4, 'little') for elem in odd_sums)
+
+ +The [https://github.com/sipa/minisketch/ minisketch] library implements the construction, merging, and decoding of these sketches efficiently. + +====Truncated transaction IDs==== + +For announcing and relaying transaction outside of reconciliation, we need an unambiguous, unsalted way to refer to transactions to deduplicate transaction requests. As we're introducing a new scheme anyway, this is a good opportunity to switch to wtxid-based requests rather than txid-based ones. While using full 256-bit wtxids is possible, this is overkill as they contribute significantly to the total bandwidth as well. Instead, we truncate the wtxid to just their first 128 bits. These are referred to as truncated IDs. + +===Intended Protocol Flow=== + +Set reconciliation primarily consists of the transmission and decoding of a reconciliation set sketch upon request. + +[[File:bip-0330/recon_scheme_merged.png|framed|center|Set reconciliation protocol flow]] + +====Bisection==== + +If a node is unable to reconstruct the set difference from the received sketch, the node then makes an additional reconciliation request, similar to the initial one, but this request is applied to only a fraction of possible transactions (e.g., in the range 0x0–0x8). Because of the linearity of sketches, a sketch of a subset of transactions would allow the node to compute a sketch for the remainder, which saves bandwidth. + +[[File:bip-0330/bisection.png|framed|300px|center|Bisection]] + +===New messages=== +Several new protocol messages are added: sendrecon, reqreconcil, sketch, reqbisec, reconcildiff, invtx, gettx. This section describes their serialization, contents, and semantics. + +In what follows, all integers are serialized in little-endian byte order. Boolean values are encoded as a single byte that must be 0 or 1 exactly. Arrays are serialized with the CompactSize prefix that encodes their length, as is common in other P2P messages. + +====sendrecon==== +The sendrecon message announces support for the reconciliation protocol. It is expected to be only sent once, and ignored by nodes that don't support it. + +Its payload consists of: +{|class="wikitable" +! Data type !! Name !! Description +|- +| bool || sender || Indicates whether the sender will send "reqreconcil" message +|- +| bool || responder || Indicates whether the sender will respond to "reqreconcil" messages. +|- +| uint32 || version || Sender must set this to 1 currently, otherwise receiver should ignore the message. +|- +| uint64 || salt || The salt used in the short transaction ID computation. +|} + +"reqreconcil" messages can only be sent if the sender has sent a "sendrecon" message with sender=true, and the receiver has sent a "sendrecon" message with responder=true. + +====reqreconcil==== +The reqreconcil message initiates a reconciliation round. + +{|class="wikitable" +! Data type !! Name !! Description +|- +| uint16 || set_size || Size of the sender's reconciliation set, used to estimate set difference. +|- +| uint8 || q || Coefficient used to estimate set difference. Multiplied by PRECISION=2^6 and rounded up by the sender and divided by PRECISION by the receiver. +|} + +Upon receipt of a "reqreconcil" message, the receiver: +* Constructs and sends a "sketch" message (see below), with a sketch of capacity computed as ''|set_size - local_set_size| + q * (set_size + local_set_size) + c'', where ''local_set_size'' represents size of the receiver's reconciliation set. +* Makes a snapshot of their current reconciliation set, and clears the set itself. The snapshot is kept until a "reconcildiff" message is received by the node. + +It is suggested to use ''c=1'' to avoid sending empty sketches and reduce the overhead caused by under-estimations. + +Intuitively, ''q'' represents the discrepancy in sets: the closer the sets are, the lower optimal ''q'' is. +As suggested by Erlay, ''q'' should be derived as an optimal ''q'' value for the previous reconciliation with a given peer, once the actual set sizes and set difference are known. Alternatively, ''q=0.1'' should be used as a default value. +For example, if in previous round ''set_size=30'' and ''local_set_size=20'', and the *actual* difference was ''4'', then a node should compute ''q'' as following: +''q=(|30-20| - 1) / (30+20)=0.18'' +The derivation of ''q'' can be changed according to the version of the protocol. + +No new "reqreconcil" message can be sent until a "reconcildiff" message is sent. + +====sketch==== +The sketch message is used to communicate a sketch required to perform set reconciliation. + +{|class="wikitable" +! Data type !! Name !! Description +|- +| byte[] || skdata || The sketch of the sender's reconciliation snapshot +|} + +Upon receipt of a "sketch" message, a node computes the set difference by combining the receiver sketch with a sketch computed locally for a corresponding reconciliation set. If this is the 2nd time for this round a "sketch" message was received, the bisection approach is used, and by combining the new sketch with the previous one, two difference sketches are obtained, one for the first half and one for the second half of the short id range. The receiving node then tries to decode this sketch (or sketches), and based on the result: +* If decoding fails, a "reconcildiff" message is sent with the failure flag set (success=false). If this was the first "sketch" in the round, a "reqbisec" message may be sent instead. +* If decoding succeeds, a "reconcildiff" message is sent with the truncated IDs of all locally known transactions that appear in the decode result, and the short IDs of the unrecognized ones. + +The receiver also makes snapshot of their current reconciliation set, and clears the set itself. The snapshot is kept until a "reconcildiff" message is sent by the node. + +====reqbisec==== +The reqbisec message is used to signal that set reconciliation has failed and an extra sketch is needed to find set difference. + +It has an empty payload. + +Upon receipt of a "reqbisec" message, a node responds to it with a "sketch" message, which contains a sketch of a subset of corresponding reconciliation set snapshot (stored when "reqreconcil" message for the current round was processed) (values in range ''[0..(2^31)]''). + +====reconcildiff==== +The reconcildiff message is used to announce transactions which are found to be missing during set reconciliation on the sender's side. + +{|class="wikitable" +! Data type !! Name !! Description +|- +| uint8 || success || Indicates whether sender of the message succeeded at set difference decoding. +|- +| uint32[] || ask_shortids || The short IDs that the sender did not have. +|} + +Upon receipt a "reconcildiff" message with ''success=1'', a node sends a "invtx" message for the transactions requested by 32-bit IDs (first vector) containing their 128-bit truncated IDs (with parent transactions occuring before their dependencies), and can request announced transactions (second vector) it does not have via a "gettx" message. +Otherwise if ''success=0'', receiver should request bisection via ''reqbisec'' (if failure happened for the first time). +If failure happened for the second time, receiver should announce the transactions from the reconciliation set via an "invtx" message, excluding the transactions announced from the sender. + +The snapshot of the corresponding reconciliation set is cleared by the sender and the receiver of the message. + +The sender should also send their own "invtx" message along with the reconcildiff message to announce transactions which are missing on the receiver's side. + +====invtx==== +The invtx message is used to announce transactions (both along with reconcildiff message and as a response to the reconcildiff message). It is the truncated ID analogue of "inv" (which cannot be used because it has 256-bit elements). + +{|class="wikitable" +! Data type !! Name !! Description +|- +| uint128[] || inv_truncids || The truncated IDs of transactions the sender believes the receiver does not have. +|} + +Upon receipt a "invtx" message, a node requests announced transactions it does not have. +The snapshot of the corresponding reconciliation set is cleared by the sender of the message. + +====gettx==== +The gettx message is used to request transactions by 128-bit truncated IDs. It is the truncated ID analogue of "getdata". + +{|class="wikitable" +! Data type !! Name !! Description +|- +| uint128[] || ask_truncids || The truncated IDs of transactions the sender wants the full transaction data for. +|} + +Upon receipt a "gettx" message, a node sends "tx" messages for the requested transactions. + +==Local state== + +This BIP suggests a stateful protocol and it requires storing several variables at every node to operate properly. + +====Reconciliation sets==== +Every node stores a set of 128-bit truncated IDs for every peer which supports transaction reconciliation, representing the transactions which would have been sent according to the regular flooding protocol. +Incoming transactions are added to sets when those transactions are received (if they satisfy the policies such as minimum fee set by a peer). +A reconciliation set is moved to the corresponding set snapshot after the transmission of the initial sketch. + +====Reconciliation set snapshot==== +After the transmitting of the initial sketch (either sending or receiving of reconcildiff message), every node should store the snapshot of the current reconciliation set, and clear the set. +This is important to make bisection more stable during the reconciliation round (bisection should be applied to the snapshot). +The snapshot is also used to efficiently lookup the transactions requested by short ID. +The snapshot is cleared after the end of the reconciliation round (sending or receiving of the reconcildiff message). + +====q-coefficient==== +The q value should be stored to make efficient difference estimation. It is shared across peers and changed after every reconciliation. +q-coefficient represents the discrepancy in sets: the closer the sets are, the lower optimal ''q'' is. +In future implementations, q could vary across different peers or become static. + + +==Backward compatibility== + +Older clients remain fully compatible and interoperable after this change. + +Clients which do not implement this protocol remain fully compatible after this change using existing protocols, because transaction announcement reconciliation is used only for peers that negotiate support for it. + +==Rationale== + +====Why using PinSketch for set reconciliation?==== + +PinSketch is more bandwidth efficient than IBLT, especially for the small differences in sets we expect to operate over. +PinSketch is as bandwidth efficient as CPISync, but PinSketch has quadratic decoding complexity, while CPISync have cubic decoding complexity. This makes PinSketch significantly faster. + +====Why using 32-bit short transaction IDs?==== + +To use Minisketch in practice, transaction IDs should be shortened (ideally, not more than 64 bits per element). +Small number of bits per transaction also allows to save extra bandwidth and make operations over sketches faster. +According to our estimates, 32 bits provides low collision rate in a non-adversarial model (which is enabled by using independent salts per-link). + +====Why using 128-bit short IDs?==== + +To avoid problems caused by the delays in the network, our protocol requires extra round of announcing unsalted transaction IDs. [https://arxiv.org/pdf/1905.10518.pdf Erlay] protocol on top of this work also requires announcing unsalted transaction IDs for flooding. +Both of these measures allow to deduplicate transaction announcements across the peers. +However, using full 256-bit IDs to uniquely identify transactions seems to be an overkill. +128 is the highest power of 2 which provides good enough collision-resistance in an adversarial model, and trivially saves a significant portion of the bandwidth related to these announcements. + +====Why using bisection instead of extending the sketch?==== + +Unlike extended sketches, bisection does not require operating over sketches of higher order. +This allows to avoid the high computational cost caused by quadratic decoding complexity. + +==Implementation== + +TODO + +==Acknowledgments== + +A large fraction of this proposal was done during designing Erlay with Gregory Maxwell, Sasha Fedorova and Ivan Beschastnikh. +We would like to thank Suhas Daftuar for contributions to the design and BIP structure. +We would like to thank Ben Woosley for contributions to the high-level description of the idea. + +==Copyright== + +This document is licensed under the Creative Commons CC0 1.0 Universal license. diff --git a/bip-330.mediawiki b/bip-330.mediawiki deleted file mode 100644 index 197f94e..0000000 --- a/bip-330.mediawiki +++ /dev/null @@ -1,299 +0,0 @@ -
-  BIP: 330
-  Layer: Peer Services
-  Title: Transaction announcements reconciliation
-  Author: Gleb Naumenko 
-          Pieter Wuille 
-  Comments-Summary: ???
-  Comments-URI: ???
-  Status: Draft
-  Type: Standards Track
-  Created: 2010-00-00
-  License: CC0-1.0
-  License-Code: MIT
-
- -==Abstract== - -This document specifies a P2P protocol extension for reconciliation of transaction announcements between 2 nodes, which is a building block for efficient transaction relay protocols (e.g., [https://arxiv.org/pdf/1905.10518.pdf Erlay]). This is a step towards increasing the connectivity of the network for almost no bandwidth cost. - -==Motivation== - -Currently in the Bitcoin network, every 32-byte transaction ID is announced in at least one direction between every pair of connected peers, via INV messages. This results in high cost of announcing transactions: ''O(nodes * connections_per_node)''. - -A reconciliation-based protocol which uses the technique suggested in this document can have better scaling properties than INV-based flooding. - -Increasing the connectivity of the network makes the network more robust to partitioning attacks; thus, improving the bandwidth scaling of transaction relay to ''O(nodes)'' (and without a high constant overhead) would allow us to improve the security of the network by increasing connectivity. It would also reduce the bandwidth required to run a Bitcoin node and potentially enable more users to run full nodes. - -===Erlay=== - -[https://arxiv.org/pdf/1905.10518.pdf Erlay] is an example of a high-level transaction relay protocol which employs set reconciliation for bandwidth efficiency. - -Erlay uses both flooding (announcing using INV messages to all peers) and reconciliation to announce transactions. -Flooding is expensive, so Erlay seeks to use it sparingly and in strategic locations - only well-connected publicly reachable nodes flood transactions to other publicly reachable nodes via outbound connections. -Since every unreachable node is directly connected to several reachable nodes, this policy ensures that a transaction is quickly propagated to be within one hop from most of the nodes in the network. - -All transactions not propagated through flooding are propagated through efficient set reconciliation. -To do this, every node keeps a reconciliation set for each peer, in which transactions are placed which would have been announced using INV messages absent this protocol. Every 2 seconds every node chooses a peer from its outbound connections in a predetermined order to reconcile with, resulting in both sides learning the transactions known to the other side. After every reconciliation round, the corresponding reconciliation set is cleared. -A more detailed description of a set reconciliation round and other implementation details can be found in the paper. - -Erlay allows us to: -* save 40% of the bandwidth consumed by a node, given typical network connectivity as of July 2019. -* achieve similar latency -* increase network connectivity for almost no bandwidth or latency cost -* improves privacy as a side-effect - -This document proposes a P2P-layer extension which is required to enable efficient reconciliation-based protocols (like Erlay) for transaction relay. - -==Specification== - -===New data structures=== - -Several new data structures are introduced to the P2P protocol first, to aid with efficient transaction relay. - -====32-bit short transaction IDs==== - -During reconciliation, significantly abbreviated transaction IDs are used of just 32 bits in size. To prevent attackers from constructing sets of transactions that cause network-wide collisions, the short ID computation is salted on a per-link basis using 64 bits of entropy contributed by both communication partners. - -Short IDs are computed as follows: -* Let ''salt1'' and ''salt2'' be the entropy contributed by both sides; see the "sendrecon" message further for details how they are exchanged. -* Sort the two salts such that ''salt1 ≤ salt2'' (which side sent what doesn't matter). -* Compute ''h = SHA256("Tx Relay Salting" || salt1 || salt2)'', where the two salts are encoded in 64-bit little-endian byte order. -* Let ''k0'' be the 64-bit integer obtained by interpreting the first 8 bytes of ''h'' in little-endian byte order. -* Let ''k1'' be the 64-bit integer obtained by interpreting the second 8 bytes of ''h'' in little-endian byte order. -* Let ''s = SipHash-2-4((k0,k1),wtxid)'', where ''wtxid'' is the transaction hash including witness data as defined by BIP141. -* The short ID is equal to ''1 + (s mod 0xFFFFFFFF)''. - -This results in approximately uniformly distributed IDs in the range ''[1..0xFFFFFFFF]'', which is a requirement for using them as elements in 32-bit sketches. See the next paragraph for details. - -====Short transaction ID sketches==== - -Reconciliation-based relay uses [https://www.cs.bu.edu/~reyzin/code/fuzzy.html PinSketch] BCH-based secure sketches as introduced by the [https://www.cs.bu.edu/~reyzin/fuzzy.html Fuzzy Extractors paper]. They are a form of set checksums with the following properties: -* Sketches have a predetermined capacity, and when the number of elements in the set does not exceed the capacity, it is always possible to recover the entire set from the sketch by decoding the sketch. A sketch of nonzero b-bit elements with capacity c can be stored in bc bits. -* A sketch of the [https://en.wikipedia.org/wiki/Symmetric_difference symmetric difference] between the two sets (i.e., all elements that occur in one but not both input sets), can be obtained by combining the sketches of those sets. - -The sketches used here consists of elements of the [https://en.wikipedia.org/wiki/Finite_field finite field] ''GF(232)''. Specifically, we represent finite field elements as polynomials in ''x'' over ''GF(2)'' modulo ''x327 + x3 + x2 + 1''. To map integers to finite field elements, simply treat each bit ''i'' (with value ''2i'') in the integer as the coefficient of ''xi'' in the polynomial representation. For example the integer ''101 = 26 + 25 + 22 + 1'' is mapped to field element ''x6 + x5 + x2 + 1''. These field elements can be added and multiplied together, but the specifics of that are out of scope for this document. - -A short ID sketch with capacity ''c'' consists of a sequence of ''c'' field elements. The first is the sum of all short IDs in the set, the second is the sum of the 3rd powers of all short IDs, the third is the sum of the 5th powers etc., up to the last element with is the sum of the ''(2c-1)''th powers. These elements are then encoded as 32-bit integers in little endian byte order, resulting in a ''4c''-byte serialization. - -The following Python 3.2+ code implements the creation of sketches:
-FIELD_BITS = 32
-FIELD_MODULUS = (1 << FIELD_BITS) + 0b10001101
-
-def mul2(x):
-    """Compute 2*x in GF(2^FIELD_BITS)"""
-    return (x << 1) ^ (FIELD_MODULUS if x.bit_length() >= FIELD_BITS else 0)
-
-def mul(x, y):
-    """Compute x*y in GF(2^FIELD_BITS)"""
-    ret = 0
-    for bit in [(x >> i) & 1 for i in range(x.bit_length())]:
-        ret, y = ret ^ bit * y, mul2(y)
-    return ret
-
-def create_sketch(shortids, capacity):
-    """Compute the bytes of a sketch for given shortids and given capacity."""
-    odd_sums = [0 for _ in range(capacity)]
-    for shortid in shortids:
-        squared = mul(shortid, shortid)
-        for i in range(capacity):
-            odd_sums[i] ^= shortid
-            shortid = mul(shortid, squared)
-    return b''.join(elem.to_bytes(4, 'little') for elem in odd_sums)
-
- -The [https://github.com/sipa/minisketch/ minisketch] library implements the construction, merging, and decoding of these sketches efficiently. - -====Truncated transaction IDs==== - -For announcing and relaying transaction outside of reconciliation, we need an unambiguous, unsalted way to refer to transactions to deduplicate transaction requests. As we're introducing a new scheme anyway, this is a good opportunity to switch to wtxid-based requests rather than txid-based ones. While using full 256-bit wtxids is possible, this is overkill as they contribute significantly to the total bandwidth as well. Instead, we truncate the wtxid to just their first 128 bits. These are referred to as truncated IDs. - -===Intended Protocol Flow=== - -Set reconciliation primarily consists of the transmission and decoding of a reconciliation set sketch upon request. - -[[File:bip-0330/recon_scheme_merged.png|framed|center|Set reconciliation protocol flow]] - -====Bisection==== - -If a node is unable to reconstruct the set difference from the received sketch, the node then makes an additional reconciliation request, similar to the initial one, but this request is applied to only a fraction of possible transactions (e.g., in the range 0x0–0x8). Because of the linearity of sketches, a sketch of a subset of transactions would allow the node to compute a sketch for the remainder, which saves bandwidth. - -[[File:bip-0330/bisection.png|framed|300px|center|Bisection]] - -===New messages=== -Several new protocol messages are added: sendrecon, reqreconcil, sketch, reqbisec, reconcildiff, invtx, gettx. This section describes their serialization, contents, and semantics. - -In what follows, all integers are serialized in little-endian byte order. Boolean values are encoded as a single byte that must be 0 or 1 exactly. Arrays are serialized with the CompactSize prefix that encodes their length, as is common in other P2P messages. - -====sendrecon==== -The sendrecon message announces support for the reconciliation protocol. It is expected to be only sent once, and ignored by nodes that don't support it. - -Its payload consists of: -{|class="wikitable" -! Data type !! Name !! Description -|- -| bool || sender || Indicates whether the sender will send "reqreconcil" message -|- -| bool || responder || Indicates whether the sender will respond to "reqreconcil" messages. -|- -| uint32 || version || Sender must set this to 1 currently, otherwise receiver should ignore the message. -|- -| uint64 || salt || The salt used in the short transaction ID computation. -|} - -"reqreconcil" messages can only be sent if the sender has sent a "sendrecon" message with sender=true, and the receiver has sent a "sendrecon" message with responder=true. - -====reqreconcil==== -The reqreconcil message initiates a reconciliation round. - -{|class="wikitable" -! Data type !! Name !! Description -|- -| uint16 || set_size || Size of the sender's reconciliation set, used to estimate set difference. -|- -| uint8 || q || Coefficient used to estimate set difference. Multiplied by PRECISION=2^6 and rounded up by the sender and divided by PRECISION by the receiver. -|} - -Upon receipt of a "reqreconcil" message, the receiver: -* Constructs and sends a "sketch" message (see below), with a sketch of capacity computed as ''|set_size - local_set_size| + q * (set_size + local_set_size) + c'', where ''local_set_size'' represents size of the receiver's reconciliation set. -* Makes a snapshot of their current reconciliation set, and clears the set itself. The snapshot is kept until a "reconcildiff" message is received by the node. - -It is suggested to use ''c=1'' to avoid sending empty sketches and reduce the overhead caused by under-estimations. - -Intuitively, ''q'' represents the discrepancy in sets: the closer the sets are, the lower optimal ''q'' is. -As suggested by Erlay, ''q'' should be derived as an optimal ''q'' value for the previous reconciliation with a given peer, once the actual set sizes and set difference are known. Alternatively, ''q=0.1'' should be used as a default value. -For example, if in previous round ''set_size=30'' and ''local_set_size=20'', and the *actual* difference was ''4'', then a node should compute ''q'' as following: -''q=(|30-20| - 1) / (30+20)=0.18'' -The derivation of ''q'' can be changed according to the version of the protocol. - -No new "reqreconcil" message can be sent until a "reconcildiff" message is sent. - -====sketch==== -The sketch message is used to communicate a sketch required to perform set reconciliation. - -{|class="wikitable" -! Data type !! Name !! Description -|- -| byte[] || skdata || The sketch of the sender's reconciliation snapshot -|} - -Upon receipt of a "sketch" message, a node computes the set difference by combining the receiver sketch with a sketch computed locally for a corresponding reconciliation set. If this is the 2nd time for this round a "sketch" message was received, the bisection approach is used, and by combining the new sketch with the previous one, two difference sketches are obtained, one for the first half and one for the second half of the short id range. The receiving node then tries to decode this sketch (or sketches), and based on the result: -* If decoding fails, a "reconcildiff" message is sent with the failure flag set (success=false). If this was the first "sketch" in the round, a "reqbisec" message may be sent instead. -* If decoding succeeds, a "reconcildiff" message is sent with the truncated IDs of all locally known transactions that appear in the decode result, and the short IDs of the unrecognized ones. - -The receiver also makes snapshot of their current reconciliation set, and clears the set itself. The snapshot is kept until a "reconcildiff" message is sent by the node. - -====reqbisec==== -The reqbisec message is used to signal that set reconciliation has failed and an extra sketch is needed to find set difference. - -It has an empty payload. - -Upon receipt of a "reqbisec" message, a node responds to it with a "sketch" message, which contains a sketch of a subset of corresponding reconciliation set snapshot (stored when "reqreconcil" message for the current round was processed) (values in range ''[0..(2^31)]''). - -====reconcildiff==== -The reconcildiff message is used to announce transactions which are found to be missing during set reconciliation on the sender's side. - -{|class="wikitable" -! Data type !! Name !! Description -|- -| uint8 || success || Indicates whether sender of the message succeeded at set difference decoding. -|- -| uint32[] || ask_shortids || The short IDs that the sender did not have. -|} - -Upon receipt a "reconcildiff" message with ''success=1'', a node sends a "invtx" message for the transactions requested by 32-bit IDs (first vector) containing their 128-bit truncated IDs (with parent transactions occuring before their dependencies), and can request announced transactions (second vector) it does not have via a "gettx" message. -Otherwise if ''success=0'', receiver should request bisection via ''reqbisec'' (if failure happened for the first time). -If failure happened for the second time, receiver should announce the transactions from the reconciliation set via an "invtx" message, excluding the transactions announced from the sender. - -The snapshot of the corresponding reconciliation set is cleared by the sender and the receiver of the message. - -The sender should also send their own "invtx" message along with the reconcildiff message to announce transactions which are missing on the receiver's side. - -====invtx==== -The invtx message is used to announce transactions (both along with reconcildiff message and as a response to the reconcildiff message). It is the truncated ID analogue of "inv" (which cannot be used because it has 256-bit elements). - -{|class="wikitable" -! Data type !! Name !! Description -|- -| uint128[] || inv_truncids || The truncated IDs of transactions the sender believes the receiver does not have. -|} - -Upon receipt a "invtx" message, a node requests announced transactions it does not have. -The snapshot of the corresponding reconciliation set is cleared by the sender of the message. - -====gettx==== -The gettx message is used to request transactions by 128-bit truncated IDs. It is the truncated ID analogue of "getdata". - -{|class="wikitable" -! Data type !! Name !! Description -|- -| uint128[] || ask_truncids || The truncated IDs of transactions the sender wants the full transaction data for. -|} - -Upon receipt a "gettx" message, a node sends "tx" messages for the requested transactions. - -==Local state== - -This BIP suggests a stateful protocol and it requires storing several variables at every node to operate properly. - -====Reconciliation sets==== -Every node stores a set of 128-bit truncated IDs for every peer which supports transaction reconciliation, representing the transactions which would have been sent according to the regular flooding protocol. -Incoming transactions are added to sets when those transactions are received (if they satisfy the policies such as minimum fee set by a peer). -A reconciliation set is moved to the corresponding set snapshot after the transmission of the initial sketch. - -====Reconciliation set snapshot==== -After the transmitting of the initial sketch (either sending or receiving of reconcildiff message), every node should store the snapshot of the current reconciliation set, and clear the set. -This is important to make bisection more stable during the reconciliation round (bisection should be applied to the snapshot). -The snapshot is also used to efficiently lookup the transactions requested by short ID. -The snapshot is cleared after the end of the reconciliation round (sending or receiving of the reconcildiff message). - -====q-coefficient==== -The q value should be stored to make efficient difference estimation. It is shared across peers and changed after every reconciliation. -q-coefficient represents the discrepancy in sets: the closer the sets are, the lower optimal ''q'' is. -In future implementations, q could vary across different peers or become static. - - -==Backward compatibility== - -Older clients remain fully compatible and interoperable after this change. - -Clients which do not implement this protocol remain fully compatible after this change using existing protocols, because transaction announcement reconciliation is used only for peers that negotiate support for it. - -==Rationale== - -====Why using PinSketch for set reconciliation?==== - -PinSketch is more bandwidth efficient than IBLT, especially for the small differences in sets we expect to operate over. -PinSketch is as bandwidth efficient as CPISync, but PinSketch has quadratic decoding complexity, while CPISync have cubic decoding complexity. This makes PinSketch significantly faster. - -====Why using 32-bit short transaction IDs?==== - -To use Minisketch in practice, transaction IDs should be shortened (ideally, not more than 64 bits per element). -Small number of bits per transaction also allows to save extra bandwidth and make operations over sketches faster. -According to our estimates, 32 bits provides low collision rate in a non-adversarial model (which is enabled by using independent salts per-link). - -====Why using 128-bit short IDs?==== - -To avoid problems caused by the delays in the network, our protocol requires extra round of announcing unsalted transaction IDs. [https://arxiv.org/pdf/1905.10518.pdf Erlay] protocol on top of this work also requires announcing unsalted transaction IDs for flooding. -Both of these measures allow to deduplicate transaction announcements across the peers. -However, using full 256-bit IDs to uniquely identify transactions seems to be an overkill. -128 is the highest power of 2 which provides good enough collision-resistance in an adversarial model, and trivially saves a significant portion of the bandwidth related to these announcements. - -====Why using bisection instead of extending the sketch?==== - -Unlike extended sketches, bisection does not require operating over sketches of higher order. -This allows to avoid the high computational cost caused by quadratic decoding complexity. - -==Implementation== - -TODO - -==Acknowledgments== - -A large fraction of this proposal was done during designing Erlay with Gregory Maxwell, Sasha Fedorova and Ivan Beschastnikh. -We would like to thank Suhas Daftuar for contributions to the design and BIP structure. -We would like to thank Ben Woosley for contributions to the high-level description of the idea. - -==Copyright== - -This document is licensed under the Creative Commons CC0 1.0 Universal license. -- cgit v1.2.3 From affe5cb8810b4009e249b05abe5cb193b05d86a4 Mon Sep 17 00:00:00 2001 From: User Date: Tue, 5 Nov 2019 11:55:26 -0500 Subject: Add comments links and created date. --- bip-0330.mediawiki | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/bip-0330.mediawiki b/bip-0330.mediawiki index 197f94e..581b6ae 100644 --- a/bip-0330.mediawiki +++ b/bip-0330.mediawiki @@ -4,11 +4,11 @@ Title: Transaction announcements reconciliation Author: Gleb Naumenko Pieter Wuille - Comments-Summary: ??? - Comments-URI: ??? + Comments-Summary: No comments yet. + Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0330 Status: Draft Type: Standards Track - Created: 2010-00-00 + Created: 2019-09-25 License: CC0-1.0 License-Code: MIT
-- cgit v1.2.3 From 544e883488831d67b54f84214006eb8ad761f87b Mon Sep 17 00:00:00 2001 From: User Date: Tue, 5 Nov 2019 12:38:51 -0500 Subject: update readme --- README.mediawiki | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/README.mediawiki b/README.mediawiki index de98f05..b081c5e 100644 --- a/README.mediawiki +++ b/README.mediawiki @@ -910,6 +910,13 @@ Those proposing changes should consider that ultimately consent may rest with th | Karl-Johan Alm | Standard | Draft +|- +| [[bip-0330.mediawiki|330]] +| Peer Services +| Transaction announcements reconciliation +| Gleb Naumenko, Pieter Wuille +| Standard +| Draft |} -- cgit v1.2.3 From 2a270d9419a433efdc73928997baca2da04033b0 Mon Sep 17 00:00:00 2001 From: Karl-Johan Alm Date: Wed, 20 Mar 2019 10:27:47 +0900 Subject: BIP 325: Signet --- README.mediawiki | 7 +++++ bip-0325.mediawiki | 85 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 92 insertions(+) create mode 100644 bip-0325.mediawiki diff --git a/README.mediawiki b/README.mediawiki index 00a09be..e0cd1e1 100644 --- a/README.mediawiki +++ b/README.mediawiki @@ -911,6 +911,13 @@ Those proposing changes should consider that ultimately consent may rest with th | Standard | Draft |- +| [[bip-0325.mediawiki|325]] +| Applications +| Signet +| Karl-Johan Alm +| Standard +| Draft +|- | [[bip-0330.mediawiki|330]] | Peer Services | Transaction announcements reconciliation diff --git a/bip-0325.mediawiki b/bip-0325.mediawiki new file mode 100644 index 0000000..51ec634 --- /dev/null +++ b/bip-0325.mediawiki @@ -0,0 +1,85 @@ +
+  BIP: 325
+  Layer: Applications
+  Title: Signet
+  Author: Karl-Johan Alm 
+  Comments-Summary: No comments yet.
+  Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0325
+  Status: Draft
+  Type: Standards Track
+  Created: 2019-03-20
+  License: CC0-1.0
+
+ +== Abstract == + +A new type of test network where signatures are used in addition to proof of work for block progress, enabling much better coordination and robustness (be reliably unreliable), for persistent, longer-term testing scenarios involving multiple independent parties. + +== Motivation == + +Testnet is a great place to try out new things without risking real money, but it is notoriously unreliable. Huge block reorgs, long gaps in between blocks being mined or sudden bursts of blocks in rapid succession mean that realistic testing of software, especially involving multiple independent parties running software over an extended period of time, becomes infeasible in practice. + +A new type of test network would be more suitable for integration testing by organizations such as exchanges, or testing of next generation Layer-2 protocols like Eltoo or sidechain pegs. The goal is not to be perfectly reliable but rather to have a predictable amount of unreliability. You want a test network to behave like mainnet (i.e. no thousands of block reorgs) while also making it easier to trigger expected but rare events like a 6-block reorg. Regtest is not suitable for longer-term scenarios involving multiple independent parties because creating blocks costs nothing, so any party can completely control the test network. + + +== Specification == + +A new type of network ("signet"), which takes an additional consensus parameter called the challenge (scriptPubKey). The challenge can be a simple pubkey (P2PKH style), or a k-of-n multisig, or any other script you would want. + +The witness commitment of the coinbase transaction is extended to include a secondary commitment (the signature/solution): + + 1-4 bytes - Push the following (x + 4) bytes + 4 bytes - Signet header (0xecc7daa2) + x bytes - Solution (sigScript) + +Any push operations that do not start with the 4 byte signet header are ignored. Multiple push operations with the 4 byte signet header are ignored except for the first entry. + +Any signature operations contained within the challenge use SHA256d(modifiedBlockHash), i.e. the double-SHA256 digest of the following data as the sighash: + +{|class="wikitable" style="text-align: center;" +|- +!Type +!Size +!Name +|- +|Int32||4||nVersion +|- +|Uint256||32||hashPrevBlock +|- +|Uint256||32||modifiedMerkleRoot +|- +|Uint32||4||nTime +|- +|Uint32||4||nBits +|} + +The modifiedMerkleRoot hash is obtained by generating the merkle root of the block transactions, with the coinbase witness commitment as is, without the signet extension. This means the merkle root of the block is different from the merkle root in the signet commitment. This is needed, because the signature can never be included in the very message (in this case, a block) that is being signed. Apart from the signature, to facilitate block generation (mining), the block nonce value is the only other component of the block that the signet signature does not commit to. When grinding proof of work, the extended nonce cannot be used as it would invalidate the signature. Instead, simply resigning the same (or an updated) block will give a new search space. + +A block is considered fully validated if the above commitment is found, and its solution is valid. It is recommended that this verification is done directly before or after the witness commitment verification, as the data required to do both is approximately the same. + +== Compatibility == + +This specification is backwards compatible in the sense that existing software can use Signet out of the box. + +Simply by adding the network parameters for signet (magic number, etc), a client can connect to and use any signet network without further modifications. The block headers have valid proof of work, so clients can trivially check that blocks are "probably" valid. + +However, anyone can mine blocks that are accepted by the client for any given signet network. These blocks do not contain the required signatures, however, so any fully validating node will promptly reject them. As such, clients need to either validate the block signature inside the coinbase transaction, or connect to trusted peers. + +Other software need not add block signature validation code that they will not use in production. This is adequate for non-production test purposes where the goal is to have a network behave as much like mainnet as possible. + +== Reference implementation == + +Pull request at https://github.com/bitcoin/bitcoin/pull/16411 + +== Acknowledgements == + +TODO + +== References == + +# Original mailing list thread: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2019-March/016734.html +# Bitcoin Wiki entry: https://en.bitcoin.it/wiki/Signet + +== Copyright == + +This document is licensed under the Creative Commons CC0 1.0 Universal license. -- cgit v1.2.3 From d3ff4b1e9e3358893c5c5c5c2b5b1642f65ea98c Mon Sep 17 00:00:00 2001 From: Emil Engler Date: Thu, 17 Oct 2019 21:26:08 +0200 Subject: Add BIP 179: Name for payment recipient identifiers --- README.mediawiki | 7 +++++++ bip-0179.mediawiki | 58 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 65 insertions(+) create mode 100644 bip-0179.mediawiki diff --git a/README.mediawiki b/README.mediawiki index 3e2927e..681b57c 100644 --- a/README.mediawiki +++ b/README.mediawiki @@ -855,6 +855,13 @@ Those proposing changes should consider that ultimately consent may rest with th | Standard | Draft |- +| [[bip-0179.mediawiki|179]] +| +| Name for payment recipient identifiers +| Emil Engler, MarcoFalke, Luke Dashjr +| Informational +| Draft +|- | [[bip-0180.mediawiki|180]] | Peer Services | Block size/weight fraud proof diff --git a/bip-0179.mediawiki b/bip-0179.mediawiki new file mode 100644 index 0000000..7894f2d --- /dev/null +++ b/bip-0179.mediawiki @@ -0,0 +1,58 @@ +
+  BIP: 179
+  Title: Name for payment recipient identifiers
+  Author: Emil Engler 
+          MarcoFalke 
+          Luke Dashjr 
+  Comments-Summary: No comments yet.
+  Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0179
+  Status: Draft
+  Type: Informational
+  Created: 2019-10-17
+  License: CC0-1.0
+
+ +==Abstract== +This BIP proposes a new term for 'address' + +==Specification== +The new term is: +''Bitcoin'' '''Invoice''' ''Address'' + +The ''Bitcoin'' and ''Address'' parts are optional. +The address suffix should only be used as a transitional step. + +A ''Bitcoin'' Invoice ''Address'' is a string of characters that can be used to indicate the intended recipient and purpose of a transaction. + +==Motivation== +Bitcoin addresses are intended to be only used '''once''' and you should generate a new one for every new incoming payment. +The term 'address' however indicates consistency because nearly everything on the internet or the offline world with the term 'address' +is something that rarely or even never changes (postal address, email address, IP addresses (depends heavily on the provider), etc.) +The motivation for this BIP is to change the term address to something that indicates that the address is connected to a single transaction. + +==Rationale== +The reason why we use ''Bitcoin Invoice Address'' or just ''Invoice'' is to emphasize that it is single-use. +The terms ''Bitcoin'' and ''Address'' are optional for the following reasons: +For ''Bitcoin'': +* Useful for multicoin wallets to indicate that it belongs to Bitcoin +* Indicates a difference between a lightning and an on-chain invoice +For ''Address'': +* To not confuse users with a completely new term +* To show that it is where you send something to +* To not break backwards compatibility + +This gives us the four following possibilities: +* Bitcoin Invoice Address +* Bitcoin Invoice +* Invoice Address +* Invoice + +==Backwards Compatibility== +To avoid issues, the 'Address' suffix is permitted, but not recommended. +The suffix 'Address' remains so users should be immediately able to recognize it until the new term is widely known. + +==Acknowledgements== +Thanks to Chris Belcher for the suggestion of the term 'Bitcoin Invoice Address' + +==Copyright== +This BIP is released under CC0-1.0 and therefore Public Domain. -- cgit v1.2.3 From 87a85402ae23f087496883c757a066f4857b13f8 Mon Sep 17 00:00:00 2001 From: azuchi Date: Thu, 14 Nov 2019 11:53:05 +0900 Subject: BIP174: Fix wrong description about Proprietary Use Type --- bip-0174.mediawiki | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/bip-0174.mediawiki b/bip-0174.mediawiki index 49bc060..e47c0af 100644 --- a/bip-0174.mediawiki +++ b/bip-0174.mediawiki @@ -132,7 +132,7 @@ The currently defined global types are as follows: ** Value: The 32-bit little endian unsigned integer representing the version number of this PSBT. If ommitted, the version number is 0. *** {32-bit int} -* Type: Version Number PSBT_GLOBAL_PROPRIETARY = 0xFC +* Type: Proprietary Use Type PSBT_GLOBAL_PROPRIETARY = 0xFC ** Key: Variable length identifier prefix, followed by a subtype, followed by the key data itself. *** {0xFC}||{subtype}|{key data} ** Value: Any value data as defined by the proprietary type user. @@ -200,7 +200,7 @@ The currently defined per-input types are defined as follows: ** Value: The UTF-8 encoded commitment message string for the proof-of-reserves. See [[bip-0127.mediawiki|BIP 127]] for more information. *** {porCommitment} -* Type: Version Number PSBT_INPUT_PROPRIETARY = 0xFC +* Type: Proprietary Use Type PSBT_INPUT_PROPRIETARY = 0xFC ** Key: Variable length identifier prefix, followed by a subtype, followed by the key data itself. *** {0xFC}||{subtype}|{key data} ** Value: Any value data as defined by the proprietary type user. @@ -228,7 +228,7 @@ determine which outputs are change outputs and verify that the change is returni ** Value: The master key fingerprint concatenated with the derivation path of the public key. The derivation path is represented as 32 bit unsigned integer indexes concatenated with each other. Public keys are those needed to spend this output. *** {master key fingerprint}|{32-bit int}|...|{32-bit int} -* Type: Version Number PSBT_OUTPUT_PROPRIETARY = 0xFC +* Type: Proprietary Use Type PSBT_OUTPUT_PROPRIETARY = 0xFC ** Key: Variable length identifier prefix, followed by a subtype, followed by the key data itself. *** {0xFC}||{subtype}|{key data} ** Value: Any value data as defined by the proprietary type user. -- cgit v1.2.3 From 267c02a4b5c5550309b84bd60b6f72d9989ffb9b Mon Sep 17 00:00:00 2001 From: Dmitry Petukhov Date: Mon, 9 Dec 2019 12:21:09 +0500 Subject: BIP174: remove 'first byte is the type' comment for key data As the key type is now defined as compact size integer, `At the beginning of each key is a compact size unsigned integer representing the type`, the comment in the first table in the document, about first byte of the key being the key type is no longer accurate. As the structure of the key data is described further in the text after the table, and the comment that it starts with the compact size integer seems a bit long to be in that table, I think it is better to just remove the comment about the key data structure from the table, and leave the explanation to the text after the table. --- bip-0174.mediawiki | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bip-0174.mediawiki b/bip-0174.mediawiki index 49bc060..b964c81 100644 --- a/bip-0174.mediawiki +++ b/bip-0174.mediawiki @@ -69,7 +69,7 @@ the length of that data. {..} indicates the raw data itself. |- | Key | byte[] -| The key itself with the first byte being the type of the key-value pair +| The Key itself |- | Value Length | Compact Size Unsigned Integer -- cgit v1.2.3 From c7191c935e253506d6710d73b3d721f7b66ae371 Mon Sep 17 00:00:00 2001 From: Ben Carman Date: Mon, 9 Dec 2019 01:44:43 -0600 Subject: Specify 32 bit itns as unsigned and their endianess --- bip-0174.mediawiki | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/bip-0174.mediawiki b/bip-0174.mediawiki index 49bc060..c5c069b 100644 --- a/bip-0174.mediawiki +++ b/bip-0174.mediawiki @@ -123,14 +123,14 @@ The currently defined global types are as follows: * Type: Extended Public Key PSBT_GLOBAL_XPUB = 0x01 ** Key: The type followed by the 78 byte serialized extended public key as defined by BIP 32. Extended public keys are those that can be used to derive public keys used in the inputs and outputs of this transaction. It should be the public key at the highest hardened derivation index so that the unhardened child keys used in the transaction can be derived. *** {0x01}|{xpub} -** Value: The master key fingerprint as defined by BIP 32 concatenated with the derivation path of the public key. The derivation path is represented as 32 bit unsigned integer indexes concatenated with each other. The number of 32 bit unsigned integer indexes must match the depth provided in the extended public key. -*** {master key fingerprint}|{32-bit int}|...|{32-bit int} +** Value: The master key fingerprint as defined by BIP 32 concatenated with the derivation path of the public key. The derivation path is represented as 32-bit little endian unsigned integer indexes concatenated with each other. The number of 32 bit unsigned integer indexes must match the depth provided in the extended public key. +*** {master key fingerprint}|{32-bit uint}|...|{32-bit uint} * Type: Version Number PSBT_GLOBAL_VERSION = 0xFB ** Key: None. The key must only contain the 1 byte type. *** {0xFB} ** Value: The 32-bit little endian unsigned integer representing the version number of this PSBT. If ommitted, the version number is 0. -*** {32-bit int} +*** {32-bit uint} * Type: Version Number PSBT_GLOBAL_PROPRIETARY = 0xFC ** Key: Variable length identifier prefix, followed by a subtype, followed by the key data itself. @@ -180,7 +180,7 @@ The currently defined per-input types are defined as follows: ** Key: The public key *** {0x06}|{public key} ** Value: The master key fingerprint as defined by BIP 32 concatenated with the derivation path of the public key. The derivation path is represented as 32 bit unsigned integer indexes concatenated with each other. Public keys are those that will be needed to sign this input. -*** {master key fingerprint}|{32-bit int}|...|{32-bit int} +*** {master key fingerprint}|{32-bit uint}|...|{32-bit uint} * Type: Finalized scriptSig PSBT_IN_FINAL_SCRIPTSIG = 0x07 ** Key: None. The key must only contain the 1 byte type. @@ -225,8 +225,8 @@ determine which outputs are change outputs and verify that the change is returni * Type: BIP 32 Derivation Path PSBT_OUT_BIP32_DERIVATION = 0x02 ** Key: The public key *** {0x02}|{public key} -** Value: The master key fingerprint concatenated with the derivation path of the public key. The derivation path is represented as 32 bit unsigned integer indexes concatenated with each other. Public keys are those needed to spend this output. -*** {master key fingerprint}|{32-bit int}|...|{32-bit int} +** Value: The master key fingerprint concatenated with the derivation path of the public key. The derivation path is represented as 32-bit little endian unsigned integer indexes concatenated with each other. Public keys are those needed to spend this output. +*** {master key fingerprint}|{32-bit uint}|...|{32-bit uint} * Type: Version Number PSBT_OUTPUT_PROPRIETARY = 0xFC ** Key: Variable length identifier prefix, followed by a subtype, followed by the key data itself. -- cgit v1.2.3 From 65f0b3dd62ecc55e43436173c5f84e893809d0fa Mon Sep 17 00:00:00 2001 From: Dmitry Petukhov Date: Mon, 9 Dec 2019 17:30:47 +0500 Subject: BIP-174: test data: fix value length In the test case "Case: PSBT With invalid output witnessScript typed key", after PSBT_OUT_WITNESS_SCRIPT key with garbage data (which ends with `...478ef51309d`, follows value `2b` which would denote the length of the data value of the key. But the length of actual remaining data is only 7 bytes. Thus, an implementation that reads key-value pairs and checks for validity of the key data after it has read the current key-value pair, will not be able to hit the exact condition intended for this test case: extra data within the key itself. This is because such implementation will hit serialization error when it will try to read the data of the value and will get the short read. Reading full key-value pair and then checking key format afterwards is fairly normal thing to do, as the format of the keys with all their meaning is an abstraction of higher level than just the simple key-value serialization format. The proposed change is to replace byte `2b` after the key data to `06` and thus make the value length in the key-value pair valid (not going beyond the end of the data). base64 encoding has been changed accordingly. --- bip-0174.mediawiki | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/bip-0174.mediawiki b/bip-0174.mediawiki index 49bc060..56d7eb2 100644 --- a/bip-0174.mediawiki +++ b/bip-0174.mediawiki @@ -573,8 +573,8 @@ The following are invalid PSBTs: ** Base64 String:
cHNidP8BAHMCAAAAATAa6YblFqHsisW0vGVz0y+DtGXiOtdhZ9aLOOcwtNvbAAAAAAD/////AnR7AQAAAAAAF6kUA6oXrogrXQ1Usl1jEE5P/s57nqKHYEOZOwAAAAAXqRS5IbG6b3IuS/qDtlV6MTmYakLsg4cAAAAAAAEBHwDKmjsAAAAAFgAU0tlLZK4IWH7vyO6xh8YB6Tn5A3wAAgAAFgAUYunpgv/zTdgjlhAxawkM0qO3R8sAAQAiACCHa62DLx0WgBXtQSMqnqZaGBXZ7xPA74dZ9ktbKyeKZQEBJVEhA7fOI6AcW0vwCmQlN836uzFbZoMyhnR471EwnSvVf4qHUa4A
* Case: PSBT With invalid output witnessScript typed key -** Bytes in Hex:
70736274ff0100730200000001301ae986e516a1ec8ac5b4bc6573d32f83b465e23ad76167d68b38e730b4dbdb0000000000ffffffff02747b01000000000017a91403aa17ae882b5d0d54b25d63104e4ffece7b9ea2876043993b0000000017a914b921b1ba6f722e4bfa83b6557a3139986a42ec8387000000000001011f00ca9a3b00000000160014d2d94b64ae08587eefc8eeb187c601e939f9037c00010016001462e9e982fff34dd8239610316b090cd2a3b747cb000100220020876bad832f1d168015ed41232a9ea65a1815d9ef13c0ef8759f64b5b2b278a6521010025512103b7ce23a01c5b4bf00a642537cdfabb315b668332867478ef51309d2bd57f8a8751ae00
-** Base64 String:
cHNidP8BAHMCAAAAATAa6YblFqHsisW0vGVz0y+DtGXiOtdhZ9aLOOcwtNvbAAAAAAD/////AnR7AQAAAAAAF6kUA6oXrogrXQ1Usl1jEE5P/s57nqKHYEOZOwAAAAAXqRS5IbG6b3IuS/qDtlV6MTmYakLsg4cAAAAAAAEBHwDKmjsAAAAAFgAU0tlLZK4IWH7vyO6xh8YB6Tn5A3wAAQAWABRi6emC//NN2COWEDFrCQzSo7dHywABACIAIIdrrYMvHRaAFe1BIyqeploYFdnvE8Dvh1n2S1srJ4plIQEAJVEhA7fOI6AcW0vwCmQlN836uzFbZoMyhnR471EwnSvVf4qHUa4A
+** Bytes in Hex:
70736274ff0100730200000001301ae986e516a1ec8ac5b4bc6573d32f83b465e23ad76167d68b38e730b4dbdb0000000000ffffffff02747b01000000000017a91403aa17ae882b5d0d54b25d63104e4ffece7b9ea2876043993b0000000017a914b921b1ba6f722e4bfa83b6557a3139986a42ec8387000000000001011f00ca9a3b00000000160014d2d94b64ae08587eefc8eeb187c601e939f9037c00010016001462e9e982fff34dd8239610316b090cd2a3b747cb000100220020876bad832f1d168015ed41232a9ea65a1815d9ef13c0ef8759f64b5b2b278a6521010025512103b7ce23a01c5b4bf00a642537cdfabb315b668332867478ef51309d06d57f8a8751ae00
+** Base64 String:
cHNidP8BAHMCAAAAATAa6YblFqHsisW0vGVz0y+DtGXiOtdhZ9aLOOcwtNvbAAAAAAD/////AnR7AQAAAAAAF6kUA6oXrogrXQ1Usl1jEE5P/s57nqKHYEOZOwAAAAAXqRS5IbG6b3IuS/qDtlV6MTmYakLsg4cAAAAAAAEBHwDKmjsAAAAAFgAU0tlLZK4IWH7vyO6xh8YB6Tn5A3wAAQAWABRi6emC//NN2COWEDFrCQzSo7dHywABACIAIIdrrYMvHRaAFe1BIyqeploYFdnvE8Dvh1n2S1srJ4plIQEAJVEhA7fOI6AcW0vwCmQlN836uzFbZoMyhnR471EwnQbVf4qHUa4A
The following are valid PSBTs: -- cgit v1.2.3 From e097b1d38adc8643a8e0b1bb00e94a97b175314e Mon Sep 17 00:00:00 2001 From: Ben Carman Date: Wed, 11 Dec 2019 15:53:06 -0600 Subject: BIP 174: Specify that separator only appears at end of the map --- bip-0174.mediawiki | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bip-0174.mediawiki b/bip-0174.mediawiki index 49bc060..b14efe2 100644 --- a/bip-0174.mediawiki +++ b/bip-0174.mediawiki @@ -103,7 +103,7 @@ The format of each key-value map is as follows: | separator | char | 0x00 -| Must be 0x00. +| Must be 0x00 at the end of the map. |} At the beginning of each key is a compact size unsigned integer representing the type. -- cgit v1.2.3 From 8faf97e720f0800953e71cf156c899b1446ff455 Mon Sep 17 00:00:00 2001 From: Dmitry Petukhov Date: Sat, 14 Dec 2019 20:39:40 +0500 Subject: BIP-174: add missing types to Appendix A; fix proprietary type names PSBT_INPUT_PROPRIETARY -> PSBT_IN_PROPRIETARY PSBT_OUTPUT_PROPRIETARY -> PSBT_OUT_PROPRIETARY to be consistent with other in/out type names that use shortened `IN` and `OUT` --- bip-0174.mediawiki | 29 +++++++++++++++++++++++++++-- 1 file changed, 27 insertions(+), 2 deletions(-) diff --git a/bip-0174.mediawiki b/bip-0174.mediawiki index d95b119..3121012 100644 --- a/bip-0174.mediawiki +++ b/bip-0174.mediawiki @@ -200,7 +200,7 @@ The currently defined per-input types are defined as follows: ** Value: The UTF-8 encoded commitment message string for the proof-of-reserves. See [[bip-0127.mediawiki|BIP 127]] for more information. *** {porCommitment} -* Type: Proprietary Use Type PSBT_INPUT_PROPRIETARY = 0xFC +* Type: Proprietary Use Type PSBT_IN_PROPRIETARY = 0xFC ** Key: Variable length identifier prefix, followed by a subtype, followed by the key data itself. *** {0xFC}||{subtype}|{key data} ** Value: Any value data as defined by the proprietary type user. @@ -228,7 +228,7 @@ determine which outputs are change outputs and verify that the change is returni ** Value: The master key fingerprint concatenated with the derivation path of the public key. The derivation path is represented as 32-bit little endian unsigned integer indexes concatenated with each other. Public keys are those needed to spend this output. *** {master key fingerprint}|{32-bit uint}|...|{32-bit uint} -* Type: Proprietary Use Type PSBT_OUTPUT_PROPRIETARY = 0xFC +* Type: Proprietary Use Type PSBT_OUT_PROPRIETARY = 0xFC ** Key: Variable length identifier prefix, followed by a subtype, followed by the key data itself. *** {0xFC}||{subtype}|{key data} ** Value: Any value data as defined by the proprietary type user. @@ -748,6 +748,21 @@ Any data types, their associated scope and BIP number must be defined here | PSBT_GLOBAL_UNSIGNED_TX | BIP 174 |- +| Global +| 1 +| PSBT_GLOBAL_XPUB +| BIP 174 +|- +| Global +| 251 +| PSBT_GLOBAL_VERSION +| BIP 174 +|- +| Global +| 252 +| PSBT_GLOBAL_PROPRIETARY +| BIP 174 +|- | Input | 0 | PSBT_IN_NON_WITNESS_UTXO @@ -798,6 +813,11 @@ Any data types, their associated scope and BIP number must be defined here | PSBT_IN_POR_COMMITMENT | [[bip-0127.mediawiki|BIP 127]] |- +| Input +| 252 +| PSBT_IN_PROPRIETARY +| BIP 174 +|- | Output | 0 | PSBT_OUT_REDEEM_SCRIPT @@ -812,4 +832,9 @@ Any data types, their associated scope and BIP number must be defined here | 2 | PSBT_OUT_BIP32_DERIVATION | BIP 174 +|- +| Output +| 252 +| PSBT_OUT_PROPRIETARY +| BIP 174 |} -- cgit v1.2.3