Merge #16248: Make whitebind/whitelist permissions more flexible

c5b404e8f1973afe071a07c63ba1038eefe13f0f Add functional tests for flexible whitebind/list (nicolas.dorier)
d541fa391844f658bd7035659b5b16695733dd56 Replace the use of fWhitelisted by permission checks (nicolas.dorier)
ecd5cf7ea4c3644a30092100ffc399e30e193275 Do not disconnect peer for asking mempool if it has NO_BAN permission (nicolas.dorier)
e5b26deaaa6842f7dd7c4537ede000f965ea0189 Make whitebind/whitelist permissions more flexible (nicolas.dorier)

Pull request description:

  # Motivation

  In 0.19, bloom filter will be disabled by default. I tried to make [a PR](https://github.com/bitcoin/bitcoin/pull/16176) to enable bloom filter for whitelisted peers regardless of `-peerbloomfilters`.

  Bloom filter have non existent privacy and server can omit filter's matches. However, both problems are completely irrelevant when you connect to your own node. If you connect to your own node, bloom filters are the most bandwidth efficient way to synchronize your light client without the need of some middleware like Electrum.

  It is also a superior alternative to BIP157 as it does not require to maintain an additional index and it would work well on pruned nodes.

  When I attempted to allow bloom filters for whitelisted peer, my proposal has been NACKed in favor of [a more flexible approach](https://github.com/bitcoin/bitcoin/pull/16176#issuecomment-500762907) which should allow node operator to set fine grained permissions instead of a global `whitelisted` attribute.

  Doing so will also make follow up idea very easy to implement in a backward compatible way.

  # Implementation details

  The PR propose a new format for `--white{list,bind}`. I added a way to specify permissions granted to inbound connection matching `white{list,bind}`.

  The following permissions exists:
  * ForceRelay
  * Relay
  * NoBan
  * BloomFilter
  * Mempool

  Example:
  * `-whitelist=bloomfilter@127.0.0.1/32`.
  * `-whitebind=bloomfilter,relay,noban@127.0.0.1:10020`.

  If no permissions are specified, `NoBan | Mempool` is assumed. (making this PR backward compatible)

  When we receive an inbound connection, we calculate the effective permissions for this peer by fetching the permissions granted from `whitelist`  and add to it the permissions granted from `whitebind`.

  To keep backward compatibility, if no permissions are specified in `white{list,bind}` (e.g. `--whitelist=127.0.0.1`) then parameters `-whitelistforcerelay` and `-whiterelay` will add the permissions `ForceRelay` and `Relay` to the inbound node.

  `-whitelistforcerelay` and `-whiterelay` are ignored if the permissions flags are explicitly set in `white{bind,list}`.

  # Follow up idea

  Based on this PR, other changes become quite easy to code in a trivially review-able, backward compatible way:

  * Changing `connect` at rpc and config file level to understand the permissions flags.
  * Changing the permissions of a peer at RPC level.

ACKs for top commit:
  laanwj:
    re-ACK c5b404e8f1973afe071a07c63ba1038eefe13f0f

Tree-SHA512: adfefb373d09e68cae401247c8fc64034e305694cdef104bdcdacb9f1704277bd53b18f52a2427a5cffdbc77bda410d221aed252bc2ece698ffbb9cf1b830577

3 files changed, 99 insertions, 0 deletions

diff --git a/test/functional/p2p_permissions.py b/test/functional/p2p_permissions.py
new file mode 100644
index 0000000000..1013055420
--- /dev/null
+++ b/test/functional/p2p_permissions.py
@@ -0,0 +1,97 @@
+#!/usr/bin/env python3
+# Copyright (c) 2015-2018 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test p2p permission message.
+
+Test that permissions are correctly calculated and applied
+"""
+
+from test_framework.test_node import ErrorMatch
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import (
+    assert_equal,
+    connect_nodes,
+    p2p_port,
+)
+
+class P2PPermissionsTests(BitcoinTestFramework):
+    def set_test_params(self):
+        self.num_nodes = 2
+        self.setup_clean_chain = True
+        self.extra_args = [[],[]]
+
+    def run_test(self):
+        self.checkpermission(
+        # relay permission added
+        ["-whitelist=127.0.0.1", "-whitelistrelay"],
+        ["relay", "noban", "mempool"],
+        True)
+
+        self.checkpermission(
+        # forcerelay and relay permission added
+        # Legacy parameter interaction which set whitelistrelay to true
+        # if whitelistforcerelay is true
+        ["-whitelist=127.0.0.1", "-whitelistforcerelay"],
+        ["forcerelay", "relay", "noban", "mempool"],
+        True)
+
+        # Let's make sure permissions are merged correctly
+        # For this, we need to use whitebind instead of bind
+        # by modifying the configuration file.
+        ip_port = "127.0.0.1:{}".format(p2p_port(1))
+        self.replaceinconfig(1, "bind=127.0.0.1", "whitebind=bloomfilter,forcerelay@" + ip_port)
+        self.checkpermission(
+        ["-whitelist=noban@127.0.0.1" ],
+        # Check parameter interaction forcerelay should activate relay
+        ["noban", "bloomfilter", "forcerelay", "relay" ],
+        False)
+        self.replaceinconfig(1, "whitebind=bloomfilter,forcerelay@" + ip_port, "bind=127.0.0.1")
+
+        self.checkpermission(
+        # legacy whitelistrelay should be ignored
+        ["-whitelist=noban,mempool@127.0.0.1", "-whitelistrelay"],
+        ["noban", "mempool"],
+        False)
+
+        self.checkpermission(
+        # legacy whitelistforcerelay should be ignored
+        ["-whitelist=noban,mempool@127.0.0.1", "-whitelistforcerelay"],
+        ["noban", "mempool"],
+        False)
+
+        self.checkpermission(
+        # missing mempool permission to be considered legacy whitelisted
+        ["-whitelist=noban@127.0.0.1"],
+        ["noban"],
+        False)
+
+        self.checkpermission(
+        # all permission added
+        ["-whitelist=all@127.0.0.1"],
+        ["forcerelay", "noban", "mempool", "bloomfilter", "relay"],
+        False)
+
+        self.stop_node(1)
+        self.nodes[1].assert_start_raises_init_error(["-whitelist=oopsie@127.0.0.1"], "Invalid P2P permission", match=ErrorMatch.PARTIAL_REGEX)
+        self.nodes[1].assert_start_raises_init_error(["-whitelist=noban@127.0.0.1:230"], "Invalid netmask specified in", match=ErrorMatch.PARTIAL_REGEX)
+        self.nodes[1].assert_start_raises_init_error(["-whitebind=noban@127.0.0.1/10"], "Cannot resolve -whitebind address", match=ErrorMatch.PARTIAL_REGEX)
+
+    def checkpermission(self, args, expectedPermissions, whitelisted):
+        self.restart_node(1, args)
+        connect_nodes(self.nodes[0], 1)
+        peerinfo = self.nodes[1].getpeerinfo()[0]
+        assert_equal(peerinfo['whitelisted'], whitelisted)
+        assert_equal(len(expectedPermissions), len(peerinfo['permissions']))
+        for p in expectedPermissions:
+            if not p in peerinfo['permissions']:
+                raise AssertionError("Expected permissions %r is not granted." % p)
+
+    def replaceinconfig(self, nodeid, old, new):
+        with open(self.nodes[nodeid].bitcoinconf, encoding="utf8") as f:
+            newText=f.read().replace(old, new)
+        with open(self.nodes[nodeid].bitcoinconf, 'w', encoding="utf8") as f:
+            f.write(newText)
+
+if __name__ == '__main__':
+    P2PPermissionsTests().main()
diff --git a/test/functional/test_framework/test_node.py b/test/functional/test_framework/test_node.py
index cac5281764..df027397d2 100755
--- a/test/functional/test_framework/test_node.py
+++ b/test/functional/test_framework/test_node.py
@@ -68,6 +68,7 @@ class TestNode():
 
         self.index = i
         self.datadir = datadir
+        self.bitcoinconf = os.path.join(self.datadir, "bitcoin.conf")
         self.stdout_dir = os.path.join(self.datadir, "stdout")
         self.stderr_dir = os.path.join(self.datadir, "stderr")
         self.chain = chain
diff --git a/test/functional/test_runner.py b/test/functional/test_runner.py
index 79efa6131c..1742a8e319 100755
--- a/test/functional/test_runner.py
+++ b/test/functional/test_runner.py
@@ -200,6 +200,7 @@ BASE_SCRIPTS = [
     'rpc_scantxoutset.py',
     'feature_logging.py',
     'p2p_node_network_limited.py',
+    'p2p_permissions.py',
     'feature_blocksdir.py',
     'feature_config_args.py',
     'rpc_help.py',