Merge bitcoin/bitcoin#22235: script: add script to generate example bitcoin.conf

b42643c2537ffbe99d6d94fb1fb3b7f9d5234f93 doc: update init.cpp -conf help text (josibake)
970b9987ad5bcb72e581c40a7cdd408d94a48c81 doc: update devtools, release-process readmes (josibake)
50635d27b45d125b6264ac2abfbd6a1129c7228f build: include bitcoin.conf in build outputs (josibake)
6aac946f49aea243de1dc50631bb72f0186bbf58 doc: update bitcoin-conf.md (Josiah Baker)
1c7e820ded0846ef6ab4be9616b0de452336ef64 script: add script to generate example bitcoin.conf (josibake)
b483084d866c16d97a34251ae652bac94f85f61d doc: replace bitcoin.conf with placeholder file (josibake)

Pull request description:

  create a script for parsing the output from `bitcoind --help` to create an example conf file for new users

  ## problem

  per #10746 , `bitcoin.conf` not being put into the data directory during installation causes some confusion for users when running bitcoin. in the discussion on the issue, one proposed solution was to have an example config file and instruct users to `cp` it into their data directory after startup. in addition to #10746 , there have been other requests for a "skeleton config file" (https://github.com/bitcoin/bitcoin/issues/19641) to help users get started with configuring bitcoind.

  the main issue with an example config file is that it creates a second source of truth regarding what options are available for configuring bitcoind. this means any changes to the options (including the addition or removal of options) would have to be updated for the command line and also updated in the example file.

  this PR addresses this issue by providing a script to generate an example file directly from the `bitcoind --help` on-demand by running `contrib/devtools/gen-bitcoin-conf.sh`. this solution was originally proposed on #10746 and would also solve #19641 . this guarantees any changes made to the command-line options or the command-line options help would also be reflected in the example file after compiling and running the script.

  the main purpose of this script is to generate a config file to be included with releases, same as `gen-manpages.sh`. this ensures every release also includes an up-to-date, full example config file for users to edit. the script is also available for users who compile from source for generating an example config for their compiled binary.

  ## special considerations

  this removes the `bitcoin.conf` example file from the repo as it is now generated by this script. the original example file did contain extra text related to how to use certain options but going forward all option help docs should be moved into `init.cpp`

  this also edits `init.cpp` to have the option help indicate that `-conf` is not usable from the config file. this is similar to how `-includeconf` 's help indicates it cannot be used from the command line

ACKs for top commit:
  laanwj:
    Tested and code review ACK b42643c2537ffbe99d6d94fb1fb3b7f9d5234f93

Tree-SHA512: 4546e0cef92aa1398da553294ce4712d02e616dd72dcbe0b921af474e54f24750464ec813661f1283802472d1e8774e634dd1cc26fbf1f13286d3e0406c02c09

7 files changed, 114 insertions, 193 deletions

diff --git a/contrib/devtools/README.md b/contrib/devtools/README.md
index 79b0134adc..54b1a85588 100644
--- a/contrib/devtools/README.md
+++ b/contrib/devtools/README.md
@@ -90,6 +90,21 @@ example:
 BUILDDIR=$PWD/build contrib/devtools/gen-manpages.py
 ```
 
+gen-bitcoin-conf.sh
+===================
+
+Generates a bitcoin.conf file in `share/examples/` by parsing the output from `bitcoind --help`. This script is run during the
+release process to include a bitcoin.conf with the release binaries and can also be run by users to generate a file locally.
+When generating a file as part of the release process, make sure to commit the changes after running the script.
+
+With in-tree builds this tool can be run from any directory within the
+repository. To use this tool with out-of-tree builds set `BUILDDIR`. For
+example:
+
+```bash
+BUILDDIR=$PWD/build contrib/devtools/gen-bitcoin-conf.sh
+```
+
 security-check.py and test-security-check.py
 ============================================
 
diff --git a/contrib/devtools/gen-bitcoin-conf.sh b/contrib/devtools/gen-bitcoin-conf.sh
new file mode 100755
index 0000000000..2ebbd42022
--- /dev/null
+++ b/contrib/devtools/gen-bitcoin-conf.sh
@@ -0,0 +1,83 @@
+#!/usr/bin/env bash
+# Copyright (c) 2021 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+
+export LC_ALL=C
+TOPDIR=${TOPDIR:-$(git rev-parse --show-toplevel)}
+BUILDDIR=${BUILDDIR:-$TOPDIR}
+BINDIR=${BINDIR:-$BUILDDIR/src}
+BITCOIND=${BITCOIND:-$BINDIR/bitcoind}
+SHARE_EXAMPLES_DIR=${SHARE_EXAMPLES_DIR:-$TOPDIR/share/examples}
+EXAMPLE_CONF_FILE=${EXAMPLE_CONF_FILE:-$SHARE_EXAMPLES_DIR/bitcoin.conf}
+
+[ ! -x "$BITCOIND" ] && echo "$BITCOIND not found or not executable." && exit 1
+
+DIRTY=""
+VERSION_OUTPUT=$($BITCOIND --version)
+if [[ $VERSION_OUTPUT == *"dirty"* ]]; then
+  DIRTY="${DIRTY}${BITCOIND}\n"
+fi
+
+if [ -n "$DIRTY" ]
+then
+  echo -e "WARNING: $BITCOIND was built from a dirty tree.\n"
+  echo -e "To safely generate a bitcoin.conf file, please commit your changes to $BITCOIND, rebuild, then run this script again.\n"
+fi
+
+echo 'Generating example bitcoin.conf file in share/examples/'
+
+# create the directory, if it doesn't exist
+mkdir -p "${SHARE_EXAMPLES_DIR}"
+
+# create the header text
+cat > "${EXAMPLE_CONF_FILE}" << 'EOF'
+##
+## bitcoin.conf configuration file.
+## Generated by contrib/devtools/gen-bitcoin-conf.sh.
+##
+## Lines beginning with # are comments.
+## All possible configuration options are provided. To use, copy this file
+## to your data directory (default or specified by -datadir), uncomment
+## options you would like to change, and save the file.
+##
+
+
+### Options
+EOF
+
+# parse the output from bitcoind --help
+# adding newlines is a bit funky to ensure portability for BSD
+# see here for more details: https://stackoverflow.com/a/24575385
+${BITCOIND} --help \
+    | sed '1,/Print this help message and exit/d' \
+    | sed -E 's/^[[:space:]]{2}\-/#/' \
+    | sed -E 's/^[[:space:]]{7}/# /' \
+    | sed -E '/[=[:space:]]/!s/#.*$/&=1/' \
+    | awk '/^#[a-z]/{x=$0;next}{if (NF==0) print x"\n",x="";else print}' \
+    | sed 's,\(^[[:upper:]].*\)\:$,\
+### \1,' \
+    | sed 's/[[:space:]]*$//' >> "${EXAMPLE_CONF_FILE}"
+
+# create the footer text
+cat >> "${EXAMPLE_CONF_FILE}" << 'EOF'
+
+# [Sections]
+# Most options will apply to all networks. To confine an option to a specific
+# network, add it under the relevant section below.
+#
+# Note: If not specified under a network section, the options addnode, connect,
+# port, bind, rpcport, rpcbind, and wallet will only apply to mainnet.
+
+# Options for mainnet
+[main]
+
+# Options for testnet
+[test]
+
+# Options for signet
+[signet]
+
+# Options for regtest
+[regtest]
+EOF
diff --git a/contrib/guix/libexec/build.sh b/contrib/guix/libexec/build.sh
index 2757d10a7c..cdf0020d4d 100755
--- a/contrib/guix/libexec/build.sh
+++ b/contrib/guix/libexec/build.sh
@@ -368,6 +368,10 @@ mkdir -p "$DISTSRC"
                 ;;
         esac
 
+        # copy over the example bitcoin.conf file. if contrib/devtools/gen-bitcoin-conf.sh
+        # has not been run before buildling, this file will be a stub
+        cp "${DISTSRC}/share/examples/bitcoin.conf" "${DISTNAME}/"
+
         # Finally, deterministically produce {non-,}debug binary tarballs ready
         # for release
         case "$HOST" in
diff --git a/doc/bitcoin-conf.md b/doc/bitcoin-conf.md
index 8c9035c45b..acd767f234 100644
--- a/doc/bitcoin-conf.md
+++ b/doc/bitcoin-conf.md
@@ -61,4 +61,12 @@ Windows | `%APPDATA%\Bitcoin\` | `C:\Users\username\AppData\Roaming\Bitcoin\bitc
 Linux | `$HOME/.bitcoin/` | `/home/username/.bitcoin/bitcoin.conf`
 macOS | `$HOME/Library/Application Support/Bitcoin/` | `/Users/username/Library/Application Support/Bitcoin/bitcoin.conf`
 
-You can find an example bitcoin.conf file in [share/examples/bitcoin.conf](../share/examples/bitcoin.conf).
+An example configuration file can be generated by [contrib/devtools/gen-bitcoin-conf.sh](../contrib/devtools/gen-bitcoin-conf.sh).
+Run this script after compiling to generate an up-to-date configuration file.
+The output is placed under `share/examples/bitcoin.conf`.
+To use the generated configuration file, copy the example file into your data directory and edit it there, like so:
+
+```
+# example copy command for linux user
+cp share/examples/bitcoin.conf ~/.bitcoin
+```
diff --git a/doc/release-process.md b/doc/release-process.md
index 2f3a163a8e..03cea6f194 100644
--- a/doc/release-process.md
+++ b/doc/release-process.md
@@ -8,6 +8,7 @@ Release Process
 * Update translations see [translation_process.md](https://github.com/bitcoin/bitcoin/blob/master/doc/translation_process.md#synchronising-translations).
 * Update release candidate version in `configure.ac` (`CLIENT_VERSION_RC`).
 * Update manpages (after rebuilding the binaries), see [gen-manpages.py](https://github.com/bitcoin/bitcoin/blob/master/contrib/devtools/README.md#gen-manpagespy).
+* Update bitcoin.conf and commit, see [gen-bitcoin-conf.sh](https://github.com/bitcoin/bitcoin/blob/master/contrib/devtools/README.md#gen-bitcoin-confsh).
 
 ### Before every major and minor release
 
diff --git a/share/examples/bitcoin.conf b/share/examples/bitcoin.conf
index c5b79709c7..5bee4bf92e 100644
--- a/share/examples/bitcoin.conf
+++ b/share/examples/bitcoin.conf
@@ -1,191 +1 @@
-##
-## bitcoin.conf configuration file. Lines beginning with # are comments.
-##
-
-# Network-related settings:
-
-# Note that if you use testnet, signet or regtest, particularly with the options
-# addnode, connect, port, bind, rpcport, rpcbind or wallet, you will also
-# want to read "[Sections]" further down.
-
-# Run on the testnet network
-#testnet=0
-
-# Run on a signet network
-#signet=0
-
-# Run a regression test network
-#regtest=0
-
-# Connect via a SOCKS5 proxy
-#proxy=127.0.0.1:9050
-
-# Bind to given address and always listen on it. Use [host]:port notation for IPv6
-#bind=<addr>
-
-# Bind to given address and add permission flags to peers connecting to it. Use [host]:port notation for IPv6
-#whitebind=perm@<addr>
-
-##############################################################
-##            Quick Primer on addnode vs connect            ##
-##  Let's say for instance you use addnode=4.2.2.4          ##
-##  addnode will connect you to and tell you about the      ##
-##    nodes connected to 4.2.2.4.  In addition it will tell ##
-##    the other nodes connected to it that you exist so     ##
-##    they can connect to you.                              ##
-##  connect will not do the above when you 'connect' to it. ##
-##    It will *only* connect you to 4.2.2.4 and no one else.##
-##                                                          ##
-##  So if you're behind a firewall, or have other problems  ##
-##  finding nodes, add some using 'addnode'.                ##
-##                                                          ##
-##  If you want to stay private, use 'connect' to only      ##
-##  connect to "trusted" nodes.                             ##
-##                                                          ##
-##  If you run multiple nodes on a LAN, there's no need for ##
-##  all of them to open lots of connections.  Instead       ##
-##  'connect' them all to one node that is port forwarded   ##
-##  and has lots of connections.                            ##
-##       Thanks goes to [Noodle] on Freenode.               ##
-##############################################################
-
-# Use as many addnode= settings as you like to connect to specific peers
-#addnode=69.164.218.197
-#addnode=10.0.0.2:8333
-
-# Alternatively use as many connect= settings as you like to connect ONLY to specific peers
-#connect=69.164.218.197
-#connect=10.0.0.1:8333
-
-# Listening mode, enabled by default except when 'connect' is being used
-#listen=1
-
-# Port on which to listen for connections (default: 8333, testnet: 18333, signet: 38333, regtest: 18444)
-#port=
-
-# Maximum number of inbound + outbound connections (default: 125). This option
-# applies only if inbound connections are enabled; otherwise, the number of connections
-# will not be more than 11: 8 full-relay connections, 2 block-relay-only ones, and
-# occasionally 1 short-lived feeler or extra outbound block-relay-only connection.
-# These limits do not apply to connections added manually with the -addnode
-# configuration option or the addnode RPC, which have a separate limit of 8 connections.
-#maxconnections=
-
-# Maximum upload bandwidth target in MiB per day (e.g. 'maxuploadtarget=1024' is 1 GiB per day).
-# This limits the upload bandwidth for those with bandwidth limits. 0 = no limit (default: 0).
-# -maxuploadtarget does not apply to peers with 'download' permission.
-# For more information on reducing bandwidth utilization, see: doc/reduce-traffic.md.
-#maxuploadtarget=
-
-#
-# JSON-RPC options (for controlling a running Bitcoin/bitcoind process)
-#
-
-# server=1 tells Bitcoin-Qt and bitcoind to accept JSON-RPC commands
-#server=0
-
-# Bind to given address to listen for JSON-RPC connections.
-# Refer to the manpage or bitcoind -help for further details.
-#rpcbind=<addr>
-
-# If no rpcpassword is set, rpc cookie auth is sought. The default `-rpccookiefile` name
-# is .cookie and found in the `-datadir` being used for bitcoind. This option is typically used
-# when the server and client are run as the same user.
-#
-# If not, you must set rpcuser and rpcpassword to secure the JSON-RPC API.
-#
-# The config option `rpcauth` can be added to server startup argument. It is set at initialization time
-# using the output from the script in share/rpcauth/rpcauth.py after providing a username:
-#
-# ./share/rpcauth/rpcauth.py alice
-# String to be appended to bitcoin.conf:
-# rpcauth=alice:f7efda5c189b999524f151318c0c86$d5b51b3beffbc02b724e5d095828e0bc8b2456e9ac8757ae3211a5d9b16a22ae
-# Your password:
-# DONT_USE_THIS_YOU_WILL_GET_ROBBED_8ak1gI25KFTvjovL3gAM967mies3E=
-#
-# On client-side, you add the normal user/password pair to send commands:
-#rpcuser=alice
-#rpcpassword=DONT_USE_THIS_YOU_WILL_GET_ROBBED_8ak1gI25KFTvjovL3gAM967mies3E=
-#
-# You can even add multiple entries of these to the server conf file, and client can use any of them:
-# rpcauth=bob:b2dd077cb54591a2f3139e69a897ac$4e71f08d48b4347cf8eff3815c0e25ae2e9a4340474079f55705f40574f4ec99
-
-# How many seconds bitcoin will wait for a complete RPC HTTP request.
-# after the HTTP connection is established.
-#rpcclienttimeout=30
-
-# By default, only RPC connections from localhost are allowed.
-# Specify as many rpcallowip= settings as you like to allow connections from other hosts,
-# either as a single IPv4/IPv6 or with a subnet specification.
-
-# NOTE: opening up the RPC port to hosts outside your local trusted network is NOT RECOMMENDED,
-# because the rpcpassword is transmitted over the network unencrypted.
-
-# server=1 tells Bitcoin-Qt to accept JSON-RPC commands.
-# it is also read by bitcoind to determine if RPC should be enabled
-#rpcallowip=10.1.1.34/255.255.255.0
-#rpcallowip=1.2.3.4/24
-#rpcallowip=2001:db8:85a3:0:0:8a2e:370:7334/96
-
-# Listen for RPC connections on this TCP port:
-#rpcport=8332
-
-# You can use Bitcoin or bitcoind to send commands to Bitcoin/bitcoind
-# running on another host using this option:
-#rpcconnect=127.0.0.1
-
-# Wallet options
-
-# Specify where to find wallet, lockfile and logs. If not present, those files will be
-# created as new.
-#wallet=</path/to/dir>
-
-# Create transactions that have enough fees so they are likely to begin confirmation within n blocks (default: 6).
-# This setting is over-ridden by the -paytxfee option.
-#txconfirmtarget=n
-
-# Pay a transaction fee every time you send bitcoins.
-#paytxfee=0.000x
-
-# Miscellaneous options
-
-# Pre-generate this many public/private key pairs, so wallet backups will be valid for
-# both prior transactions and several dozen future transactions.
-#keypool=100
-
-# Maintain coinstats index used by the gettxoutsetinfo RPC (default: 0).
-#coinstatsindex=1
-
-# Enable pruning to reduce storage requirements by deleting old blocks.
-# This mode is incompatible with -txindex and -coinstatsindex.
-# 0 = default (no pruning).
-# 1 = allows manual pruning via RPC.
-# >=550 = target to stay under in MiB.
-#prune=550
-
-# User interface options
-
-# Start Bitcoin minimized
-#min=1
-
-# Minimize to the system tray
-#minimizetotray=1
-
-# [Sections]
-# Most options apply to mainnet, testnet, signet and regtest.
-# If you want to confine an option to just one network, you should add it in the
-# relevant section below.
-# EXCEPTIONS: The options addnode, connect, port, bind, rpcport, rpcbind and wallet
-# only apply to mainnet unless they appear in the appropriate section below.
-
-# Options only for mainnet
-[main]
-
-# Options only for testnet
-[test]
-
-# Options only for signet
-[signet]
-
-# Options only for regtest
-[regtest]
+# This is a placeholder file. Please follow the instructions in `contrib/devtools/README.md` to generate a bitcoin.conf file.
diff --git a/src/init.cpp b/src/init.cpp
index 8cc81f1500..13a4b3f433 100644
--- a/src/init.cpp
+++ b/src/init.cpp
@@ -409,7 +409,7 @@ void SetupServerArgs(ArgsManager& argsman)
     argsman.AddArg("-blockreconstructionextratxn=<n>", strprintf("Extra transactions to keep in memory for compact block reconstructions (default: %u)", DEFAULT_BLOCK_RECONSTRUCTION_EXTRA_TXN), ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS);
     argsman.AddArg("-blocksonly", strprintf("Whether to reject transactions from network peers. Automatic broadcast and rebroadcast of any transactions from inbound peers is disabled, unless the peer has the 'forcerelay' permission. RPC transactions are not affected. (default: %u)", DEFAULT_BLOCKSONLY), ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS);
     argsman.AddArg("-coinstatsindex", strprintf("Maintain coinstats index used by the gettxoutsetinfo RPC (default: %u)", DEFAULT_COINSTATSINDEX), ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS);
-    argsman.AddArg("-conf=<file>", strprintf("Specify path to read-only configuration file. Relative paths will be prefixed by datadir location. (default: %s)", BITCOIN_CONF_FILENAME), ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS);
+    argsman.AddArg("-conf=<file>", strprintf("Specify path to read-only configuration file. Relative paths will be prefixed by datadir location (only useable from command line, not configuration file) (default: %s)", BITCOIN_CONF_FILENAME), ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS);
     argsman.AddArg("-datadir=<dir>", "Specify data directory", ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS);
     argsman.AddArg("-dbbatchsize", strprintf("Maximum database write batch size in bytes (default: %u)", nDefaultDbBatchSize), ArgsManager::ALLOW_ANY | ArgsManager::DEBUG_ONLY, OptionsCategory::OPTIONS);
     argsman.AddArg("-dbcache=<n>", strprintf("Maximum database cache size <n> MiB (%d to %d, default: %d). In addition, unused mempool memory is shared for this cache (see -maxmempool).", nMinDbCache, nMaxDbCache, nDefaultDbCache), ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS);