Merge bitcoin/bitcoin#21711: guix: Add full installation and usage documentation

fac4814106c796b8786dd90053513cc35142dfe5 doc/release-process: Add torrent creation details (Carl Dong)
5d24cc3d82dad6812f8370c3ccc7c2b5a6c12c11 guix/INSTALL: Guix installs init scripts in libdir (Carl Dong)
5da2ee49d5b44de803b671aedbdd14e5c1d71ea9 guix/INSTALL: Add coreutils/inotify-dir-recreate troubleshooting (Carl Dong)
318c60700b7bbb7ec09a29bf037e7c2787646be6 guix: Adapt release-process.md to new Guix process (Carl Dong)
fcab35b2292f9221eaba521740e8b3b2511a8b78 guix-attest: Produce and sign normalized documents (Carl Dong)
c2541fd0ca99481a5a792a8f2772925d64fb0491 guix: Overhaul README (Carl Dong)
46ce6ce3782dfbd8f9d26dc2ba0f284755e75f2d tree-wide: Rename gitian-keys to builder-keys (Carl Dong)
fc4f8449f34e32b0b9ac9d218d6c3264b02467ba guix: Update various check_tools lists (Carl Dong)
263220a85c1df218431fafbda07c8b23ccc4ce4d guix: Check for a sane services database (Carl Dong)

Pull request description:

  Based on: #21462

  Keeping the README in one file so that it's easy to search through. Will add more jumping links later so navigation is easier.

  Current TODOs:
  - [x] Shell installer option: prompt user to re-login for `/etc/profile.d` entry to be picked up
  - [x] Binary tarball option: prompt user to create `/etc/profile.d` entry and re-login
  - [x] Fanquake docker option: complete section
  - [x] Arch Linux AUR option: prompt to start `guix-daemon-latest` unit after finishing "optional setup" section
  - [x] Building from source option: Insert dependency tree diagram that I made
  - [x] Building from source option: redo sectioning, kind of a mess right now
  - [x] Optional setup: make clear which parts are only needed if building from source
  - [x] Workaround 1 for GnuTLS: perhaps mention how to remove Guix build farm's key
  - [x] Overall (after everything): Make the links work.

  Note to self: wherever possible, tell user how to check that something is true rather than branching by installation option.

ACKs for top commit:
  fanquake:
    ACK fac4814106c796b8786dd90053513cc35142dfe5 - going to go ahead and merge this now. It's a lot of documentation, and could probably be nit-picked / improved further, however, that can continue over the next few weeks. I'm sure more (backportable) improvements / clarifications will be made while we progress through RCs towards a new release.

Tree-SHA512: dc46c0ecdfc67c7c7743ca26e4a603eb3f54adbf81be2f4c1f4c20577ebb84b5250b9c9ec89c0e9860337ab1c7cff94d7963c603287267deecfe1cd987fa070a

1 files changed, 103 insertions, 166 deletions

diff --git a/doc/release-process.md b/doc/release-process.md
index 75a574ee31..0ac67b9146 100644
--- a/doc/release-process.md
+++ b/doc/release-process.md
@@ -44,10 +44,6 @@ Release Process
     - Make an announcement that translators can start translating for the new version. You can use one of the [previous announcements](https://www.transifex.com/bitcoin/bitcoin/announcements/) as a template.
     - Change the auto-update URL for the resource to `master`, e.g. `https://raw.githubusercontent.com/bitcoin/bitcoin/master/src/qt/locale/bitcoin_en.xlf`. (Do this only after the previous steps, to prevent an auto-update from interfering.)
 
-#### After branch-off (on master)
-
-- Update the version of `contrib/gitian-descriptors/*.yml`.
-
 #### After branch-off (on the major release branch)
 
 - Update the versions.
@@ -72,14 +68,14 @@ This will perform a few last-minute consistency checks in the build system files
 
 ### First time / New builders
 
-If you're using the automated script (found in [contrib/gitian-build.py](/contrib/gitian-build.py)), then at this point you should run it with the "--setup" command. Otherwise ignore this.
+Install Guix using one of the installation methods detailed in
+[contrib/guix/INSTALL.md](/contrib/guix/INSTALL.md).
 
 Check out the source code in the following directory hierarchy.
 
     cd /path/to/your/toplevel/build
-    git clone https://github.com/bitcoin-core/gitian.sigs.git
+    git clone https://github.com/bitcoin-core/guix.sigs.git
     git clone https://github.com/bitcoin-core/bitcoin-detached-sigs.git
-    git clone https://github.com/devrandom/gitian-builder.git
     git clone https://github.com/bitcoin/bitcoin.git
 
 ### Write the release notes
@@ -94,110 +90,56 @@ Generate list of authors:
 
     git log --format='- %aN' v(current version, e.g. 0.20.0)..v(new version, e.g. 0.20.1) | sort -fiu
 
-### Setup and perform Gitian builds
-
-If you're using the automated script (found in [contrib/gitian-build.py](/contrib/gitian-build.py)), then at this point you should run it with the "--build" command. Otherwise ignore this.
-
-Setup Gitian descriptors:
-
-    pushd ./bitcoin
-    export SIGNER="(your Gitian key, ie bluematt, sipa, etc)"
-    export VERSION=(new version, e.g. 0.20.0)
-    git fetch
-    git checkout v${VERSION}
-    popd
-
-Ensure your gitian.sigs are up-to-date if you wish to gverify your builds against other Gitian signatures.
-
-    pushd ./gitian.sigs
-    git pull
-    popd
-
-Ensure gitian-builder is up-to-date:
-
-    pushd ./gitian-builder
-    git pull
-    popd
-
-### Fetch and create inputs: (first time, or when dependency versions change)
-
-    pushd ./gitian-builder
-    mkdir -p inputs
-    wget -O inputs/osslsigncode-2.0.tar.gz https://github.com/mtrojnar/osslsigncode/archive/2.0.tar.gz
-    echo '5a60e0a4b3e0b4d655317b2f12a810211c50242138322b16e7e01c6fbb89d92f inputs/osslsigncode-2.0.tar.gz' | sha256sum -c
-    popd
-
-Create the macOS SDK tarball, see the [macdeploy instructions](/contrib/macdeploy/README.md#deterministic-macos-dmg-notes) for details, and copy it into the inputs directory.
-
-### Optional: Seed the Gitian sources cache and offline git repositories
-
-NOTE: Gitian is sometimes unable to download files. If you have errors, try the step below.
-
-By default, Gitian will fetch source files as needed. To cache them ahead of time, make sure you have checked out the tag you want to build in bitcoin, then:
-
-    pushd ./gitian-builder
-    make -C ../bitcoin/depends download SOURCES_PATH=`pwd`/cache/common
-    popd
-
-Only missing files will be fetched, so this is safe to re-run for each build.
-
-NOTE: Offline builds must use the --url flag to ensure Gitian fetches only from local URLs. For example:
-
-    pushd ./gitian-builder
-    ./bin/gbuild --url bitcoin=/path/to/bitcoin,signature=/path/to/sigs {rest of arguments}
-    popd
+### Setup and perform Guix builds
 
-The gbuild invocations below <b>DO NOT DO THIS</b> by default.
+Checkout the Bitcoin Core version you'd like to build:
 
-### Build and sign Bitcoin Core for Linux, Windows, and macOS:
+```sh
+pushd ./bitcoin
+SIGNER='(your builder key, ie bluematt, sipa, etc)'
+VERSION='(new version without v-prefix, e.g. 0.20.0)'
+git fetch "v${VERSION}"
+git checkout "v${VERSION}"
+popd
+```
 
-    pushd ./gitian-builder
-    ./bin/gbuild --num-make 2 --memory 3000 --commit bitcoin=v${VERSION} ../bitcoin/contrib/gitian-descriptors/gitian-linux.yml
-    ./bin/gsign --signer "$SIGNER" --release ${VERSION}-linux --destination ../gitian.sigs/ ../bitcoin/contrib/gitian-descriptors/gitian-linux.yml
-    mv build/out/bitcoin-*.tar.gz build/out/src/bitcoin-*.tar.gz ../
+Ensure your guix.sigs are up-to-date if you wish to `guix-verify` your builds
+against other `guix-attest` signatures.
 
-    ./bin/gbuild --num-make 2 --memory 3000 --commit bitcoin=v${VERSION} ../bitcoin/contrib/gitian-descriptors/gitian-win.yml
-    ./bin/gsign --signer "$SIGNER" --release ${VERSION}-win-unsigned --destination ../gitian.sigs/ ../bitcoin/contrib/gitian-descriptors/gitian-win.yml
-    mv build/out/bitcoin-*-win-unsigned.tar.gz inputs/bitcoin-win-unsigned.tar.gz
-    mv build/out/bitcoin-*.zip build/out/bitcoin-*.exe ../
+```sh
+git -C ./guix.sigs pull
+```
 
-    ./bin/gbuild --num-make 2 --memory 3000 --commit bitcoin=v${VERSION} ../bitcoin/contrib/gitian-descriptors/gitian-osx.yml
-    ./bin/gsign --signer "$SIGNER" --release ${VERSION}-osx-unsigned --destination ../gitian.sigs/ ../bitcoin/contrib/gitian-descriptors/gitian-osx.yml
-    mv build/out/bitcoin-*-osx-unsigned.tar.gz inputs/bitcoin-osx-unsigned.tar.gz
-    mv build/out/bitcoin-*.tar.gz build/out/bitcoin-*.dmg ../
-    popd
+### Create the macOS SDK tarball: (first time, or when SDK version changes)
 
-Build output expected:
+Create the macOS SDK tarball, see the [macdeploy
+instructions](/contrib/macdeploy/README.md#deterministic-macos-dmg-notes) for
+details.
 
-  1. source tarball (`bitcoin-${VERSION}.tar.gz`)
-  2. linux 32-bit and 64-bit dist tarballs (`bitcoin-${VERSION}-linux[32|64].tar.gz`)
-  3. windows 32-bit and 64-bit unsigned installers and dist zips (`bitcoin-${VERSION}-win[32|64]-setup-unsigned.exe`, `bitcoin-${VERSION}-win[32|64].zip`)
-  4. macOS unsigned installer and dist tarball (`bitcoin-${VERSION}-osx-unsigned.dmg`, `bitcoin-${VERSION}-osx64.tar.gz`)
-  5. Gitian signatures (in `gitian.sigs/${VERSION}-<linux|{win,osx}-unsigned>/(your Gitian key)/`)
+### Build and attest to build outputs:
 
-### Verify other gitian builders signatures to your own. (Optional)
+Follow the relevant Guix README.md sections:
+- [Performing a build](/contrib/guix/README.md#performing-a-build)
+- [Attesting to build outputs](/contrib/guix/README.md#attesting-to-build-outputs)
 
-Add other gitian builders keys to your gpg keyring, and/or refresh keys: See `../bitcoin/contrib/gitian-keys/README.md`.
+### Verify other builders' signatures to your own. (Optional)
 
-Verify the signatures
+Add other builders keys to your gpg keyring, and/or refresh keys: See `../bitcoin/contrib/builder-keys/README.md`.
 
-    pushd ./gitian-builder
-    ./bin/gverify -v -d ../gitian.sigs/ -r ${VERSION}-linux ../bitcoin/contrib/gitian-descriptors/gitian-linux.yml
-    ./bin/gverify -v -d ../gitian.sigs/ -r ${VERSION}-win-unsigned ../bitcoin/contrib/gitian-descriptors/gitian-win.yml
-    ./bin/gverify -v -d ../gitian.sigs/ -r ${VERSION}-osx-unsigned ../bitcoin/contrib/gitian-descriptors/gitian-osx.yml
-    popd
+Follow the relevant Guix README.md sections:
+- [Verifying build output attestations](/contrib/guix/README.md#verifying-build-output-attestations)
 
 ### Next steps:
 
-Commit your signature to gitian.sigs:
+Commit your signature to guix.sigs:
 
-    pushd gitian.sigs
-    git add ${VERSION}-linux/"${SIGNER}"
-    git add ${VERSION}-win-unsigned/"${SIGNER}"
-    git add ${VERSION}-osx-unsigned/"${SIGNER}"
-    git commit -m "Add ${VERSION} unsigned sigs for ${SIGNER}"
-    git push  # Assuming you can push to the gitian.sigs tree
-    popd
+```sh
+pushd ./guix.sigs
+git add "${VERSION}/${SIGNER}"/noncodesigned.SHA256SUMS{,.asc}
+git commit -m "Add ${VERSION} unsigned sigs for ${SIGNER}"
+git push  # Assuming you can push to the guix.sigs tree
+popd
+```
 
 Codesigner only: Create Windows/macOS detached signatures:
 - Only one person handles codesigning. Everyone else should skip to the next step.
@@ -209,7 +151,7 @@ Codesigner only: Sign the macOS binary:
     tar xf bitcoin-osx-unsigned.tar.gz
     ./detached-sig-create.sh -s "Key ID"
     Enter the keychain password and authorize the signature
-    Move signature-osx.tar.gz back to the gitian host
+    Move signature-osx.tar.gz back to the guix-build host
 
 Codesigner only: Sign the windows binaries:
 
@@ -220,93 +162,88 @@ Codesigner only: Sign the windows binaries:
 
 Codesigner only: Commit the detached codesign payloads:
 
-    cd ~/bitcoin-detached-sigs
-    checkout the appropriate branch for this release series
-    rm -rf *
-    tar xf signature-osx.tar.gz
-    tar xf signature-win.tar.gz
-    git add -A
-    git commit -m "point to ${VERSION}"
-    git tag -s v${VERSION} HEAD
-    git push the current branch and new tag
+```sh
+pushd ./bitcoin-detached-sigs
+# checkout the appropriate branch for this release series
+rm -rf ./*
+tar xf signature-osx.tar.gz
+tar xf signature-win.tar.gz
+git add -A
+git commit -m "point to ${VERSION}"
+git tag -s "v${VERSION}" HEAD
+git push the current branch and new tag
+popd
+```
 
 Non-codesigners: wait for Windows/macOS detached signatures:
 
 - Once the Windows/macOS builds each have 3 matching signatures, they will be signed with their respective release keys.
 - Detached signatures will then be committed to the [bitcoin-detached-sigs](https://github.com/bitcoin-core/bitcoin-detached-sigs) repository, which can be combined with the unsigned apps to create signed binaries.
 
-Create (and optionally verify) the signed macOS binary:
+Create (and optionally verify) the codesigned outputs:
 
-    pushd ./gitian-builder
-    ./bin/gbuild -i --commit signature=v${VERSION} ../bitcoin/contrib/gitian-descriptors/gitian-osx-signer.yml
-    ./bin/gsign --signer "$SIGNER" --release ${VERSION}-osx-signed --destination ../gitian.sigs/ ../bitcoin/contrib/gitian-descriptors/gitian-osx-signer.yml
-    ./bin/gverify -v -d ../gitian.sigs/ -r ${VERSION}-osx-signed ../bitcoin/contrib/gitian-descriptors/gitian-osx-signer.yml
-    mv build/out/bitcoin-osx-signed.dmg ../bitcoin-${VERSION}-osx.dmg
-    popd
+- [Codesigning](/contrib/guix/README.md#codesigning)
 
-Create (and optionally verify) the signed Windows binaries:
+Commit your signature for the signed macOS/Windows binaries:
 
-    pushd ./gitian-builder
-    ./bin/gbuild -i --commit signature=v${VERSION} ../bitcoin/contrib/gitian-descriptors/gitian-win-signer.yml
-    ./bin/gsign --signer "$SIGNER" --release ${VERSION}-win-signed --destination ../gitian.sigs/ ../bitcoin/contrib/gitian-descriptors/gitian-win-signer.yml
-    ./bin/gverify -v -d ../gitian.sigs/ -r ${VERSION}-win-signed ../bitcoin/contrib/gitian-descriptors/gitian-win-signer.yml
-    mv build/out/bitcoin-*win64-setup.exe ../bitcoin-${VERSION}-win64-setup.exe
-    popd
+```sh
+pushd ./guix.sigs
+git add "${VERSION}/${SIGNER}"/all.SHA256SUMS{,.asc}
+git commit -m "Add ${SIGNER} ${VERSION} signed binaries signatures"
+git push  # Assuming you can push to the guix.sigs tree
+popd
+```
 
-Commit your signature for the signed macOS/Windows binaries:
+### After 3 or more people have guix-built and their results match:
 
-    pushd gitian.sigs
-    git add ${VERSION}-osx-signed/"${SIGNER}"
-    git add ${VERSION}-win-signed/"${SIGNER}"
-    git commit -m "Add ${SIGNER} ${VERSION} signed binaries signatures"
-    git push  # Assuming you can push to the gitian.sigs tree
-    popd
+Combine `all.SHA256SUMS` and `all.SHA256SUMS.asc` into a clear-signed
+`SHA256SUMS.asc` message:
 
-### After 3 or more people have gitian-built and their results match:
+```sh
+echo -e "-----BEGIN PGP SIGNED MESSAGE-----\nHash: SHA256\n\n$(cat all.SHA256SUMS)\n$(cat filename.txt.asc)" > SHA256SUMS.asc
+```
 
-- Create `SHA256SUMS.asc` for the builds, and GPG-sign it:
+Here's an equivalent, more readable command if you're confident that you won't
+mess up whitespaces when copy-pasting:
 
 ```bash
-sha256sum * > SHA256SUMS
-```
+cat << EOF > SHA256SUMS.asc
+-----BEGIN PGP SIGNED MESSAGE-----
+Hash: SHA256
 
-The list of files should be:
+$(cat all.SHA256SUMS)
+$(cat all.SHA256SUMS.asc)
+EOF
 ```
-bitcoin-${VERSION}-aarch64-linux-gnu.tar.gz
-bitcoin-${VERSION}-arm-linux-gnueabihf.tar.gz
-bitcoin-${VERSION}-riscv64-linux-gnu.tar.gz
-bitcoin-${VERSION}-x86_64-linux-gnu.tar.gz
-bitcoin-${VERSION}-osx64.tar.gz
-bitcoin-${VERSION}-osx.dmg
-bitcoin-${VERSION}.tar.gz
-bitcoin-${VERSION}-win64-setup.exe
-bitcoin-${VERSION}-win64.zip
-```
-The `*-debug*` files generated by the gitian build contain debug symbols
-for troubleshooting by developers. It is assumed that anyone that is interested
-in debugging can run gitian to generate the files for themselves. To avoid
-end-user confusion about which file to pick, as well as save storage
-space *do not upload these to the bitcoincore.org server, nor put them in the torrent*.
 
-- GPG-sign it, delete the unsigned file:
-```
-gpg --digest-algo sha256 --clearsign SHA256SUMS # outputs SHA256SUMS.asc
-rm SHA256SUMS
-```
-(the digest algorithm is forced to sha256 to avoid confusion of the `Hash:` header that GPG adds with the SHA256 used for the files)
-Note: check that SHA256SUMS itself doesn't end up in SHA256SUMS, which is a spurious/nonsensical entry.
+- Upload to the bitcoincore.org server (`/var/www/bin/bitcoin-core-${VERSION}`):
+    1. The contents of `./bitcoin/guix-build-${VERSION}/output`, except for
+       `*-debug*` files.
 
-- Upload zips and installers, as well as `SHA256SUMS.asc` from last step, to the bitcoincore.org server
-  into `/var/www/bin/bitcoin-core-${VERSION}`
+       The `*-debug*` files generated by the guix build contain debug symbols
+       for troubleshooting by developers. It is assumed that anyone that is
+       interested in debugging can run guix to generate the files for
+       themselves. To avoid end-user confusion about which file to pick, as well
+       as save storage space *do not upload these to the bitcoincore.org server,
+       nor put them in the torrent*.
 
-- A `.torrent` will appear in the directory after a few minutes. Optionally help seed this torrent. To get the `magnet:` URI use:
-```bash
-transmission-show -m <torrent file>
-```
-Insert the magnet URI into the announcement sent to mailing lists. This permits
-people without access to `bitcoincore.org` to download the binary distribution.
-Also put it into the `optional_magnetlink:` slot in the YAML file for
-bitcoincore.org.
+    2. The combined clear-signed message you just created `SHA256SUMS.asc`
+
+- Create a torrent of the `/var/www/bin/bitcoin-core-${VERSION}` directory such
+  that at the top level there is only one file: the `bitcoin-core-${VERSION}`
+  directory containing everything else. Name the torrent
+  `bitcoin-${VERSION}.torrent` (note that there is no `-core-` in this name).
+
+  Optionally help seed this torrent. To get the `magnet:` URI use:
+
+  ```sh
+  transmission-show -m <torrent file>
+  ```
+
+  Insert the magnet URI into the announcement sent to mailing lists. This permits
+  people without access to `bitcoincore.org` to download the binary distribution.
+  Also put it into the `optional_magnetlink:` slot in the YAML file for
+  bitcoincore.org.
 
 - Update other repositories and websites for new version
 
@@ -344,14 +281,14 @@ bitcoincore.org.
         - https://code.launchpad.net/~bitcoin-core/bitcoin-core-snap/+git/packaging/+ref/0.xx (Click "Create snap package")
         - Name it "bitcoin-core-snap-0.xx"
         - Leave owner and series as-is
-        - Select architectures that are compiled via gitian
+        - Select architectures that are compiled via guix
         - Leave "automatically build when branch changes" unticked
         - Tick "automatically upload to store"
         - Put "bitcoin-core" in the registered store package name field
         - Tick the "edge" box
         - Put "0.xx" in the track field
         - Click "create snap package"
-        - Click "Request builds" for every new release on this branch (after updating the snapcraft.yml in the branch to reflect the latest gitian results)
+        - Click "Request builds" for every new release on this branch (after updating the snapcraft.yml in the branch to reflect the latest guix results)
         - Promote release on https://snapcraft.io/bitcoin-core/releases if it passes sanity checks
 
   - This repo