Merge bitcoin/bitcoin#19160: multiprocess: Add basic spawn and IPC support

84934bf70e11fe4cda1cfda60113a54895d4fdd5 multiprocess: Add echoipc RPC method and test (Russell Yanofsky)
7d76cf667eff512043a28d4407cc89f58796c42b multiprocess: Add comments and documentation (Russell Yanofsky)
ddf7ecc8dfc64cf121099fb047e1ac871de94f4c multiprocess: Add bitcoin-node process spawning support (Russell Yanofsky)
10afdf0280fa93bfffb0a7665c60dc155cd84514 multiprocess: Add Ipc interface implementation (Russell Yanofsky)
745c9cebd50fea1664efef571dc1ee1bddc96102 multiprocess: Add Ipc and Init interface definitions (Russell Yanofsky)
5d62d7f6cd48bbc4e9f37ecc369f38d5e1e0036c Update libmultiprocess library (Russell Yanofsky)

Pull request description:

  This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10).

  ---

  This PR adds basic process spawning and IPC method call support to `bitcoin-node` executables built with `--enable-multiprocess`[*].

  These changes are used in https://github.com/bitcoin/bitcoin/pull/10102 to let node, gui, and wallet functionality run in different processes, and extended in https://github.com/bitcoin/bitcoin/pull/19460 and https://github.com/bitcoin/bitcoin/pull/19461 after that to allow gui and wallet processes to be started and stopped independently and connect to the node over a socket.

  These changes can also be used to implement new functionality outside the `bitcoin-node` process like external indexes or pluggable transports (https://github.com/bitcoin/bitcoin/pull/18988). The `Ipc::spawnProcess` and `Ipc::serveProcess` methods added here are entry points for spawning a child process and serving a parent process, and being able to make bidirectional, multithreaded method calls between the processes. A simple example of this is implemented in commit "Add echoipc RPC method and test."

  Changes in this PR aside from the echo test were originally part of #10102, but have been split and moved here for easier review, and so they can be used for other applications like external plugins.

  Additional notes about this PR can be found at https://bitcoincore.reviews/19160

  [*] Note: the `--enable-multiprocess` feature is still experimental, and not enabled by default, and not yet supported on windows. More information can be found in [doc/multiprocess.md](https://github.com/bitcoin/bitcoin/blob/master/doc/multiprocess.md)

ACKs for top commit:
  fjahr:
    re-ACK 84934bf70e11fe4cda1cfda60113a54895d4fdd5
  ariard:
    ACK 84934bf. Changes since last ACK fixes the silent merge conflict about `EnsureAnyNodeContext()`. Rebuilt and checked again debug command `echoipc`.

Tree-SHA512: 52a948b5e18a26d7d7a09b83003eaae9b1ed2981978c36c959fe9a55abf70ae6a627c4ff913a3428be17400a3dace30c58b5057fa75c319662c3be98f19810c6

1 files changed, 71 insertions, 0 deletions

diff --git a/src/interfaces/ipc.h b/src/interfaces/ipc.h
new file mode 100644
index 0000000000..e9e6c78053
--- /dev/null
+++ b/src/interfaces/ipc.h
@@ -0,0 +1,71 @@
+// Copyright (c) 2021 The Bitcoin Core developers
+// Distributed under the MIT software license, see the accompanying
+// file COPYING or http://www.opensource.org/licenses/mit-license.php.
+
+#ifndef BITCOIN_INTERFACES_IPC_H
+#define BITCOIN_INTERFACES_IPC_H
+
+#include <functional>
+#include <memory>
+#include <typeindex>
+
+namespace interfaces {
+class Init;
+
+//! Interface providing access to interprocess-communication (IPC)
+//! functionality. The IPC implementation is responsible for establishing
+//! connections between a controlling process and a process being controlled.
+//! When a connection is established, the process being controlled returns an
+//! interfaces::Init pointer to the controlling process, which the controlling
+//! process can use to get access to other interfaces and functionality.
+//!
+//! When spawning a new process, the steps are:
+//!
+//! 1. The controlling process calls interfaces::Ipc::spawnProcess(), which
+//!    calls ipc::Process::spawn(), which spawns a new process and returns a
+//!    socketpair file descriptor for communicating with it.
+//!    interfaces::Ipc::spawnProcess() then calls ipc::Protocol::connect()
+//!    passing the socketpair descriptor, which returns a local proxy
+//!    interfaces::Init implementation calling remote interfaces::Init methods.
+//! 2. The spawned process calls interfaces::Ipc::startSpawnProcess(), which
+//!    calls ipc::Process::checkSpawned() to read command line arguments and
+//!    determine whether it is a spawned process and what socketpair file
+//!    descriptor it should use. It then calls ipc::Protocol::serve() to handle
+//!    incoming requests from the socketpair and invoke interfaces::Init
+//!    interface methods, and exit when the socket is closed.
+//! 3. The controlling process calls local proxy interfaces::Init object methods
+//!    to make other proxy objects calling other remote interfaces. It can also
+//!    destroy the initial interfaces::Init object to close the connection and
+//!    shut down the spawned process.
+class Ipc
+{
+public:
+    virtual ~Ipc() = default;
+
+    //! Spawn a child process returning pointer to its Init interface.
+    virtual std::unique_ptr<Init> spawnProcess(const char* exe_name) = 0;
+
+    //! If this is a spawned process, block and handle requests from the parent
+    //! process by forwarding them to this process's Init interface, then return
+    //! true. If this is not a spawned child process, return false.
+    virtual bool startSpawnedProcess(int argc, char* argv[], int& exit_status) = 0;
+
+    //! Add cleanup callback to remote interface that will run when the
+    //! interface is deleted.
+    template<typename Interface>
+    void addCleanup(Interface& iface, std::function<void()> cleanup)
+    {
+        addCleanup(typeid(Interface), &iface, std::move(cleanup));
+    }
+
+protected:
+    //! Internal implementation of public addCleanup method (above) as a
+    //! type-erased virtual function, since template functions can't be virtual.
+    virtual void addCleanup(std::type_index type, void* iface, std::function<void()> cleanup) = 0;
+};
+
+//! Return implementation of Ipc interface.
+std::unique_ptr<Ipc> MakeIpc(const char* exe_name, const char* process_argv0, Init& init);
+} // namespace interfaces
+
+#endif // BITCOIN_INTERFACES_IPC_H