diff options
author | Ava Chow <github@achow101.com> | 2024-09-25 16:39:07 -0400 |
---|---|---|
committer | Ava Chow <github@achow101.com> | 2024-09-25 16:39:07 -0400 |
commit | 65f6e7078b17e6e73d74dfeb00159878099fee1e (patch) | |
tree | c89ecf5075a72279f2416ebf466719103c3700ef /doc | |
parent | da612cea032ba241e2d82418baf171b06e5f0142 (diff) | |
parent | 1a332817665f77f55090fa166930fec0e5500727 (diff) |
Merge bitcoin/bitcoin#30510: multiprocess: Add IPC wrapper for Mining interface
1a332817665f77f55090fa166930fec0e5500727 doc: multiprocess documentation improvements (Ryan Ofsky)
d043950ba245cdcd09dc389a71d43b0a58e41a8b multiprocess: Add serialization code for BlockValidationState (Ryan Ofsky)
33c2eee285e35db50b33efb6e782ff002f9d18ec multiprocess: Add IPC wrapper for Mining interface (Ryan Ofsky)
06882f84017f6b569b46a644f39b6d3c120ec6cf multiprocess: Add serialization code for vector<char> (Russell Yanofsky)
095286f790acda4a32f04c77aa86106007e2a0d8 multiprocess: Add serialization code for CTransaction (Russell Yanofsky)
69dfeb18761cfd933b8a9a6e69ce9d3c516ba949 multiprocess: update common-types.h to use C++20 concepts (Ryan Ofsky)
206c6e78eec7c71d317666a1af07acf357ba001e build: Make bitcoin_ipc_test depend on bitcoin_ipc (Ryan Ofsky)
070e6a32d5ff4be2f4f1f6a8200fba2f61df16e3 depends: Update libmultiprocess library for cmake headers target (Ryan Ofsky)
Pull request description:
Add Cap'n Proto wrapper for the Mining interface introduced in #30200, and its associated types.
This PR combined with #30509 will allow a separate mining process, like the one being implemented in https://github.com/Sjors/bitcoin/pull/48, to connect to the node over IPC, and create, manage, and submit block templates. (#30437 shows another simpler demo of a process using the Mining interface.)
---
This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/issues/28722).
ACKs for top commit:
achow101:
ACK 1a332817665f77f55090fa166930fec0e5500727
TheCharlatan:
ACK 1a332817665f77f55090fa166930fec0e5500727
itornaza:
ACK 1a332817665f77f55090fa166930fec0e5500727
Tree-SHA512: 0791078dd6885dbd81e3d14c75fffff3da8d1277873af379ea6f9633e910c11485bb324e4cde3d936d50d343b16a10b0e8fc1e0fc6d7bdca7f522211da50c01e
Diffstat (limited to 'doc')
-rw-r--r-- | doc/design/multiprocess.md | 8 | ||||
-rw-r--r-- | doc/multiprocess.md | 4 |
2 files changed, 7 insertions, 5 deletions
diff --git a/doc/design/multiprocess.md b/doc/design/multiprocess.md index 49410a4213..a781da8d1b 100644 --- a/doc/design/multiprocess.md +++ b/doc/design/multiprocess.md @@ -81,7 +81,7 @@ This section describes the major components of the Inter-Process Communication ( - In the generated code, we have C++ client subclasses that inherit from the abstract classes in [`src/interfaces/`](../../src/interfaces/). These subclasses are the workhorses of the IPC mechanism. - They implement all the methods of the interface, marshalling arguments into a structured format, sending them as requests to the IPC server via a UNIX socket, and handling the responses. - These subclasses effectively mask the complexity of IPC, presenting a familiar C++ interface to developers. -- Internally, the client subclasses generated by the `mpgen` tool wrap [client classes generated by Cap'n Proto](https://capnproto.org/cxxrpc.html#clients), and use them to send IPC requests. +- Internally, the client subclasses generated by the `mpgen` tool wrap [client classes generated by Cap'n Proto](https://capnproto.org/cxxrpc.html#clients), and use them to send IPC requests. The Cap'n Proto client classes are low-level, with non-blocking methods that use asynchronous I/O and pass request and response objects, while mpgen client subclasses provide normal C++ methods that block while executing and convert between request/response objects and arguments/return values. ### C++ Server Classes in Generated Code - On the server side, corresponding generated C++ classes receive IPC requests. These server classes are responsible for unmarshalling method arguments, invoking the corresponding methods in the local [`src/interfaces/`](../../src/interfaces/) objects, and creating the IPC response. @@ -94,7 +94,7 @@ This section describes the major components of the Inter-Process Communication ( - **Asynchronous I/O and Thread Management**: The library is also responsible for managing I/O and threading. Particularly, it ensures that IPC requests never block each other and that new threads on either side of a connection can always make client calls. It also manages worker threads on the server side of calls, ensuring that calls from the same client thread always execute on the same server thread (to avoid locking issues and support nested callbacks). ### Type Hooks in [`src/ipc/capnp/*-types.h`](../../src/ipc/capnp/) -- **Custom Type Conversions**: In [`src/ipc/capnp/*-types.h`](../../src/ipc/capnp/), function overloads of two `libmultiprocess` C++ functions, `mp::CustomReadField` and `mp::CustomBuildFields`, are defined. These overloads are used for customizing the conversion of specific C++ types to and from Cap’n Proto types. +- **Custom Type Conversions**: In [`src/ipc/capnp/*-types.h`](../../src/ipc/capnp/), function overloads of `libmultiprocess` C++ functions, `mp::CustomReadField`, `mp::CustomBuildField`, `mp::CustomReadMessage` and `mp::CustomBuildMessage`, are defined. These overloads are used for customizing the conversion of specific C++ types to and from Cap’n Proto types. - **Handling Special Cases**: The `mpgen` tool and `libmultiprocess` library can convert most C++ types to and from Cap’n Proto types automatically, including interface types, primitive C++ types, standard C++ types like `std::vector`, `std::set`, `std::map`, `std::tuple`, and `std::function`, as well as simple C++ structs that consist of aforementioned types and whose fields correspond 1:1 with Cap’n Proto struct fields. For other types, `*-types.h` files provide custom code to convert between C++ and Cap’n Proto data representations. ### Protocol-Agnostic IPC Code in [`src/ipc/`](../../src/ipc/) @@ -197,7 +197,7 @@ sequenceDiagram - Upon receiving the request, the Cap'n Proto dispatching code in the `bitcoin-node` process calls the `getBlockHash` method of the `Chain` [server class](#c-server-classes-in-generated-code). - The server class is automatically generated by the `mpgen` tool from the [`chain.capnp`](https://github.com/ryanofsky/bitcoin/blob/pr/ipc/src/ipc/capnp/chain.capnp) file in [`src/ipc/capnp/`](../../src/ipc/capnp/). - The `getBlockHash` method of the generated `Chain` server subclass in `bitcoin-wallet` receives a Cap’n Proto request object with the `height` parameter, and calls the `getBlockHash` method on its local `Chain` object with the provided `height`. - - When the call returns, it encapsulates the return value in a Cap’n Proto response, which it sends back to the `bitcoin-wallet` process, + - When the call returns, it encapsulates the return value in a Cap’n Proto response, which it sends back to the `bitcoin-wallet` process. 5. **Response and Return** - The `getBlockHash` method of the generated `Chain` client subclass in `bitcoin-wallet` which sent the request now receives the response. @@ -232,7 +232,7 @@ This modularization represents an advancement in Bitcoin Core's architecture, of - **Cap’n Proto struct**: A structured data format used in Cap’n Proto, similar to structs in C++, for organizing and transporting data across different processes. -- **client class (in generated code)**: A C++ class generated from a Cap’n Proto interface which inherits from a Bitcoin core abstract class, and implements each virtual method to send IPC requests to another process. (see also [components section](#c-client-subclasses-in-generated-code)) +- **client class (in generated code)**: A C++ class generated from a Cap’n Proto interface which inherits from a Bitcoin Core abstract class, and implements each virtual method to send IPC requests to another process. (see also [components section](#c-client-subclasses-in-generated-code)) - **IPC (inter-process communication)**: Mechanisms that enable processes to exchange requests and data. diff --git a/doc/multiprocess.md b/doc/multiprocess.md index fee50142d7..1757296eed 100644 --- a/doc/multiprocess.md +++ b/doc/multiprocess.md @@ -17,7 +17,9 @@ The multiprocess feature requires [Cap'n Proto](https://capnproto.org/) and [lib ``` cd <BITCOIN_SOURCE_DIRECTORY> make -C depends NO_QT=1 MULTIPROCESS=1 -cmake -B build --toolchain=depends/x86_64-pc-linux-gnu/toolchain.cmake +# Set host platform to output of gcc -dumpmachine or clang -dumpmachine or check the depends/ directory for the generated subdirectory name +HOST_PLATFORM="x86_64-pc-linux-gnu" +cmake -B build --toolchain=depends/$HOST_PLATFORM/toolchain.cmake cmake --build build build/src/bitcoin-node -regtest -printtoconsole -debug=ipc BITCOIND=$(pwd)/build/src/bitcoin-node build/test/functional/test_runner.py |