Merge bitcoin/bitcoin#26844: Net: Pass `MSG_MORE` flag when sending non-final network messages (round 2)

691eaf8873fe2f189153ca637506a0291504c97a Pass MSG_MORE flag when sending non-final network messages (Matt Whitlock)

Pull request description:

  **N.B.:** This is my second attempt at introducing this optimization. #12519 (2018) was closed in deference to switching to doing gathering socket writes using `sendmsg(2)`, which I agree would have superior performance due to fewer syscalls, but that work was apparently abandoned in late 2018. Ever since, Bitcoin Core has continued writing tons of runt packets to the wire. Can we proceed with my halfway solution for now?

  ----

  Since Nagle's algorithm is disabled, each and every call to `send(2)` can potentially generate a separate TCP segment on the wire. This is especially inefficient when sending the tiny header preceding each message payload.

  Linux implements a `MSG_MORE` flag that tells the kernel not to push the passed data immediately to the connected peer but rather to collect it in the socket's internal transmit buffer where it can be combined with data from successive calls to `send(2)`. Where available, specify this flag when calling `send(2)` in `CConnman::SocketSendData(CNode &)` if the data buffer being sent is not the last one in `node.vSendMsg`.

ACKs for top commit:
  sipa:
    ACK 691eaf8873fe2f189153ca637506a0291504c97a
  vasild:
    ACK 691eaf8873fe2f189153ca637506a0291504c97a

Tree-SHA512: 9a7f46bc12edbf78d488f05d1c46760110a24c95af74b627d2604fcd198fa3f511c5956bac36d0034e88c632d432f7d394147e667a11b027af0a30f70a546d70

1 files changed, 7 insertions, 1 deletions

diff --git a/src/net.cpp b/src/net.cpp
index 98500bd30e..4f4e443976 100644
--- a/src/net.cpp
+++ b/src/net.cpp
@@ -825,7 +825,13 @@ size_t CConnman::SocketSendData(CNode& node) const
             if (!node.m_sock) {
                 break;
             }
-            nBytes = node.m_sock->Send(reinterpret_cast<const char*>(data.data()) + node.nSendOffset, data.size() - node.nSendOffset, MSG_NOSIGNAL | MSG_DONTWAIT);
+            int flags = MSG_NOSIGNAL | MSG_DONTWAIT;
+#ifdef MSG_MORE
+            if (it + 1 != node.vSendMsg.end()) {
+                flags |= MSG_MORE;
+            }
+#endif
+            nBytes = node.m_sock->Send(reinterpret_cast<const char*>(data.data()) + node.nSendOffset, data.size() - node.nSendOffset, flags);
         }
         if (nBytes > 0) {
             node.m_last_send = GetTime<std::chrono::seconds>();