Merge #19866: eBPF Linux tracepoints

22eb7930a6ae021438aa0b8e750170534944f296 tracing: add tracing framework (William Casarin)
933ab8a720cb9b3341adec4109cffb6dc5b322a5 build: detect sys/sdt.h for eBPF tracing (William Casarin)

Pull request description:

  Instead of writing ad-hoc logging everywhere (eg: #19509), we can take advantage of linux user static defined traces, aka. USDTs ( not the stablecoin :sweat_smile: )

  The linux kernel can hook into these tracepoints at runtime, but otherwise they have little to no performance impact. Traces can pass data which can be printed externally via tools such as bpftrace. For example, here's one that prints incoming and outgoing network messages:

  # Examples

  ## Network Messages

  ```
  #!/usr/bin/env bpftrace

  BEGIN
  {
    printf("bitcoin net msgs\n");
    @start = nsecs;
  }

  usdt:./src/bitcoind:net:push_message
  {
    $ip = str(arg0);
    $peer_id = (int64)arg1;
    $command = str(arg2);
    $data_len = arg3;
    $data = buf(arg3,arg4);
    $t = (nsecs - @start) / 100000;

    printf("%zu outbound %s %s %zu %d %r\n", $t, $command, $ip, $peer_id, $data_len, $data);

    @outbound[$command]++;
  }

  usdt:./src/bitcoind:net:process_message
  {
    $ip = str(arg0);
    $peer_id = (int64)arg1;
    $command = str(arg2);
    $data_len = arg3;
    $data = buf(arg3,arg4);
    $t = (nsecs - @start) / 100000;

    printf("%zu inbound %s %s %zu %d %r\n", $t, $command, $ip, $peer_id, $data_len, $data);

    @inbound[$ip, $command]++;
  }

  ```

      $ sudo bpftrace netmsg.bt

  output: https://jb55.com/s/b11312484b601fb3.txt

  if you look at the bottom of the output you can see a histogram of all the messages grouped by message type and IP. nice!

  ## IBD Benchmarking

  ```
  #!/usr/bin/env bpftrace
  BEGIN
  {
    printf("IBD to 500,000 bench\n");
  }

  usdt:./src/bitcoind:CChainState:ConnectBlock
  {
    $height = (uint32)arg0;

    if ($height == 1) {
      printf("block 1 found, starting benchmark\n");
      @start = nsecs;
    }

    if ($height >= 500000) {
      @end = nsecs;
      @duration = @end - @start;
      exit();
    }
  }

  END {
    printf("duration %d ms\n", @duration / 1000000)
  }
  ```
  This one hooks into ConnectBlock and prints the IBD time to height 500,000 starting from the first call to ConnectBlock

  Userspace static tracepoints give lots of flexibility without invasive logging code. It's also more flexible than ad-hoc logging code, allowing you to instrument many different aspects of the system without having to enable per-subsystem logging.

  Other ideas: tracepoints for lock contention, threads, what else?

  Let me know what ya'll think and if this is worth adding to bitcoin.

  ## TODO

  - [ ] docs?
  - [x] Integrate systemtap-std-dev/libsystemtap into build (provides the <sys/sdt.h> header)
  - [x] ~dtrace macos support? (is this still a thing?)~ going to focus on linux for now

ACKs for top commit:
  laanwj:
    Tested ACK 22eb7930a6ae021438aa0b8e750170534944f296
  0xB10C:
    Tested ACK 22eb7930a6ae021438aa0b8e750170534944f296

Tree-SHA512: 69242242112b679c8a12a22b3bc50252c305894fb3055ae6e13d5f56221d858e58af1d698af55e23b69bdb7abedb5565ac6b45fa5144087b77a17acd04646a75

1 files changed, 18 insertions, 0 deletions

diff --git a/configure.ac b/configure.ac
index b64c22450c..1592ed352f 100644
--- a/configure.ac
+++ b/configure.ac
@@ -132,6 +132,12 @@ AC_ARG_WITH([bdb],
   [use_bdb=$withval],
   [use_bdb=auto])
 
+AC_ARG_ENABLE([ebpf],
+  [AS_HELP_STRING([--enable-ebpf],
+  [enable eBPF tracing (default is yes if sys/sdt.h is found)])],
+  [use_ebpf=$enableval],
+  [use_ebpf=yes])
+
 AC_ARG_WITH([miniupnpc],
   [AS_HELP_STRING([--with-miniupnpc],
   [enable UPNP (default is yes if libminiupnpc is found)])],
@@ -1295,6 +1301,16 @@ if test x$enable_wallet != xno; then
     fi
 fi
 
+if test x$use_ebpf != xno; then
+  AC_CHECK_HEADER([sys/sdt.h], [have_sdt=yes], [have_sdt=no])
+else
+  have_sdt=no
+fi
+
+if test x$have_sdt = xyes; then
+  AC_DEFINE([ENABLE_TRACING], [1], [Define to 1 to enable eBPF user static defined tracepoints])
+fi
+
 dnl Check for libminiupnpc (optional)
 if test x$use_upnp != xno; then
   AC_CHECK_HEADERS(
@@ -1689,6 +1705,7 @@ AM_CONDITIONAL([TARGET_WINDOWS], [test x$TARGET_OS = xwindows])
 AM_CONDITIONAL([ENABLE_WALLET],[test x$enable_wallet = xyes])
 AM_CONDITIONAL([USE_SQLITE], [test "x$use_sqlite" = "xyes"])
 AM_CONDITIONAL([USE_BDB], [test "x$use_bdb" = "xyes"])
+AM_CONDITIONAL([ENABLE_TRACING],[test x$have_sdt = xyes])
 AM_CONDITIONAL([ENABLE_TESTS],[test x$BUILD_TEST = xyes])
 AM_CONDITIONAL([ENABLE_FUZZ],[test x$enable_fuzz = xyes])
 AM_CONDITIONAL([ENABLE_FUZZ_LINK_ALL],[test x$enable_danger_fuzz_link_all = xyes])
@@ -1855,6 +1872,7 @@ echo "  with bench    = $use_bench"
 echo "  with upnp     = $use_upnp"
 echo "  with natpmp   = $use_natpmp"
 echo "  use asm       = $use_asm"
+echo "  ebpf tracing  = $have_sdt"
 echo "  sanitizers    = $use_sanitizers"
 echo "  debug enabled = $enable_debug"
 echo "  gprof enabled = $enable_gprof"