Merge #19764: depends: Split boost into build/host packages + bump + cleanup

f190343c96520db254d6689f8f24c9eb36bead6b depends: boost: Specify cflags+compileflags (Carl Dong)
b2328b7989997652af52295a4b2e988e68b8428b depends: boost: Remove unnecessary _archiver_ (Carl Dong)
ab9e047cc2226198d22384c286cb30c4c7d51e83 depends: boost: Cleanup toolset selection (Carl Dong)
86002e7e90d7c78d5a76a159bf17db48aec94f94 depends: boost: Cleanup architecture/address-model (Carl Dong)
d7048fa73fafaf0d89d891d98e51a251a64a1495 depends: boost: Disable all compression (Carl Dong)
9cf2ee54d366c3f5fcfce96244988858599e7e95 depends: boost: Split into non-/native packages (Carl Dong)
a57b49856033f3de4a861367b70aba85aec58cf8 depends: boost: Bump to 1.71.0 (Carl Dong)
800655ff3199782b803e671f5baa77ff90a5591d depends: boost: Refer to version in URL (Carl Dong)

Pull request description:

  This PR improves the robustness of our boost package in depends, most notably:
  1. Bumps boost from `1.70.0` to `1.71.0`, because `1.71.0`:
  	1. Removes the need to patch out the unused variable.
  		https://github.com/bitcoin/bitcoin/blob/f8462a6d2794be728cf8550f45d19a354aae59cf/depends/packages/boost.mk#L36
  		Upstream boost patched it out in https://github.com/boostorg/process/commit/d20b64cf37a773e452b84ce619752e3785be756c, which was first included in the `1.71.0` release
  	2. Comes packaged with a version of `b2` which allows us to override its `CXX` and `CXXFLAGS`. Previously, choosing a toolset while building `b2` such as `clang` or `gcc` would force `b2`'s build system to invoke the compiler as a bare, hardcoded `clang` or `gcc`. However, our `depends` build system often want to customize this behaviour, adding extra flags or invoking the compiler by an alternate name. So this is useful.
  		1. Commit where `CXX` was introduced: https://github.com/boostorg/build/commit/374f96516a6210687cdf63c987710501634bcac9
  		2. Commit where `CXXFLAGS` was introduced: https://github.com/boostorg/build/commit/5d49abc1f291573d4bdcd2ee647b05a66f9c6497
  2. The boost package is now split into `native_b2` and `boost`, better representing what actually happens.
  	- In our `depends` build system, we have a distinction between `native` packages and non-`native` packages. The output of `native` packages are meant to be used on the machine that's performing the build, and the output of non-`native` packages are meant to be used on/for the machine that will ultimately be running bitcoin. Previously, `boost` existed in `depends` as a non-`native` package, but that's partly inaccurate because the `./bootstrap.sh` invocation in its `$(package)_config_cmds` stage actually produced a binary called `b2`, which is run on the machine that's performing the build. This means that `b2` is a `native` package which is being built in an environment set up for the non-`native` package `boost`. This reveals a hidden unintended behavior in our `depends` build system: for linux->darwin cross builds, we use `gcc` for `native` packages, and `clang` for non-`native` packages. But `b2` was actually being built using `clang`, since it was being built in an environment set up for non-`native` packages.

  theuni you might be interested in taking a look

ACKs for top commit:
  laanwj:
    Concept and code review ACK f190343c96520db254d6689f8f24c9eb36bead6b

Tree-SHA512: f8b728a34da4f0a9a985a819a5762f2fc2689ea24c7eba1d24d26dfbd4c59f202227c699b0a4069dab10b6329cf9f4c6dd95082685776ee43dd5f7b659acdef1

4 files changed, 38 insertions, 43 deletions

diff --git a/depends/packages/boost.mk b/depends/packages/boost.mk
index d8bce108b1..0446b07009 100644
--- a/depends/packages/boost.mk
+++ b/depends/packages/boost.mk
@@ -1,31 +1,27 @@
 package=boost
-$(package)_version=1_70_0
-$(package)_download_path=https://dl.bintray.com/boostorg/release/1.70.0/source/
+$(package)_version=1_71_0
+$(package)_download_path=https://dl.bintray.com/boostorg/release/$(subst _,.,$($(package)_version))/source/
 $(package)_file_name=boost_$($(package)_version).tar.bz2
-$(package)_sha256_hash=430ae8354789de4fd19ee52f3b1f739e1fba576f0aded0897c3c2bc00fb38778
-$(package)_patches=unused_var_in_process.patch
+$(package)_sha256_hash=d73a8da01e8bf8c7eda40b4c84915071a8c8a0df4a6734537ddde4a8580524ee
+$(package)_dependencies=native_b2
 
 define $(package)_set_vars
 $(package)_config_opts_release=variant=release
 $(package)_config_opts_debug=variant=debug
 $(package)_config_opts=--layout=tagged --build-type=complete --user-config=user-config.jam
-$(package)_config_opts+=threading=multi link=static -sNO_BZIP2=1 -sNO_ZLIB=1
+$(package)_config_opts+=threading=multi link=static -sNO_COMPRESSION=1
 $(package)_config_opts_linux=target-os=linux threadapi=pthread runtime-link=shared
 $(package)_config_opts_darwin=target-os=darwin runtime-link=shared
 $(package)_config_opts_mingw32=target-os=windows binary-format=pe threadapi=win32 runtime-link=static
-$(package)_config_opts_x86_64_mingw32=address-model=64
-$(package)_config_opts_i686_mingw32=address-model=32
-$(package)_config_opts_i686_linux=address-model=32 architecture=x86
-$(package)_config_opts_i686_android=address-model=32
-$(package)_config_opts_aarch64_android=address-model=64
-$(package)_config_opts_x86_64_android=address-model=64
-$(package)_config_opts_armv7a_android=address-model=32
-$(package)_toolset_$(host_os)=gcc
-$(package)_toolset_darwin=clang
+$(package)_config_opts_x86_64=architecture=x86 address-model=64
+$(package)_config_opts_i686=architecture=x86 address-model=32
+$(package)_config_opts_aarch64=address-model=64
+$(package)_config_opts_armv7a=address-model=32
 ifneq (,$(findstring clang,$($(package)_cxx)))
-   $(package)_toolset_$(host_os)=clang
+$(package)_toolset_$(host_os)=clang
+else
+$(package)_toolset_$(host_os)=gcc
 endif
-$(package)_archiver_$(host_os)=$($(package)_ar)
 $(package)_config_libraries=filesystem,system,thread,test
 $(package)_cxxflags=-std=c++11 -fvisibility=hidden
 $(package)_cxxflags_linux=-fPIC
@@ -33,18 +29,17 @@ $(package)_cxxflags_android=-fPIC
 endef
 
 define $(package)_preprocess_cmds
-  patch -p1 < $($(package)_patch_dir)/unused_var_in_process.patch && \
-  echo "using $($(package)_toolset_$(host_os)) : : $($(package)_cxx) : <cxxflags>\"$($(package)_cxxflags) $($(package)_cppflags)\" <linkflags>\"$($(package)_ldflags)\" <archiver>\"$($(package)_archiver_$(host_os))\" <striper>\"$(host_STRIP)\"  <ranlib>\"$(host_RANLIB)\" <rc>\"$(host_WINDRES)\" : ;" > user-config.jam
+  echo "using $($(package)_toolset_$(host_os)) : : $($(package)_cxx) : <cflags>\"$($(package)_cflags)\" <cxxflags>\"$($(package)_cxxflags)\" <compileflags>\"$($(package)_cppflags)\" <linkflags>\"$($(package)_ldflags)\" <archiver>\"$($(package)_ar)\" <striper>\"$(host_STRIP)\"  <ranlib>\"$(host_RANLIB)\" <rc>\"$(host_WINDRES)\" : ;" > user-config.jam
 endef
 
 define $(package)_config_cmds
-  ./bootstrap.sh --without-icu --with-libraries=$($(package)_config_libraries) --with-toolset=$($(package)_toolset_$(host_os))
+  ./bootstrap.sh --without-icu --with-libraries=$($(package)_config_libraries) --with-toolset=$($(package)_toolset_$(host_os)) --with-bjam=b2
 endef
 
 define $(package)_build_cmds
-  ./b2 -d2 -j2 -d1 --prefix=$($(package)_staging_prefix_dir) $($(package)_config_opts) toolset=$($(package)_toolset_$(host_os)) stage
+  b2 -d2 -j2 -d1 --prefix=$($(package)_staging_prefix_dir) $($(package)_config_opts) toolset=$($(package)_toolset_$(host_os)) stage
 endef
 
 define $(package)_stage_cmds
-  ./b2 -d0 -j4 --prefix=$($(package)_staging_prefix_dir) $($(package)_config_opts) toolset=$($(package)_toolset_$(host_os)) install
+  b2 -d0 -j4 --prefix=$($(package)_staging_prefix_dir) $($(package)_config_opts) toolset=$($(package)_toolset_$(host_os)) install
 endef
diff --git a/depends/packages/native_b2.mk b/depends/packages/native_b2.mk
new file mode 100644
index 0000000000..aaa37cdcfa
--- /dev/null
+++ b/depends/packages/native_b2.mk
@@ -0,0 +1,20 @@
+package=native_b2
+$(package)_version=$(boost_version)
+$(package)_download_path=$(boost_download_path)
+$(package)_file_name=$(boost_file_name)
+$(package)_sha256_hash=$(boost_sha256_hash)
+$(package)_build_subdir=tools/build/src/engine
+ifneq (,$(findstring clang,$($(package)_cxx)))
+$(package)_toolset_$(host_os)=clang
+else
+$(package)_toolset_$(host_os)=gcc
+endif
+
+define $(package)_build_cmds
+  CXX="$($(package)_cxx)" CXXFLAGS="$($(package)_cxxflags)" ./build.sh "$($(package)_toolset_$(host_os))"
+endef
+
+define $(package)_stage_cmds
+  mkdir -p "$($(package)_staging_prefix_dir)"/bin/ && \
+  cp b2 "$($(package)_staging_prefix_dir)"/bin/
+endef
diff --git a/depends/packages/packages.mk b/depends/packages/packages.mk
index ca06b6dd7e..4b4cc7d9ff 100644
--- a/depends/packages/packages.mk
+++ b/depends/packages/packages.mk
@@ -22,6 +22,8 @@ multiprocess_native_packages = native_libmultiprocess native_capnp
 
 darwin_native_packages = native_ds_store native_mac_alias
 
+$(host_arch)_$(host_os)_native_packages += native_b2
+
 ifneq ($(build_os),darwin)
 darwin_native_packages += native_cctools native_cdrkit native_libdmg-hfsplus
 endif
diff --git a/depends/patches/boost/unused_var_in_process.patch b/depends/patches/boost/unused_var_in_process.patch
deleted file mode 100644
index 722f7bb5ea..0000000000
--- a/depends/patches/boost/unused_var_in_process.patch
+++ /dev/null
@@ -1,22 +0,0 @@
-commit dbd95cdaefdea95307d004f019a1c394cf9389f0
-Author: fanquake <fanquake@gmail.com>
-Date:   Mon Aug 17 20:15:17 2020 +0800
-
-    Remove unused variable in Boost Process
-
-    This causes issues with our linters / CI.
-
-    Can be removed once depends Boost is 1.71.0 or later.
-
-diff --git a/boost/process/detail/posix/wait_group.hpp b/boost/process/detail/posix/wait_group.hpp
-index 9dc249803..2502d9772 100644
---- a/boost/process/detail/posix/wait_group.hpp
-+++ b/boost/process/detail/posix/wait_group.hpp
-@@ -137,7 +137,6 @@ inline bool wait_until(
-
-     do
-     {
--        int ret_sig = 0;
-         int status;
-         if ((::waitpid(timeout_pid, &status, WNOHANG) != 0)
-             && (WIFEXITED(status) || WIFSIGNALED(status)))