Merge bitcoin/bitcoin#30244: ci: parse TEST_RUNNER_EXTRA into an array

8131bf7483c0ea10d3573c9f2e977d19d8569b7f ci: parse TEST_RUNNER_EXTRA into an array (Max Edwards)
c4762b0aa06f2654d108bc7ca05887ffd88cf6f8 test: allow excluding func test by name and arg (Max Edwards)

Pull request description:

  While working on CI I wanted to disable some functional tests so I used the `TEST_RUNNER_EXTRA` var. The problem I had was tests that have flags such as `rpc_bind.py --ipv6` must be passed in quotes otherwise the `--ipv6` portion will be considered an argument to `test_runner.py` rather than a test name.

  This change allows proper parsing of quotes and complex values such as:

  ```shell
  TEST_RUNNER_EXTRA='--exclude "rpc_bind.py --ipv6,feature_proxy.py"'
  ```

  Update:

  While testing this it was noticed that `test_runner.py` when given `--exclude "rpc_bind.py --ipv6"` will exclude all `rpc_bind.py` tests so this PR has been updated to include a change to the test runner to only exclude the specific test if you pass an arg or exclude all tests of that name if you do not pass an arg. `--exclude rpc_bind.py` will exclude all three variants and `--exclude rpc_bind --ipv6` will only exclude the IPV6 variant.

ACKs for top commit:
  maflcko:
    ACK 8131bf7483c0ea10d3573c9f2e977d19d8569b7f
  achow101:
    ACK 8131bf7483c0ea10d3573c9f2e977d19d8569b7f
  hebasto:
    ACK 8131bf7483c0ea10d3573c9f2e977d19d8569b7f, tested on Ubuntu 23.10 and Windows 11.

Tree-SHA512: 82b73f12d627f533d8e5be4a518d455ef4427a755bbe03ccd11d0bb70c7ff3cee76220b0264fcfb236661c4cf5deba034cbfc2372b96d5861f3436c21eae8264

2 files changed, 18 insertions, 9 deletions

diff --git a/ci/test/03_test_script.sh b/ci/test/03_test_script.sh
index 7fca724a77..4de27ee60c 100755
--- a/ci/test/03_test_script.sh
+++ b/ci/test/03_test_script.sh
@@ -146,8 +146,9 @@ if [ "$RUN_UNIT_TESTS_SEQUENTIAL" = "true" ]; then
 fi
 
 if [ "$RUN_FUNCTIONAL_TESTS" = "true" ]; then
-  # shellcheck disable=SC2086
-  LD_LIBRARY_PATH="${DEPENDS_DIR}/${HOST}/lib" test/functional/test_runner.py --ci "${MAKEJOBS}" --tmpdirprefix "${BASE_SCRATCH_DIR}"/test_runner/ --ansi --combinedlogslen=99999999 --timeout-factor="${TEST_RUNNER_TIMEOUT_FACTOR}" ${TEST_RUNNER_EXTRA} --quiet --failfast
+  # parses TEST_RUNNER_EXTRA as an array which allows for multiple arguments such as TEST_RUNNER_EXTRA='--exclude "rpc_bind.py --ipv6"'
+  eval "TEST_RUNNER_EXTRA=($TEST_RUNNER_EXTRA)"
+  LD_LIBRARY_PATH="${DEPENDS_DIR}/${HOST}/lib" test/functional/test_runner.py --ci "${MAKEJOBS}" --tmpdirprefix "${BASE_SCRATCH_DIR}"/test_runner/ --ansi --combinedlogslen=99999999 --timeout-factor="${TEST_RUNNER_TIMEOUT_FACTOR}" "${TEST_RUNNER_EXTRA[@]}" --quiet --failfast
 fi
 
 if [ "${RUN_TIDY}" = "true" ]; then
diff --git a/test/functional/test_runner.py b/test/functional/test_runner.py
index 86e8a10af8..bee9622709 100755
--- a/test/functional/test_runner.py
+++ b/test/functional/test_runner.py
@@ -523,14 +523,22 @@ def main():
 
     # Remove the test cases that the user has explicitly asked to exclude.
     if args.exclude:
-        exclude_tests = [test.split('.py')[0] for test in args.exclude.split(',')]
+        def print_warning_missing_test(test_name):
+            print("{}WARNING!{} Test '{}' not found in current test list.".format(BOLD[1], BOLD[0], test_name))
+        exclude_tests = [test.strip() for test in args.exclude.split(",")]
         for exclude_test in exclude_tests:
-            # Remove <test_name>.py and <test_name>.py --arg from the test list
-            exclude_list = [test for test in test_list if test.split('.py')[0] == exclude_test]
-            for exclude_item in exclude_list:
-                test_list.remove(exclude_item)
-            if not exclude_list:
-                print("{}WARNING!{} Test '{}' not found in current test list.".format(BOLD[1], BOLD[0], exclude_test))
+            if exclude_test.endswith('.py'):
+                # Remove <test_name>.py and <test_name>.py --arg from the test list
+                exclude_list = [test for test in test_list if test.split('.py')[0] == exclude_test.split('.py')[0]]
+                if not exclude_list:
+                    print_warning_missing_test(exclude_test)
+                for exclude_item in exclude_list:
+                    test_list.remove(exclude_item)
+            else:
+                try:
+                    test_list.remove(exclude_test)
+                except ValueError:
+                    print_warning_missing_test(exclude_test)
 
     if args.filter:
         test_list = list(filter(re.compile(args.filter).search, test_list))