Merge #16374: test: Enable passing wildcard test names to test runner from root

e142ee03e7a139168aa1dbf5910c616f60d25042 doc: describe how to pass wildcard names to test runner (Jon Atack)
6a7a70b8cf05a82737c72020fd2b0eebc97cb5e4 test: enable passing wildcards with path to test runner (Jon Atack)

Pull request description:

  Currently, passing wildcard testname args to the test runner from outside the test/functional/ directory does not work, even though developers expect it to. See these recent IRC discussions for more background: http://www.erisian.com.au/bitcoin-core-dev/log-2019-07-10.html#l-262 (lines 262 to 323) and http://www.erisian.com.au/bitcoin-core-dev/log-2019-07-11.html#l-134.

  1. [BUGFIX] Enable passing wildcards with paths. Examples:
      - `test/functional/test_runner.py test/functional/wallet*`
      - `functional/test_runner.py functional/wallet*`
      - `test/functional/test_runner.py ./test/functional/tool* test/functional/mempool*`
      - A current limitation this PR does not change: 9 test files with arguments in their filename are not picked up by wildcard search.

  2. [Docs] Describe how to pass wildcard names (multiple and with paths) to the test runner in test/README.md.

ACKs for top commit:
  jnewbery:
    tested ACK e142ee03e7a139168aa1dbf5910c616f60d25042
  jachiang:
    Tested ACK https://github.com/bitcoin/bitcoin/commit/e142ee03e7a139168aa1dbf5910c616f60d25042. Thanks a lot for this fix!
  MarcoFalke:
    ACK e142ee03e7a139168aa1dbf5910c616f60d25042, fine with me

Tree-SHA512: cb3d994880cdc9b8918546b573a25faa5b4c7339826ac7cfe20f076aac6e731a34271609c0cf5a7ee5e4a2d5ae205298319d24bf36ef5b5d569a1a0c57883e54

2 files changed, 38 insertions, 4 deletions

diff --git a/test/README.md b/test/README.md
index ecea3213ab..8f08b7afe4 100644
--- a/test/README.md
+++ b/test/README.md
@@ -49,6 +49,29 @@ You can run any combination (incl. duplicates) of tests by calling:
 test/functional/test_runner.py <testname1> <testname2> <testname3> ...
 ```
 
+Wildcard test names can be passed, if the paths are coherent and the test runner
+is called from a `bash` shell or similar that does the globbing. For example,
+to run all the wallet tests:
+
+```
+test/functional/test_runner.py test/functional/wallet*
+functional/test_runner.py functional/wallet* (called from the test/ directory)
+test_runner.py wallet* (called from the test/functional/ directory)
+```
+
+but not
+
+```
+test/functional/test_runner.py wallet*
+```
+
+Combinations of wildcards can be passed:
+
+```
+test/functional/test_runner.py ./test/functional/tool* test/functional/mempool*
+test_runner.py tool* mempool*
+```
+
 Run the regression test suite with:
 
 ```
diff --git a/test/functional/test_runner.py b/test/functional/test_runner.py
index 96786e41b4..71738be1d0 100755
--- a/test/functional/test_runner.py
+++ b/test/functional/test_runner.py
@@ -270,11 +270,22 @@ def main():
     test_list = []
     if tests:
         # Individual tests have been specified. Run specified tests that exist
-        # in the ALL_SCRIPTS list. Accept the name with or without .py extension.
-        tests = [test + ".py" if ".py" not in test else test for test in tests]
+        # in the ALL_SCRIPTS list. Accept names with or without a .py extension.
+        # Specified tests can contain wildcards, but in that case the supplied
+        # paths should be coherent, e.g. the same path as that provided to call
+        # test_runner.py. Examples:
+        #   `test/functional/test_runner.py test/functional/wallet*`
+        #   `test/functional/test_runner.py ./test/functional/wallet*`
+        #   `test_runner.py wallet*`
+        #   but not:
+        #   `test/functional/test_runner.py wallet*`
+        # Multiple wildcards can be passed:
+        #   `test_runner.py tool* mempool*`
         for test in tests:
-            if test in ALL_SCRIPTS:
-                test_list.append(test)
+            script = test.split("/")[-1]
+            script = script + ".py" if ".py" not in script else script
+            if script in ALL_SCRIPTS:
+                test_list.append(script)
             else:
                 print("{}WARNING!{} Test '{}' not found in full test list.".format(BOLD[1], BOLD[0], test))
     elif args.extended: