aboutsummaryrefslogtreecommitdiff
path: root/.github/ISSUE_TEMPLATE
diff options
context:
space:
mode:
authorAva Chow <github@achow101.com>2024-04-26 16:00:37 -0400
committerAva Chow <github@achow101.com>2024-04-26 16:06:37 -0400
commit1ffbd96349820cae7f076ae3253d8a9d28155fd2 (patch)
treefa6e787955bfb1313381ea2923854abd0ee54746 /.github/ISSUE_TEMPLATE
parent7973a670915632b75a6aa16f24f98b936865c48f (diff)
parentf19f0a2e5af6c2a64900f1f229e21b6f1668bd3d (diff)
Merge bitcoin/bitcoin#29771: test: Run framework unit tests in parallel
f19f0a2e5af6c2a64900f1f229e21b6f1668bd3d test: Run framework unit tests in parallel (tdb3) Pull request description: Functional test framework unit tests are currently run prior to all other functional tests. This PR enables execution of the test framework unit tests in parallel with the functional tests, rather than before the functional tests, saving runtime and more efficiently using available cores. This is a follow up to https://github.com/bitcoin/bitcoin/pull/29470#issuecomment-1962313977 ### New behavior: 1) When running all tests, the framework unit tests are run in parallel with the other tests (unless explicitly skipped with `--exclude`). This parallelization introduces marginal time savings when running all tests, depending on the machine used. As an example, a 2-3% time savings (9 seconds) was observed on a machine using `--jobs=18` (with 18 available cores). 2) When running specific functional tests, framework unit tests are now skipped by default. Framework unit tests can be added by including `feature_framework_unit_tests.py` in the list of specific tests being executed. The rationale for skipping by default is that if the tester is running specific functional tests, there is a conscious decision to focus testing, and choosing to run all tests (where unit tests are run by default) would be a next step. 3) The `--skipunit` option is now removed since unit tests are parallelized (they no longer delay other tests). Unit tests are treated equally as functional tests. ### Implementation notes: Since `TextTestRunner` can be noisy (even with verbosity=0, and therefore trigger job failure through the presence of non-failure stderr output), the approach taken was to send output to stdout, and forward test result (as determined by `TestResult` returned). This aligns with the previous check for unit test failure (`if not result.wasSuccessful():`). This approach was tested by inserting `self.assertEquals(True, False)` into test_framework/address.py and seeing specifics of the failure reported. ``` 135/302 - feature_framework_unit_tests.py failed, Duration: 0 s stdout: .F ====================================================================== FAIL: test_bech32_decode (test_framework.address.TestFrameworkScript.test_bech32_decode) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/dev/myrepos/bitcoin/test/functional/test_framework/address.py", line 228, in test_bech32_decode self.assertEqual(True, False) AssertionError: True != False ---------------------------------------------------------------------- Ran 2 tests in 0.003s FAILED (failures=1) stderr: ``` There was an initial thought to parallelize the execution of the unit tests themselves (i.e. run the 12 unit test files in parallel), however, this is not anticipated to further reduce runtime meaningfully and is anticipated to add unnecessary complexity. ACKs for top commit: maflcko: ACK f19f0a2e5af6c2a64900f1f229e21b6f1668bd3d 🌽 achow101: ACK f19f0a2e5af6c2a64900f1f229e21b6f1668bd3d kevkevinpal: Approach ACK f19f0a2e5af6c2a64900f1f229e21b6f1668bd3d Tree-SHA512: ab9f82c30371b2242bc7a263ea0e25d35e68e2ddf223d2a55498ad940d1e5b73bba76cce8b264d71e2ed31b753430d8ef8d57efc1e4fd9ced7fb845e27f4f47e
Diffstat (limited to '.github/ISSUE_TEMPLATE')
0 files changed, 0 insertions, 0 deletions