Merge #11468: [tests] Make comp test framework more debuggable

748157913 [tests] Make comp test framework more debuggable (John Newbery)

Pull request description:

  We should remove the comparison test framework entirely (see #10603).

  Until we do that, let's make it a bit more debuggable. Currently, if there's an assert in the framework, it's very difficult to track down where we are in the test generator. Make the logging a bit better to help with debugging.

  Before this PR:

  ```
  → ./p2p-fullblocktest.py
  2017-10-09 14:05:11.302000 TestFramework (INFO): Initializing test directory /tmp/user/1000/testzdnax_yr
  2017-10-09 14:05:11.557000 TestFramework.mininode (INFO): Connecting to Bitcoin Node: 127.0.0.1:11975
  2017-10-09 14:05:11.712000 TestFramework.comptool (INFO): Test 1: PASS
  2017-10-09 14:05:11.947000 TestFramework.comptool (INFO): Test 2: PASS
  2017-10-09 14:05:12.057000 TestFramework.comptool (INFO): Test 3: PASS
  2017-10-09 14:05:12.058000 TestFramework (ERROR): Assertion failed
  Traceback (most recent call last):
    File "/home/ubuntu/bitcoin/test/functional/test_framework/test_framework.py", line 117, in main
      self.run_test()
    File "./p2p-fullblocktest.py", line 72, in run_test
      self.test.run()
    File "/home/ubuntu/bitcoin/test/functional/test_framework/comptool.py", line 306, in run
      assert test_number != 4
  AssertionError
  2017-10-09 14:05:12.059000 TestFramework (INFO): Stopping nodes
  2017-10-09 14:05:14.203000 TestFramework (WARNING): Not cleaning up dir /tmp/user/1000/testzdnax_yr
  2017-10-09 14:05:14.204000 TestFramework (ERROR): Test failed. Test logging available at /tmp/user/1000/testzdnax_yr/test_framework.log
  ```

  With this PR:

  ```
  → ./p2p-fullblocktest.py
  2017-10-09 14:03:54.069000 TestFramework (INFO): Initializing test directory /tmp/user/1000/testuey7t3tf
  2017-10-09 14:03:54.329000 TestFramework.mininode (INFO): Connecting to Bitcoin Node: 127.0.0.1:11783
  2017-10-09 14:03:54.383000 TestFramework.comptool (INFO): Running test 1: ./p2p-fullblocktest.py line 184
  2017-10-09 14:03:54.496000 TestFramework.comptool (INFO): Running test 2: ./p2p-fullblocktest.py line 193
  2017-10-09 14:03:54.758000 TestFramework.comptool (INFO): Running test 3: ./p2p-fullblocktest.py line 205
  2017-10-09 14:03:54.867000 TestFramework.comptool (INFO): Running test 4: ./p2p-fullblocktest.py line 208
  2017-10-09 14:03:54.867000 TestFramework (ERROR): Assertion failed
  Traceback (most recent call last):
    File "/home/ubuntu/bitcoin/test/functional/test_framework/test_framework.py", line 117, in main
      self.run_test()
    File "./p2p-fullblocktest.py", line 72, in run_test
      self.test.run()
    File "/home/ubuntu/bitcoin/test/functional/test_framework/comptool.py", line 309, in run
      assert test_number != 4
  AssertionError
  2017-10-09 14:03:54.868000 TestFramework (INFO): Stopping nodes
  2017-10-09 14:03:56.950000 TestFramework (WARNING): Not cleaning up dir /tmp/user/1000/testuey7t3tf
  2017-10-09 14:03:56.950000 TestFramework (ERROR): Test failed. Test logging available at /tmp/user/1000/testuey7t3tf/test_framework.log
  ```

Tree-SHA512: 5525958b0098d661c281bd955c92c72bf81359464376e96d44e6c88f18aea57ba08270ecd564edda4b47d674e3b27d20e5b1060544bf8dd5c6a68103d2bb35b8

1 files changed, 5 insertions, 5 deletions

diff --git a/test/functional/test_framework/comptool.py b/test/functional/test_framework/comptool.py
index 39a81a267f..03f967ba71 100755
--- a/test/functional/test_framework/comptool.py
+++ b/test/functional/test_framework/comptool.py
@@ -295,8 +295,11 @@ class TestManager():
         # Wait until verack is received
         self.wait_for_verack()
 
-        test_number = 1
-        for test_instance in self.test_generator.get_tests():
+        test_number = 0
+        tests = self.test_generator.get_tests()
+        for test_instance in tests:
+            test_number += 1
+            logger.info("Running test %d: %s line %s" % (test_number, tests.gi_code.co_filename, tests.gi_frame.f_lineno))
             # We use these variables to keep track of the last block
             # and last transaction in the tests, which are used
             # if we're not syncing on every block or every tx.
@@ -397,9 +400,6 @@ class TestManager():
                 if (not self.check_mempool(tx.sha256, tx_outcome)):
                     raise AssertionError("Mempool test failed at test %d" % test_number)
 
-            logger.info("Test %d: PASS" % test_number)
-            test_number += 1
-
         [ c.disconnect_node() for c in self.connections ]
         self.wait_for_disconnections()
         self.block_store.close()