aboutsummaryrefslogtreecommitdiff
path: root/test/functional/test_runner.py
AgeCommit message (Collapse)Author
2018-06-12Explicitly specify encoding when opening text files in Python codepracticalswift
2018-06-08qa: Remove portseed_offset from test runnerMarcoFalke
2018-05-23Merge #10757: RPC: Introduce getblockstats to plot thingsWladimir J. van der Laan
41d0476f62269027ec2193a5f80d508d789de8aa Tests: Add data file (Anthony Towns) 4cbfb6aad9ba8fa17b5e7ed3e9a36dc8a24f1fcf Tests: Test new getblockstats RPC (Jorge Timón) 35e77a0288bcac5594ff25c10c9679a161cb730b RPC: Introduce getblockstats (Jorge Timón) cda8e36f019dd181e5c3774961b4f1335e5602cb Refactor: RPC: Separate GetBlockChecked() from getblock() (Jorge Timón) Pull request description: It returns per block statistics about several things. It should be easy to add more if people think of other things to add or remove some if I went too far (but once written, why not keep it? EDIT: answer: not to test or maintain them). The currently available options are: minfee,maxfee,totalfee,minfeerate,maxfeerate,avgfee,avgfeerate,txs,ins,outs (EDIT: see updated list in the rpc call documentation) For the x axis, one can use height or block.nTime (I guess I could add mediantime if there's interest [EDIT: nobody showed interest but I implemented mediantime nonetheless, in fact there's no distinction between x or y axis anymore, that's for the caller to judge]). To calculate fees, -txindex is required. Tree-SHA512: 2b2787a3c7dc4a11df1fce62c8a4c748f5347d7f7104205d5f0962ffec1e0370c825b49fd4d58ce8ce86bf39d8453f698bcd46206eea505f077541ca7d59b18c
2018-05-22Tests: Test new getblockstats RPCJorge Timón
Includes commit from Anthony Towns @ajtowns: Tests: Save and load block and corresponding expected statistics
2018-05-09Merge #13188: qa: Remove unused option --srcdirWladimir J. van der Laan
fac1e1f qa: Remove unused option --srcdir (MarcoFalke) Pull request description: The `srcdir` option was both unused and misleading; It should have been called `builddir`. So remove it. Tree-SHA512: 2c24dcf2aa82219158b8cbbf03dd3f0f51f805f1f5f670faa1fd59e5a8d60fda120ffddadeccb058d8d3f20583b4952be7afd2df6bbefb9367d35c0f0a9fda3c
2018-05-09Merge #10267: New -includeconf argument for including external configuration ↵Wladimir J. van der Laan
files 25b7ab9 doc: Add release notes for -includeconf (Karl-Johan Alm) 0f0badd test: Test includeconf parameter. (Karl-Johan Alm) 629ff8c -includeconf=<path> support in config handler, for including external configuration files (Karl-Johan Alm) Pull request description: Fixes: #10071. Done: - adds `-includeconf=<path>`, where `<path>` is relative to `datadir` or to the path of the file being read, if in a file - protects against circular includes - updates help docs ~~~Thoughts:~~~ - ~~~I am not sure how to test this in a neat manner. Feedback on this would be nice. Will dig/think though.~~~ Tree-SHA512: cb31f1b2f69fbc0890d264948eb2e501ac05cf12f5e06a5942f9c1539eb15ea8dc3cae817f4073aecb2fcc21d0386747f14f89d990772003a76e2a6d25642553
2018-05-08qa: Remove unused option --srcdirMarcoFalke
2018-05-02test: Test includeconf parameter.Karl-Johan Alm
2018-04-27Add --failfast option to functional test runnerJames O'Beirne
Also cleans up run_test's arguments list (no more mutable default for `args`) and call site.
2018-04-25Merge #13051: qa: Normalize executable locationWladimir J. van der Laan
fa811b0 qa: Normalize executable location (MarcoFalke) Pull request description: This removes the need to override the executable locations by just reading them from the config file. Beside making the code easier to read, running individual test on Windows is now possible by default (without providing further command line arguments). Note: Of course, it is still possible to manually specify the location through the `BITCOIND` environment variable, e.g. `bitcoin-qt`. Tree-SHA512: bee6d22246796242d747120ca18aaab089f73067de213c9111182561985c5912228a0b0f7f9eec025ecfdb44db031f15652f30d67c489d481c995bb3232a7ac7
2018-04-24Merge #12953: Deprecate accountsWladimir J. van der Laan
cead28b [docs] Add release notes for deprecated 'account' API (John Newbery) 72c9575 [wallet] [tests] Add tests for accounts/labels APIs (John Newbery) 109e05d [wallet] [rpc] Deprecate wallet 'account' API (John Newbery) 3576ab1 [wallet] [rpc] Deprecate account RPC methods (John Newbery) 3db1ba0 [tests] Set -deprecatedrpc=accounts in tests (John Newbery) 4e671f0 [tests] Rename rpc_listtransactions.py to wallet_listtransactions.py (John Newbery) a28b907 [wallet] [rpc] Remove duplicate entries in rpcwallet.cpp's CRPCCommand table (John Newbery) Pull request description: Deprecate all accounts functionality and make it only accessible by using `-deprecatedrpc=accounts`. Accounts specific RPCs, account arguments, and account related results all require the `-deprecatedrpc=accunts` startup option now in order to see account things. Several wallet functional tests use the accounts system. Those tests are unchanged, except to start the nodes with `-deprecatedrpc=accounts`. We can slowly migrate those tests to use the 'label' API instead of the 'account' API before accounts are fully removed. Tree-SHA512: 89f4ae2fe6de4a1422f1817b0997ae22d63ab5a1a558362ce923a3871f3e42963405d6573c69c27f1764679cdee5b51bf52202cc407f1361bfd8066d652f3f37
2018-04-21qa: Normalize executable locationMarcoFalke
2018-04-16[tests] Rename rpc_listtransactions.py to wallet_listtransactions.pyJohn Newbery
listtransactions is a wallet RPC. The test name should indicate that this is a wallet test.
2018-04-13test: Add rpc_bind test to default-run testsWladimir J. van der Laan
Skip the parts that cannot be run on the host due to lack of IPv6 support or a second interface to bind on, and warn appropriately. Without no strong requirements (besides being Linux only, in which case the test is skipped) left, just add this test to the default in test_runner. Includes suggested changes by John Newbery.
2018-04-02[tests] Remove bip9-softforks.pyJohn Newbery
bip9-sofforks.py was intended to be a generic test for versionbits deployments. However, it only tests CSV activation and was not updated to test segwit activation. CSV activation is tested by bip68-112-113-p2p.py, so this test is duplicated effort. Rather than try to update it to use the BitcoinTestFramework, just remove it.
2018-04-02Merge #11742: rpc: Add testmempoolacceptWladimir J. van der Laan
b55555d rpc: Add testmempoolaccept (MarcoFalke) Pull request description: To check if a single raw transaction makes it into the current transaction pool, one had to call `sendrawtransaction`. However, on success, this adds the transaction to the mempool with no easy way to undo. The call `testmempoolaccept` is introduced to provide a way to solely check the result without changing the mempool state. Tree-SHA512: 5afd9311190135cee8fc1f229c7d39bf893f1028f29e28d34f70df820198ff97b4bf86b41cbbd6e6c36a5c30073cefa92d541c74a4939c7a2a6fa283dfd41b63
2018-03-30[tests] Test starting bitcoind with -h and -versionJohn Newbery
2018-03-28test: List any failed tests at the end of test_runner outputAnthony Towns
Change sorting output to put failed tests at the end of test_runner output.
2018-03-28test: Make summary row bold-red if any test failedWladimir J. van der Laan
Make the summary row of the test runner bold red if any test fails. This helps visibility if something fails.
2018-03-27Merge #12653: Allow to optional specify the directory for the blocks storageWladimir J. van der Laan
a192636 -blocksdir: keep blockindex leveldb database in datadir (Jonas Schnelli) f38e4fd QA: Add -blocksdir test (Jonas Schnelli) 386a6b6 Allow to optional specify the directory for the blocks storage (Jonas Schnelli) Pull request description: Since the actual block files taking up more and more space, it may be desirable to have them stored in a different location then the data directory (use case: SSD for chainstate, etc., HD for blocks). This PR adds a `-blocksdir` option that allows one to keep the blockfiles and the blockindex external from the data directory (instead of creating symlinks). I fist had an option to keep the blockindex within the datadir, but seems to make no sense since accessing the index will (always) lead to access (r/w) the block files. Tree-SHA512: f8b9e1a681679eac25076dc30e45e6e12d4b2d9ac4be907cbea928a75af081dbcb0f1dd3e97169ab975f73d0bd15824c00c2a34638f3b284b39017171fce2409
2018-03-24rpc: Add testmempoolacceptMarcoFalke
2018-03-19Rename wallet_accounts.py testRussell Yanofsky
This is a separate commit because changing the test at the same time as renaming it breaks git (and github) rename detection.
2018-03-18Avoiding 'file' function name from python2 with more descriptive variable namingJeff Rade
2018-03-18QA: Add -blocksdir testJonas Schnelli
2018-03-17Append scripts to new test_list array to fix bad assignmentJeff Rade
2018-03-16[Trivial] Simplify if-else blocks and more descriptive variable namingJeff Rade
2018-03-01Merge #11882: Disable default fallbackfee on mainnetWladimir J. van der Laan
3f592b8 [QA] add wallet-rbf test (Jonas Schnelli) 8222e05 Disable wallet fallbackfee by default on mainnet (Jonas Schnelli) Pull request description: Removes the default fallback fee on mainnet (but keeps it on testnet/regtest). Transactions using the fallbackfee in case the fallback fee has not been set are getting rejected. Tree-SHA512: e54d2594b7f954e640cc513a18b0bfbe189f15e15bdeed4fe02b7677f939bca1731fef781b073127ffd4ce08a595fb118259b8826cdaa077ff7d5ae9495810db
2018-02-25[QA] add wallet-rbf testJonas Schnelli
2018-02-15Fix typo in test_runner.py causing errorMeshCollider
2018-02-12Merge #11858: qa: Prepare tests for WindowsWladimir J. van der Laan
faefd29 qa: Prepare functional tests for Windows (MarcoFalke) Pull request description: * Pass `sys.executable` when calling a python script via the subprocess module * Don't remove the log file while it is still open and written to * Properly use os.pathsep and os.path.sep when modifying the PATH environment variable * util-tests: Use os.path.join for Windows compatibility Ref: #8227 Tree-SHA512: c507a536af104b3bde4366b6634099db826532bd3e7c35d694b5883c550920643b3eab79c76703ca67e1044ed09979e855088f7324321c8d52112514e334d614
2018-01-30[tests] Require all tests to follow naming conventionAnthony Towns
2018-01-25[tests] Remove EXPECTED_VIOLATION_COUNTAnthony Towns
2018-01-25[tests] Rename misc functional tests.Anthony Towns
2018-01-25[tests] Rename rpc_* functional tests.Anthony Towns
2018-01-25[tests] Rename p2p_* functional tests.Anthony Towns
2018-01-25[tests] Rename wallet_* functional tests.Anthony Towns
2018-01-25[tests] Rename feature_* functional tests.Anthony Towns
2018-01-24qa: Prepare functional tests for WindowsMarcoFalke
* Pass `sys.executable` when calling a python script via the subprocess module * Don't remove the log file while it is still open and written to * Properly use os.pathsep and os.path.sep when modifying the PATH environment variable * util-tests: Use os.path.join for Windows compatibility
2018-01-19test_runner: Readable output if create_cache.py failsRussell Yanofsky
Without this change, create_cache.py process output is shown as a byte() object with \n escapes in a single line that is hard to read.
2018-01-15Merge #11796: [tests] Functional test naming conventionWladimir J. van der Laan
5fecd84 [tests] Remove redundant import in blocktools.py test (Anthony Towns) 9b20bb4 [tests] Check tests conform to naming convention (Anthony Towns) 7250b4e [tests] README.md nit fixes (Anthony Towns) 82b2712 [tests] move witness util functions to blocktools.py (John Newbery) 1e10854 [tests] [docs] update README for new test naming scheme (John Newbery) Pull request description: Splitting #11774 into two parts -- this part updates the README with the proposed naming convention, and adds some checks to test_runner.py that the number of tests violating the naming convention doesn't increase too much. Idea is this part of the change should not introduce merge conflicts or require much rebasing, so reviews of the complicated bits won't become invalidated too often; while the second part will just be file renames, which will require regular rebasing and will introduce merge conflicts with pending PRs, but can be merged later, and should also be much easier to review, since it will only include relatively trivial changes. Tree-SHA512: b96557d41714addbbfe2aed62fb5a48639eaeb1eb3aba30ac1b3a86bb3cb8d796c6247f9c414c4695c4bf54c0ec9968ac88e2f88fb62483bc1a2f89368f7fc80
2018-01-12Merge #11970: Add test coverage for bitcoin-cli multiwallet callsMarcoFalke
a14dbff39e Allow multiwallet.py to be used with --usecli (Russell Yanofsky) f6ade9ce1a [tests] allow tests to be run with --usecli (John Newbery) ff9a363ff7 TestNodeCLI batch emulation (Russell Yanofsky) ca9085afc5 Prevent TestNodeCLI.args mixups (Russell Yanofsky) fcfb952bca Improve TestNodeCLI output parsing (Russell Yanofsky) Pull request description: Lack of test coverage was pointed out by @jnewbery in https://github.com/bitcoin/bitcoin/pull/11687#discussion_r158133900 Tree-SHA512: 5f10e31abad11a5edab0da4e2515e39547adb6ab9e55e50427ab2eb7ec9a43d6b896b579b15863e5edc9beee7d8bf1c84d9dabd247be0760a1b9ae39e1e8ee02
2018-01-10Merge #11403: SegWit wallet supportJonas Schnelli
b224a47a1 Add address_types test (Pieter Wuille) 7ee54fd7c Support downgrading after recovered keypool witness keys (Pieter Wuille) 940a21932 SegWit wallet support (Pieter Wuille) f37c64e47 Implicitly know about P2WPKH redeemscripts (Pieter Wuille) 57273f2b3 [test] Serialize CTransaction with witness by default (Pieter Wuille) cf2c0b6f5 Support P2WPKH and P2SH-P2WPKH in dumpprivkey (Pieter Wuille) 37c03d3e0 Support P2WPKH addresses in create/addmultisig (Pieter Wuille) 3eaa003c8 Extend validateaddress information for P2SH-embedded witness (Pieter Wuille) 30a27dc5b Expose method to find key for a single-key destination (Pieter Wuille) 985c79552 Improve witness destination types and use them more (Pieter Wuille) cbe197470 [refactor] GetAccount{PubKey,Address} -> GetAccountDestination (Pieter Wuille) 0c8ea6380 Abstract out IsSolvable from Witnessifier (Pieter Wuille) Pull request description: This implements a minimum viable implementation of SegWit wallet support, based on top of #11389, and includes part of the functionality from #11089. Two new configuration options are added: * `-addresstype`, with options `legacy`, `p2sh`, and `bech32`. It controls what kind of addresses are produced by `getnewaddress`, `getaccountaddress`, and `createmultisigaddress`. * `-changetype`, with the same options, and by default equal to `-addresstype`, that controls what kind of change is used. All wallet private and public keys can be used for any type of address. Support for address types dependent on different derivation paths will need a major overhaul of how our internal detection of outputs work. I expect that that will happen for a next major version. The above also applies to imported keys, as having a distinction there but not for normal operations is a disaster for testing, and probably for comprehension of users. This has some ugly effects, like needing to associate the provided label to `importprivkey` with each style address for the corresponding key. To deal with witness outputs requiring a corresponding redeemscript in wallet, three approaches are used: * All SegWit addresses created through `getnewaddress` or multisig RPCs explicitly get their redeemscripts added to the wallet file. This means that downgrading after creating a witness address will work, as long as the wallet file is up to date. * All SegWit keys in the wallet get an _implicit_ redeemscript added, without it being written to the file. This means recovery of an old backup will work, as long as you use new software. * All keypool keys that are seen used in transactions explicitly get their redeemscripts added to the wallet files. This means that downgrading after recovering from a backup that includes a witness address will work. These approaches correspond to solutions 3a, 1a, and 5a respectively from https://gist.github.com/sipa/125cfa1615946d0c3f3eec2ad7f250a2. As argued there, there is no full solution for dealing with the case where you both downgrade and restore a backup, so that's also not implemented. `dumpwallet`, `importwallet`, `importmulti`, `signmessage` and `verifymessage` don't work with SegWit addresses yet. They're remaining TODOs, for this PR or a follow-up. Because of that, several tests unexpectedly run with `-addresstype=legacy` for now. Tree-SHA512: d425dbe517c0422061ab8dacdc3a6ae47da071450932ed992c79559d922dff7b2574a31a8c94feccd3761c1dffb6422c50055e6dca8e3cf94a169bc95e39e959
2018-01-09Add address_types testPieter Wuille
Improvements and cleanups by John Newbery
2018-01-09SegWit wallet supportPieter Wuille
This introduces two command line flags (-addresstype and -changetype) which control the type of addresses/outputs created by the GUI and RPCs. Certain RPCs allow overriding these (`getnewaddress` and `getrawchangeaddress`). Supported types are "legacy" (P2PKH and P2SH-multisig), "p2sh-segwit" (P2SH-P2WPKH and P2SH-P2WSH-multisig), and "bech32" (P2WPKH and P2WSH-multisig). A few utility functions are added to the wallet to construct different address type and to add the necessary entries to the wallet file to be compatible with earlier versions (see `CWallet::LearnRelatedScripts`, `GetDestinationForKey`, `GetAllDestinationsForKey`, `CWallet::AddAndGetDestinationForScript`).
2018-01-08Allow multiwallet.py to be used with --usecliRussell Yanofsky
Add test coverage for bitcoin-cli multiwallet calls.
2018-01-03Merge #12062: Increment MIT Licence copyright header year on files modified ↵Wladimir J. van der Laan
in 2017 595a7ba Increment MIT Licence copyright header year on files modified in 2017 (Akira Takizawa) Pull request description: Edited via: $ contrib/devtools/copyright_header.py update . ps) It is the same commit as #9450 Tree-SHA512: 274bfcd6cf2914315ed52f6db773a68800ce9d6bd225a3142654483f0bbc3fd865009e62f9d954f65765d038c626e55d2a64e37e16843809adc2f67abe659b6d
2018-01-03Increment MIT Licence copyright header year on files modified in 2017Akira Takizawa
2017-12-21qa: Note on test order in test_runnerMarcoFalke
2017-12-20Add configuration/argument testingMeshCollider
2017-12-08[tests] Check tests conform to naming conventionAnthony Towns
Extra-Author: John Newbery <john@johnnewbery.com>