aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2018-04-04Remove direct bitcoin calls from qt/utilitydialog.cppRussell Yanofsky
2018-04-04Remove direct bitcoin calls from qt/bitcoingui.cppRussell Yanofsky
2018-04-04Remove direct bitcoin calls from qt/optionsmodel.cppRussell Yanofsky
2018-04-04Remove direct bitcoin calls from qt/bitcoin.cppRussell Yanofsky
2018-04-03Add src/interface/README.mdRussell Yanofsky
2018-04-03Merge #12846: [moveonly] Extract HelpRequested to dry up the help options ↵MarcoFalke
testing b386970d07 [moveonly] Extract HelpRequested to dry up the help options testing (Ben Woosley) Pull request description: This ensures consistency across interfaces and makes the version handling more clear. Tree-SHA512: d3f46d34dae6cf98902b0bbb279ada65c3215a25f69e5ff98b88e68f37a6b027ded265d15c12303998e31b390aa30fdb689455c61c983ab4b7527cbce8f4ec61
2018-04-02[moveonly] Extract HelpRequested to dry up the help options testingBen Woosley
This ensures consistency across interfaces and makes the version handling more clear.
2018-04-02Merge #12861: [tests] Stop feature_block.py from blowing up memory.MarcoFalke
89fe5feea2 [tests] Stop feature_block.py from blowing up memory. (John Newbery) Pull request description: The new P2PDataStore class was sending full blocks in headers messages, which meant that calls to send_blocks_and_test() would blow up memory if called with a large number of blocks. Fix that by only sending headers in headers messages. This means that python should use just over 1GB for feature_block.py (with bitcoind also using just over 1GB). That's the same as before the feature_block.py refactor. Tree-SHA512: 796ea35584748ceb7b8fa36c732a461fb924dafe0b4c52d3eccf21a00fbdb65aef41ce1d91f027aad50cde6df5d30e985aaef474cb743975c06762975469cbbb
2018-04-02[tests] Stop feature_block.py from blowing up memory.John Newbery
The new P2PDataStore class was sending full blocks in headers messages, which meant that calls to send_blocks_and_test() would blow up memory if called with a large number of blocks. Fix that by only sending headers in headers messages.
2018-04-02Merge #11742: rpc: Add testmempoolacceptWladimir J. van der Laan
b55555d rpc: Add testmempoolaccept (MarcoFalke) Pull request description: To check if a single raw transaction makes it into the current transaction pool, one had to call `sendrawtransaction`. However, on success, this adds the transaction to the mempool with no easy way to undo. The call `testmempoolaccept` is introduced to provide a way to solely check the result without changing the mempool state. Tree-SHA512: 5afd9311190135cee8fc1f229c7d39bf893f1028f29e28d34f70df820198ff97b4bf86b41cbbd6e6c36a5c30073cefa92d541c74a4939c7a2a6fa283dfd41b63
2018-04-02Merge #12857: Avoid travis lint-include-guards errorWladimir J. van der Laan
177b875 Avoid travis lint-include-guards error (Chun Kuan Lee) Pull request description: Tree-SHA512: 174c8bc2223dc3b317ab84cc09066eace229d58e061058f1d2fbf503c3587b79dc772e37afc18773379f92822fef4183652f27e9b1f6dc4abc4309af11c931e0
2018-04-02Avoid travis lint-include-guards errorChun Kuan Lee
2018-04-01Merge #12757: Clarify include guard naming conventionMarcoFalke
3bcc0059b8 Add lint-include-guards.sh which checks include guard consistency (practicalswift) 8fd6af89a0 Fix missing or inconsistent include guards (practicalswift) 8af65d96f4 Document include guard convention (practicalswift) Pull request description: * **Documentation**: Document include guard convention * **Fix**: Fix missing or inconsistent include guards * **Regression test**: Add `lint-include-guards.sh` which checks include guard consistency Tree-SHA512: 8171878f60fd08ccbea943a11e835195750592abb9d7ab74eaa4265ae7fac523b1da9d31ca13d6ab73dd596e49986bfb7593c696e5f39567c93e610165bc2acc
2018-04-01Merge #12719: tests: Add note about test suite naming convention in ↵MarcoFalke
developer-notes.md db983beba6 tests: Add lint-tests.sh which checks the test suite naming convention (practicalswift) 5fd864fe8a tests: Rename test suits not following the test suite naming convention (practicalswift) 7b4a296a71 tests: Add note about test suite naming convention (practicalswift) Pull request description: Changes: * Add note about test suite naming convention * Fix exceptions * Add regression test Rationale: * Consistent naming of test suites makes programmatic test running of specific tests/subsets of tests easier * Explicit is better than implicit Before this commit: ``` $ contrib/devtools/lint-tests.sh The test suite in file src/test/foo_tests.cpp should be named "foo_tests". Please make sure the following test suites follow that convention: src/test/blockchain_tests.cpp:BOOST_FIXTURE_TEST_SUITE(blockchain_difficulty_tests, BasicTestingSetup) src/test/prevector_tests.cpp:BOOST_FIXTURE_TEST_SUITE(PrevectorTests, TestingSetup) src/wallet/test/coinselector_tests.cpp:BOOST_FIXTURE_TEST_SUITE(coin_selection_tests, WalletTestingSetup) src/wallet/test/crypto_tests.cpp:BOOST_FIXTURE_TEST_SUITE(wallet_crypto, BasicTestingSetup) $ ``` After this commit: ``` $ contrib/devtools/lint-tests.sh $ ``` Tree-SHA512: 7258ab9a6b9b8fc1939efadc619e2f2f02cfce8034c7f2e5dc5ecc769aa12e17f6fb8e363817feaf15c026c5b958b2574525b8d2d3f6be69658679bf8ceea9e9
2018-04-01Merge #12284: Remove assigned but never used local variables. Enable Travis ↵MarcoFalke
checking for unused local variables. ea04bf7862 Enable flake8 warning F841 ("local variable 'foo' is assigned to but never used") (practicalswift) 169f3e8637 Remove assigned but never used local variables (practicalswift) Pull request description: Remove assigned but never used local variables. Enable Travis checking for unused local variables. Tree-SHA512: d6052ec9044c5d1f03d874ea3c8addd5a156779213ef9200f89d3ae53230f2fd1691aff405c3dae14178e5ef09912c4432e92f606ef4a5220ed9daa140cdee81
2018-04-01Merge #12843: [tests] Test starting bitcoind with -h and -versionMarcoFalke
63048ec73d [tests] Test starting bitcoind with -h and -version (John Newbery) Pull request description: Test that starting bitcoind/bitcoin-qt with `-h` and `-version` works as expected. Prompted by https://github.com/bitcoin/bitcoin/pull/10762#commitcomment-28345993, which is a nullpointer dereference triggered by starting bitcoin-qt with `-h`. On master, this test passes when run over bitcoind, but fails when running over bitcoin-qt. I used xvfb as a virtual frame buffer to test: ``` BITCOIND=/home/ubuntu/bitcoin/src/qt/bitcoin-qt xvfb-run ./feature_help.py --nocleanup 2018-03-30T17:09:37.767000Z TestFramework (INFO): Initializing test directory /tmp/user/1000/testdi4dre13 2018-03-30T17:09:37.767000Z TestFramework (INFO): Start bitcoin with -h for help text 2018-03-30T17:09:37.841000Z TestFramework (ERROR): Assertion failed Traceback (most recent call last): File "/home/ubuntu/bitcoin/test/functional/test_framework/test_framework.py", line 126, in main self.run_test() File "./feature_help.py", line 25, in run_test assert_equal(ret_code, 0) File "/home/ubuntu/bitcoin/test/functional/test_framework/util.py", line 39, in assert_equal raise AssertionError("not(%s)" % " == ".join(str(arg) for arg in (thing1, thing2) + args)) AssertionError: not(-11 == 0) 2018-03-30T17:09:37.842000Z TestFramework (INFO): Stopping nodes Traceback (most recent call last): File "./feature_help.py", line 42, in <module> HelpTest().main() File "/home/ubuntu/bitcoin/test/functional/test_framework/test_framework.py", line 149, in main self.stop_nodes() File "/home/ubuntu/bitcoin/test/functional/test_framework/test_framework.py", line 273, in stop_nodes node.stop_node() File "/home/ubuntu/bitcoin/test/functional/test_framework/test_node.py", line 141, in stop_node self.stop() File "/home/ubuntu/bitcoin/test/functional/test_framework/test_node.py", line 87, in __getattr__ assert self.rpc_connected and self.rpc is not None, "Error: no RPC connection" AssertionError: Error: no RPC connection ``` Passes for bitcoind and bitcoin-qt when run on #12836. Longer term, we should consider running functional tests over bitcoin-qt in one of the Travis jobs. Tree-SHA512: 0c2f40f3d5f0e78c3a1b07dbee8fd383eebab27ed0bf2a98a5b9cc66613dbd7b70e363c56163a37e02f68ae7ff7b3ae1769705d0e110ca68a00f8693315730a4
2018-04-01Merge #11817: [tests] Change feature_csv_activation.py to use ↵MarcoFalke
BitcoinTestFramework 12982682a6 [tests] Change feature_csv_activation.py to use BitcoinTestFramework (John Newbery) db7ffb9d1b [tests] Move utility functions in feature_csv_activation.py out of class. (John Newbery) 0842edf9ee [tests] Remove nested loops from feature_csv_activation.py (John Newbery) 2e511d5424 [tests] improve logging in feature_csv_activation.py (John Newbery) 6f7f5bc002 [tests] fix flake8 nits in feature_csv_activation.py (John Newbery) Pull request description: Next step in #10603. - first four commits tidy up bip68-112-113-p2p.py - fifth commit removes usage of ComparisonTestFramework Tree-SHA512: 34466be12280096ad92ac842f9c594ae40c19be9b4edc73c1e37964d03d55f4e75b80cea50c9940404096effc23705671503a883a7b7773b5866a29f653ba710
2018-03-30[tests] Test starting bitcoind with -h and -versionJohn Newbery
2018-03-30Merge #12804: [tests] Fix intermittent rpc_net.py failure.MarcoFalke
5a67c0524e [tests] Fix intermittent rpc_net.py failure. (John Newbery) Pull request description: rpc_net.py would intermittently fail on Travis, probably due to assuming that two consecutive RPC calls were atomic. Fix this by trying the test up to three times and only failing if all three attempts fail. fixes #11778 Tree-SHA512: f2f5047e05114ad2110377e6221ce057680c240952971fb33863834587d2250896c6b697ba27eef739cd0ab23faf47dfae8cafadc9833cbfab5a6f7e36dae5e2
2018-03-30Merge #12713: Track negated options in the option parserMarcoFalke
f7683cba7b Track negated arguments in the argument paser. (Evan Klitzke) 4f872b2450 Add additional tests for GetBoolArg() (Evan Klitzke) Pull request description: This change explicitly enable tracking negated options in the option parser. A negated option is one passed with a `-no` prefix. For example, `-nofoo` is the negated form of `-foo`. Negated options were originally added in the 0.6 release. The change here allows code to explicitly distinguish between cases like `-nofoo` and `-foo=0`, which was not possible previously. The option parser does not have any changed semantics as a result of this change, and existing code will parse options just as it did before. The motivation for this change is to provide a way to disable options that are otherwise not boolean options. For example, the `-debuglogfile` option is normally interpreted as a string, where the value is the log file name. With this change a user can pass in `-nodebuglogfile` and the code can see that it was explicitly negated, and use that to disable the log file. This change originally split out from #12689. Tree-SHA512: cd5a7354eb03d2d402863c7b69e512cad382781d9b8f18c1ab104fc46d45a712530818d665203082da39572c8a42313c5be09306dc2a7227cdedb20ef7314823
2018-03-30Merge #12780: Reduce variable scopesMarcoFalke
6a318e48a6 Reduce variable scopes (practicalswift) Pull request description: Reduce variable scopes. Tree-SHA512: 5c7735344024cb6cd310e739886dc811a64b640a0d6aac8b3d04f49e5987d6ff6676d978890bc84f1460527d92217176a79b007f0bf6b4147c04abfec2c67714
2018-03-30Merge #12820: contrib: Fix check-doc script regexesMarcoFalke
0c17e27630 init: Remove help text for non-existent -fuzzmessagestest arg (MarcoFalke) 136084470c contrib: Fix check-doc script regexes (MarcoFalke) Pull request description: Fixup the regexes to properly find all used args. The regex should now match all of the getter and setter methods of the `ArgsManager`. See https://dev.visucore.com/bitcoin/doxygen/class_args_manager.html#pub-methods Before: ``` Args used : 159 Args documented : 188 Args undocumented: 0 Args unknown : 29 ``` After: ``` Args used : 183 Args documented : 188 Args undocumented: 0 Args unknown : 5 ``` Tree-SHA512: 1a7fb7ea55b2f6030358a1055d8f2c19b31f69d0603be0b009e6e603564014b4e2bb824357c9d43d0fba3ce7159b7c4e7eaa60b3f962053d94f73d0e626294fc
2018-03-30Merge #12810: [Tests] Fix a typo at assert_start_raises_init_error() and use ↵MarcoFalke
specific exception for initialization error 8394300859 [Tests] fix a typo in TestNode.assert_start_raises_init_error() (Roman Zeyde) Pull request description: `self.wait_util_stopped()` should be `self.wait_until_stopped()`. Also, use a specific Exception subclass for indicating node failure to start (instead of using `AssetionError` and an `except Exception` clause). Following https://travis-ci.org/bitcoin/bitcoin/jobs/359066226#L2726 and depending on #12806 (which fixes the root cause of the Travis test failure). Tree-SHA512: 7bd5a95586a412472ef9dffdb086789d7275ddaf862724e21cebb3418d0c97e6d89b4d1a58375e42114060d028403d6eab89e3a1e9a833ffe8dadf3439ab1fe2
2018-03-30Merge #12839: test: Remove travis checkout depthWladimir J. van der Laan
9df1a9c test: Remove travis checkout depth (Wladimir J. van der Laan) Pull request description: Travis tests on branches of non-head commits are failing, because the depth of 1 doesn't allow checking them out. E.g.: https://travis-ci.org/bitcoin/bitcoin/jobs/360227253 Remove `depth` as was the case before fa44af5cd2152a21da9ef3e48c073a668bf2df27, so that Travis can determine the minimum depth to check out. Tree-SHA512: f193939cbef2aadd8d88fe522bd5ea84a3b72899f043279e3e8b546a70875a85561adf53e3f70c288833382eb71576c8fb6ef4db0588894c609fb58f5afce6df
2018-03-30test: Remove travis checkout depthWladimir J. van der Laan
Tests on branches of non-head commits are failing, because the depth of 1 doesn't allow checking them out. Remove `depth` as was the case before fa44af5cd2152a21da9ef3e48c073a668bf2df27, so that Travis can determine the minimum depth to check out.
2018-03-30Merge #12740: Add native support for serializing char arrays without FLATDATAWladimir J. van der Laan
a7c45bc Add native support for serializing char arrays without FLATDATA (Pieter Wuille) Pull request description: Support is added to serialize arrays of type `char` or `unsigned char` directly, without any wrappers. All invocations of the `FLATDATA` wrappers that are obsoleted by this are removed. This includes a patch by @ryanofsky to make `char` casting type safe. The serialization of `CSubNet` is changed to serialize a `bool` directly rather than though `FLATDATA`. This makes the serialization independent of the size of the bool type (and will use 1 byte everywhere). This is a small change taken from #10785. Tree-SHA512: a41f61ca5fdc2fadb2d0e1702351a58a23841d551f505292a9542602cdb19f90d8944b8df14b872810a56bd201648fa4c0e958f3e9427fe829886284e85b9bfd
2018-03-30Merge #12752: [MOVEONLY] Move compressor utility functions out of classWladimir J. van der Laan
76a9aac Move compressor utility functions out of class (Pieter Wuille) Pull request description: This is a refactor from #10785 with no functionality change. Move the compresion utility functions out of CScriptCompressor, as a preparation for making the class templated. I'm submitting it as a separate PR as I think it's a general improvement to code readability, and to reduce the diff further on. Tree-SHA512: 3b3d17c2b96e43f752f512dd573296a6bb15cae165fbe3c79212a0970f5196a62a59a821d5100f29638af1e7461c9171f3dccb8112f005ee08da0ec7fe0073fd
2018-03-30Merge #12822: Revert 7deba93bdc76616011a9f493cbc203d60084416f and fix ↵Wladimir J. van der Laan
expired-key-sigs properly 9471576 [verify-commits] Add some additional useful documentation. (Matt Corallo) de7e931 Add Marco-expired-key-signed-commits to allow-revsig-commits (Matt Corallo) 99f6d48 Revert "test: Update trust git root". (Matt Corallo) Pull request description: 7deba93bdc76616011a9f493cbc203d60084416f was took the wrong approach to updating verify-commits for a key expiration. Namely, adding each commit to allow-revsig-commits should have been done instead, allowing them to still be validated, but with the expired key. Tree-SHA512: 9fdc67eda8f6daa95082f6c1a2af81beb730a9ff3f8cf930bb2311fe29b5f05e1f89259aba5f112153ca2e9c62577cf60d31b4c8e9ac1bf3f5506e78f8401378
2018-03-30Merge #12172: Bugfix: RPC: savemempool: Don't save until LoadMempool() is ↵Wladimir J. van der Laan
finished cb1e319 Bugfix: RPC: savemempool: Don't save until LoadMempool() is finished (Jorge Timón) Pull request description: Fixes https://github.com/bitcoin/bitcoin/issues/12142 The tests are a little bit slow, mempool_persist.py goes from about 20 s to about 120 s in my hardware. Perhaps there's a better way to test this. Tree-SHA512: 9e6c24b32a9cf3774e8f0bd81c035b0deb53fba5ac3eb2532d85900579d21cef8a1135b75a4fa0a9d883e3822eb35e7d4b47a0838abf99789039205041962629
2018-03-29Merge #12692: Add configure options for various -fsanitize flagsWladimir J. van der Laan
6feb46c Add --with-sanitizers option to configure (Evan Klitzke) Pull request description: This adds configure options for `-fsanitize=address`, `-fsanitize=thread`, and `-fsanitize=undefined` which are all disabled by default. These flags are useful for developers who wish to do additional safety checking. Note that some of these are mutually incompatible, and these may have a large performance overhead. There's some kind of strange logic required to properly check for the availability of these flags in a way that works on both GCC and Clang, hopefully the comments make it clear what's going on. Tree-SHA512: 2d6fe402799110e59ee452dddf37f7ca2d26a7fecec50be25c8a134e4a20beb31f1e8f438dffd443641562418075896d1eeb450623425b272d80e05e3027a587
2018-03-29Merge #12829: Python3 fixupWladimir J. van der Laan
f50975b [contrib] fixup symbol-check.py Python3 support (John Newbery) 5de2b18 [contrib] fixup security-check.py Python3 support (John Newbery) Pull request description: security-check.py and symbol-check.py were broken by #11881. Fix them. Tree-SHA512: 86de3d6dc3292b1ae4cc04c2d7d7dbbf39c9270551d7b224b8d8b19e3184c30c897dbf823200403706d06bb405c0decad5cfd690cb2c0312992a235a4ffcf6bf
2018-03-29Enable flake8 warning F841 ("local variable 'foo' is assigned to but never ↵practicalswift
used")
2018-03-29Remove assigned but never used local variablespracticalswift
2018-03-29Merge #11773: [tests] Change feature_block.py to use BitcoinTestFrameworkWladimir J. van der Laan
265d7c4 [tests] Improve assert message when wait_until() fails (John Newbery) ebf053a [tests] Change feature_block.py to use BitcoinTestFramework (John Newbery) fc02c12 [tests] Add logging to feature_block.py (John Newbery) 3898c4f [tests] Tidy up feature_block.py (John Newbery) 5cd01d2 [tests] Fix flake8 warnings in feature_block.py (John Newbery) Pull request description: Next step in #10603. - first three commits tidy up feature_block.py - fourth commit removes usage of ComparisonTestFramework Longer term, it would be better to separate net_processing testing from validation testing, but I think this is still a useful PR, since it moves us away from the comparison test framework. Tree-SHA512: d0bb3ad22ad0aa1222877f4212bff075f9ce358e99c69c26d9913e4b346d931b8380e744434a9f6f37812c352cdaa75791691565bfeb18afcb619c06c6ca32a3
2018-03-29Merge #10762: [wallet] Remove Wallet dependencies from init.cppWladimir J. van der Laan
c7ec524 [wallet] Add dummy wallet init class (John Newbery) 49baa4a [wallet] Use global g_wallet_init_interface to init/destroy the wallet. (John Newbery) caaf972 [wallet] Create wallet init interface. (John Newbery) 5fb5421 [wallet] Move wallet init functions into WalletInit class. (John Newbery) Pull request description: This continues the work of #7965. This PR, along with several others, would remove the remaining dependencies from libbitcoin_server.a on libbitcoin_wallet.a. To create the interface, I've just translated all the old init.cpp wallet function calls into an interface class. I've not done any thinking about whether it makes sense to change that interface by combining/splitting those calls. This is a purely internal interface, so there's no problem in changing it later. Tree-SHA512: 32ea57615229c33fd1a7f2f29ebc11bf30337685f7211baffa899823ef74b65dcbf068289c557a161c5afffb51fdc38a2ee8180720371f64d433b12b0615cf3f
2018-03-29[verify-commits] Add some additional useful documentation.Matt Corallo
2018-03-29Merge #12759: [Docs] Improve formatting of developer notesWladimir J. van der Laan
0bd2ec5 Improve formatting of developer notes (Evan Klitzke) Pull request description: The developer notes file has gotten pretty large and unwieldly. This reorganizes some content, most notably by adding a TOC to the page. Compare how the page looks in [my branch](https://github.com/eklitzke/bitcoin/blob/developer-notes/doc/developer-notes.md) vs [master](https://github.com/eklitzke/bitcoin/blob/master/doc/developer-notes.md). I know there's long-term interest in moving a lot of this content to the bitcoin-core/docs repo on GitHub, but this makes things better now. The TOC format here is a semi-standard extension to Markdown files that you may have seen on GitHub pages in other projects. The `<!-- markdown-toc -->` comments used by these tools to know where the TOC starts/ends. The following tools all understand this format: * [Sphinx](http://www.sphinx-doc.org/en/master/) * [Pandoc](https://pandoc.org/) * [markdown-toc.el](https://github.com/ardumont/markdown-toc) (what I used) * various plugins for [vim](https://github.com/mzlogin/vim-markdown-toc) and other editors I used this TOC extension at a previous job, and my observation is that it's fine if people just edit the TOC manually. It's just text and it's not a huge deal if it gets a little out of sync. It can also be regenerated at any time by anyone with any of these tools. Tree-SHA512: 298d1605ea5e8bfc0f75e70570c23ebd6891e4ffcdedd24fefadc23edd6e4b96509d8d102209868468a1b3ddbe2c3b8462698cdda8b9421348b5bc6f7b8d0cb8
2018-03-29Merge #12790: [Tests] Use blockmaxweight where tests previously had blockmaxsizeWladimir J. van der Laan
b466f6b [Tests] Use blockmaxweight where tests previously had blockmaxsize (Conor Scott) Pull request description: Fix for #12768: `-blockmaxsize` has been removed, but some tests were using this feature, so update with `-blockmaxweight` Tree-SHA512: 5f6d643daee43366c6e61f4154a3920efb4cef1455e483575cf19b0f95bc6e56668360c1562fa18f85b336e48f64e482bd29b1ecb227d7c78c4344d7f5d32ed3
2018-03-29Merge #12784: Fix bug in memory usage calculation (unintended integer division)Wladimir J. van der Laan
a16c6d2 Fix error in memory usage calculation (unintended integer division) (practicalswift) Pull request description: Fix bug in memory usage calculation (unintended integer division). Tree-SHA512: 2df1f00c5282581c61e1fd55fef3fabc02161b5a47d8f1795b05d57117245ff3d1ee861dd689eebe0185f28176cea428007e799d5c43a1ce5dc704123439f967
2018-03-29Merge #12787: rpc: Adjust ifdef to avoid unreachable codeWladimir J. van der Laan
61f8298 rpc: Adjust ifdef to avoid unreachable code (practicalswift) Pull request description: Adjust `ifdef` to avoid unreachable code. Introduced in 1e79c055cd30d21ba5f8c7f81ef911d5d4e295a8. Tree-SHA512: c775cc9181e4034f26c5b219974e06886435275933249b169d2bc8bc98f639c4027e1e7d991f43bded62146a141acee6d3be1f2b313042d9bbc0a5d2e71d6c7c
2018-03-29Merge #12495: Increase LevelDB max_open_filesWladimir J. van der Laan
ccedbaf Increase LevelDB max_open_files unless on 32-bit Unix. (Evan Klitzke) Pull request description: Currently we set `max_open_files = 64` on all architectures due to concerns about file descriptor exhaustion. This is extremely expensive due to how LevelDB is designed. When a LevelDB file handle is opened, a bloom filter and block index are decoded, and some CRCs are checked. Bloom filters and block indexes in open table handles can be checked purely in memory. This means that when doing a key lookup, if a given table file may contain a given key, all of the lookup operations can happen completely in RAM until the block itself is fetched. In the common case fetching the block is one disk seek, because the block index stores its physical offset. This is the ideal case, and what we want to happen as often as possible. If a table file handle is not open in the table cache, then in addition to the regular system calls to open the file, the block index and bloom filter need to be decoded before they can be checked. This is expensive and is something we want to avoid. The current setting of 64 file handles means that on a synced node, only about 4% of key lookups can be satisifed by table file handles that are actually open and in memory. The original concerns about file descriptor exhaustion are unwarranted on most systems because: * On 64-bit POSIX hosts LevelDB will open up to 1000 file descriptors using `mmap()`, and it does not retain an open file descriptor for such files. * On Windows non-socket files do not interfere with the main network `select()` loop, so the same fd exhaustion issues do not apply there. This change keeps the default `max_open_files` value (which is 1000) on all systems except 32-bit POSIX hosts (which do not use `mmap()`). Open file handles use about 20 KB of memory (for the block index), so the extra file handles do not cause much memory overhead. At most 1000 will be open, and a fully synced node right now has about 1500 such files. Profile of `loadblk` thread before changes: https://monad.io/maxopenfiles-master.svg Profile of `loadblk` thread after changes: https://monad.io/maxopenfiles-increase.svg Tree-SHA512: de54f77d57e9f8999eaf8d12592aab5b02f5877be8fa727a1f42cf02da2693ce25846445eb19eb138ce4e5045d1c65e14054df72faf3ff32c7655c9cfadd27a9
2018-03-29Merge #12821: contrib: Remove unused import stringWladimir J. van der Laan
05120ee contrib: Remove unused import string (MarcoFalke) Pull request description: Tiny oversight in #11881, that broke travis. Tree-SHA512: 805c31cbbd74cf0a6c71bf1a7989c3ff1d179e24d434b70de307f20ddf358f6db8c2e08aa46dcaf2c75a44ae571b1dcf5ba15184be81680e19a13ad5a23a0991
2018-03-28Increase LevelDB max_open_files unless on 32-bit Unix.Evan Klitzke
This change significantly increases IBD performance by increasing the amount of the UTXO index that can remain in memory. To ensure this doesn't cause problems in the future, a static_assert on the LevelDB version has been added, which must be updated by anyone upgrading LevelDB.
2018-03-29Bugfix: RPC: savemempool: Don't save until LoadMempool() is finishedJorge Timón
2018-03-28[contrib] fixup symbol-check.py Python3 supportJohn Newbery
2018-03-28[contrib] fixup security-check.py Python3 supportJohn Newbery
2018-03-28[tests] Fix intermittent rpc_net.py failure.John Newbery
rpc_net.py would intermittently fail on Travis, probably due to assuming that two consecutive RPC calls were atomic. Fix this by only testing that amounts are bounded above and below rather than equal.
2018-03-28Add --with-sanitizers option to configureEvan Klitzke
This enables the use of different compiler sanitizers, coresponding to the -fsanitize option in GCC and Clang.
2018-03-28Improve formatting of developer notesEvan Klitzke
Summary of changes: * Add a TOC to the page * Make tips and tricks section use h3 headings * Reformat and clarify some sections
2018-03-28Add Marco-expired-key-signed-commits to allow-revsig-commitsMatt Corallo