diff options
| author | merge-script <fanquake@gmail.com> | 2024-12-02 10:22:53 +0000 |
|---|---|---|
| committer | merge-script <fanquake@gmail.com> | 2024-12-02 10:22:53 +0000 |
| commit | 097c66f61487a2cfb4998bf7c53e4b1d048cf612 (patch) | |
| tree | 75fd3ccdc8d2cf6730c73e005d8ad6b1bbe72e2a /src | |
| parent | 68daaea0e487d482dd7bf667353446ca7c6d506d (diff) | |
| parent | b73d3319377a4c9d7e2dd279c3d106002585bc36 (diff) | |
Merge bitcoin/bitcoin#30039: dbwrapper: Bump LevelDB max file size to 32 MiB to avoid system slowdown from high disk cache flush rate
b73d3319377a4c9d7e2dd279c3d106002585bc36 dbwrapper: Bump max file size to 32 MiB (Maciej S. Szmigiero)
Pull request description:
The default max file size for LevelDB is 2 MiB, which results in the LevelDB compaction code generating ~4 disk cache flushes per second when syncing with the Bitcoin network.
These disk cache flushes are triggered by `fdatasync()` syscall issued by the LevelDB compaction code when reaching the max file size.
If the database is on a HDD this flush rate brings the whole system to a crawl.
It also results in very slow throughput since 2 MiB * 4 flushes per second is about 8 MiB / second max throughput, while even an old HDD can pull 100 - 200 MiB / second streaming throughput.
Increase the max file size for LevelDB to 128 MiB instead so the flush rate drops to about 1 flush / 2 seconds and the system no longer gets so sluggish.
The max file size value chosen also matches the `MAX_BLOCKFILE_SIZE` file size setting already used by the block storage.
ACKs for top commit:
l0rinc:
ACK b73d3319377a4c9d7e2dd279c3d106002585bc36
davidgumberg:
ACK https://github.com/bitcoin/bitcoin/commit/b73d3319377a4c9d7e2dd279c3d106002585bc36
andrewtoth:
ACK b73d3319377a4c9d7e2dd279c3d106002585bc36
TheCharlatan:
ACK b73d3319377a4c9d7e2dd279c3d106002585bc36
willcl-ark:
ACK b73d3319377a4c9d7e2dd279c3d106002585bc36
tdb3:
ACK b73d3319377a4c9d7e2dd279c3d106002585bc36
laanwj:
ACK b73d3319377a4c9d7e2dd279c3d106002585bc36
Tree-SHA512: 5d8fb9ad1ea643fb3e42a9c59f6fc90cc5cc3b82c06d9b8d59de3a5a926fabaeb78efb51b608b1e7925f49d82dfcbd5b72c552993879789f33201efe57c278f3
Diffstat (limited to 'src')
| -rw-r--r-- | src/dbwrapper.cpp | 1 | ||||
| -rw-r--r-- | src/dbwrapper.h | 1 |
2 files changed, 2 insertions, 0 deletions
diff --git a/src/dbwrapper.cpp b/src/dbwrapper.cpp index e0f153fd61..8fb366515a 100644 --- a/src/dbwrapper.cpp +++ b/src/dbwrapper.cpp @@ -147,6 +147,7 @@ static leveldb::Options GetOptions(size_t nCacheSize) // on corruption in later versions. options.paranoid_checks = true; } + options.max_file_size = std::max(options.max_file_size, DBWRAPPER_MAX_FILE_SIZE); SetMaxOpenFiles(&options); return options; } diff --git a/src/dbwrapper.h b/src/dbwrapper.h index 63c2f99d2a..dd5daa7a1f 100644 --- a/src/dbwrapper.h +++ b/src/dbwrapper.h @@ -22,6 +22,7 @@ static const size_t DBWRAPPER_PREALLOC_KEY_SIZE = 64; static const size_t DBWRAPPER_PREALLOC_VALUE_SIZE = 1024; +static const size_t DBWRAPPER_MAX_FILE_SIZE = 32 << 20; // 32 MiB //! User-controlled performance and debug options. struct DBOptions { |