swh:1:snp:5115096b921df712aeb2a08114fede57fb3331fb
Revision 778e179046f0996232768bff2862ec1df7e40207 authored by Haobo Xu on 04 June 2013, 22:51:50 UTC, committed by Haobo Xu on 12 June 2013, 19:53:59 UTC
Summary: During compaction, we sync the output files after they are fully written out. This causes unnecessary blocking of the compaction thread and burstiness of the write traffic. This diff simply asks the OS to sync data incrementally as they are written, on the background. The hope is that, at the final sync, most of the data are already on disk and we would block less on the sync call. Thus, each compaction runs faster and we could use fewer number of compaction threads to saturate IO. In addition, the write traffic will be smoothed out, hopefully reducing the IO P99 latency too. Some quick tests show 10~20% improvement in per thread compaction throughput. Combined with posix advice on compaction read, just 5 threads are enough to almost saturate the udb flash bandwidth for 800 bytes write only benchmark. What's more promising is that, with saturated IO, iostat shows average wait time is actually smoother and much smaller. For the write only test 800bytes test: Before the change: await occillate between 10ms and 3ms After the change: await ranges 1-3ms Will test against read-modify-write workload too, see if high read latency P99 could be resolved. Will introduce a parameter to control the sync interval in a follow up diff after cleaning up EnvOptions. Test Plan: make check; db_bench; db_stress Reviewers: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D11115
1 parent 4985a9f
Tip revision: 19076c95aa2bcee55c26fcf0960cc844ad86ee9c authored by Levi Tamasi on 21 January 2021, 22:18:25 UTC
Update HISTORY.md for PR 7888 (#7890)
Update HISTORY.md for PR 7888 (#7890)
Tip revision: 19076c9
File | Mode | Size |
---|---|---|
VALGRIND_LOGS | ||
db | ||
doc | ||
hdfs | ||
helpers | ||
include | ||
java | ||
linters | ||
port | ||
scribe | ||
snappy | ||
table | ||
thrift | ||
tools | ||
util | ||
utilities | ||
.arcconfig | -rw-r--r-- | 243 bytes |
.gitignore | -rw-r--r-- | 174 bytes |
LICENSE | -rw-r--r-- | 1.4 KB |
Makefile | -rw-r--r-- | 11.8 KB |
README | -rw-r--r-- | 2.2 KB |
README.fb | -rw-r--r-- | 98 bytes |
build_detect_platform | -rwxr-xr-x | 8.9 KB |
build_detect_version | -rwxr-xr-x | 1.2 KB |
build_java.sh | -rwxr-xr-x | 1.9 KB |
e | -rw-r--r-- | 51 bytes |
fbcode.clang31.sh | -rw-r--r-- | 3.1 KB |
fbcode.gcc471.sh | -rw-r--r-- | 2.9 KB |
regression_build_test.sh | -rwxr-xr-x | 2.3 KB |
valgrind_test.sh | -rwxr-xr-x | 428 bytes |
Computing file changes ...