Revision 8ea087ad16c853ab24af2f73a9b6a0c9ae04a6e8 authored by Zhichao Cao on 06 November 2019, 20:50:33 UTC, committed by Facebook Github Bot on 06 November 2019, 21:02:20 UTC
Summary:
In the previous PR https://github.com/facebook/rocksdb/issues/4788, user can use db_bench mix_graph option to generate the workload that is from the social graph. The key is generated based on the key access hotness. In this PR, user can further model the key-range hotness and fit those to two-term-exponential distribution. First, user cuts the whole key space into small key ranges (e.g., key-ranges are the same size and the key-range number is the number of SST files). Then, user calculates the average access count per key of each key-range as the key-range hotness. Next, user fits the key-range hotness to two-term-exponential distribution (f(x) = f(x) = a*exp(b*x) + c*exp(d*x)) and generate the value of a, b, c, and d. They are the parameters in db_bench: prefix_dist_a, prefix_dist_b, prefix_dist_c, and prefix_dist_d. Finally, user can run db_bench by specify the parameters.
For example:
`./db_bench --benchmarks="mixgraph" -use_direct_io_for_flush_and_compaction=true -use_direct_reads=true -cache_size=268435456 -key_dist_a=0.002312 -key_dist_b=0.3467 -keyrange_dist_a=14.18 -keyrange_dist_b=-2.917 -keyrange_dist_c=0.0164 -keyrange_dist_d=-0.08082 -keyrange_num=30 -value_k=0.2615 -value_sigma=25.45 -iter_k=2.517 -iter_sigma=14.236 -mix_get_ratio=0.85 -mix_put_ratio=0.14 -mix_seek_ratio=0.01 -sine_mix_rate_interval_milliseconds=5000 -sine_a=350 -sine_b=0.0105 -sine_d=50000 --perf_level=2 -reads=1000000 -num=5000000 -key_size=48`
Pull Request resolved: https://github.com/facebook/rocksdb/pull/5953

Test Plan: run db_bench with different parameters and checked the results.

Differential Revision: D18053527

Pulled By: zhichao-cao

fbshipit-source-id: 171f8b3142bd76462f1967c58345ad7e4f84bab7
1 parent 5080465
History
File Mode Size
buckifier
build_tools
cache
cmake
coverage
db
docs
env
examples
file
hdfs
include
java
logging
memory
memtable
monitoring
options
port
table
test_util
third-party
tools
trace_replay
util
utilities
.clang-format -rw-r--r-- 138 bytes
.gitignore -rw-r--r-- 875 bytes
.lgtm.yml -rw-r--r-- 67 bytes
.travis.yml -rw-r--r-- 3.5 KB
.watchmanconfig -rw-r--r-- 130 bytes
AUTHORS -rw-r--r-- 322 bytes
CMakeLists.txt -rw-r--r-- 38.7 KB
CODE_OF_CONDUCT.md -rw-r--r-- 3.3 KB
CONTRIBUTING.md -rw-r--r-- 706 bytes
COPYING -rw-r--r-- 17.7 KB
DEFAULT_OPTIONS_HISTORY.md -rw-r--r-- 1.5 KB
DUMP_FORMAT.md -rw-r--r-- 763 bytes
HISTORY.md -rw-r--r-- 93.8 KB
INSTALL.md -rw-r--r-- 7.5 KB
LANGUAGE-BINDINGS.md -rw-r--r-- 1.1 KB
LICENSE.Apache -rw-r--r-- 11.1 KB
LICENSE.leveldb -rw-r--r-- 1.5 KB
Makefile -rw-r--r-- 69.8 KB
README.md -rw-r--r-- 1.8 KB
ROCKSDB_LITE.md -rw-r--r-- 1.0 KB
TARGETS -rw-r--r-- 34.4 KB
USERS.md -rw-r--r-- 6.5 KB
Vagrantfile -rw-r--r-- 1017 bytes
WINDOWS_PORT.md -rw-r--r-- 12.5 KB
appveyor.yml -rw-r--r-- 2.7 KB
defs.bzl -rw-r--r-- 1.4 KB
issue_template.md -rw-r--r-- 243 bytes
src.mk -rw-r--r-- 32.8 KB
thirdparty.inc -rw-r--r-- 7.8 KB

README.md

back to top