Revision d3d20dcdca9dc79d893a03dfa611fb1055c28f96 authored by Sagar Vemuri on 11 April 2019, 02:24:25 UTC, committed by Facebook Github Bot on 11 April 2019, 02:31:18 UTC
Summary: Introducing Periodic Compactions. This feature allows all the files in a CF to be periodically compacted. It could help in catching any corruptions that could creep into the DB proactively as every file is constantly getting re-compacted. And also, of course, it helps to cleanup data older than certain threshold. - Introduced a new option `periodic_compaction_time` to control how long a file can live without being compacted in a CF. - This works across all levels. - The files are put in the same level after going through the compaction. (Related files in the same level are picked up as `ExpandInputstoCleanCut` is used). - Compaction filters, if any, are invoked as usual. - A new table property, `file_creation_time`, is introduced to implement this feature. This property is set to the time at which the SST file was created (and that time is given by the underlying Env/OS). This feature can be enabled on its own, or in conjunction with `ttl`. It is possible to set a different time threshold for the bottom level when used in conjunction with ttl. Since `ttl` works only on 0 to last but one levels, you could set `ttl` to, say, 1 day, and `periodic_compaction_time` to, say, 7 days. Since `ttl < periodic_compaction_time` all files in last but one levels keep getting picked up based on ttl, and almost never based on periodic_compaction_time. The files in the bottom level get picked up for compaction based on `periodic_compaction_time`. Pull Request resolved: https://github.com/facebook/rocksdb/pull/5166 Differential Revision: D14884441 Pulled By: sagar0 fbshipit-source-id: 408426cbacb409c06386a98632dcf90bfa1bda47
1 parent ef0fc1b
File | Mode | Size |
---|---|---|
advisor | ||
dump | ||
rdb | ||
CMakeLists.txt | -rw-r--r-- | 544 bytes |
Dockerfile | -rw-r--r-- | 81 bytes |
analyze_txn_stress_test.sh | -rwxr-xr-x | 3.7 KB |
auto_sanity_test.sh | -rwxr-xr-x | 2.7 KB |
benchmark.sh | -rwxr-xr-x | 17.0 KB |
benchmark_leveldb.sh | -rwxr-xr-x | 5.1 KB |
blob_dump.cc | -rw-r--r-- | 3.3 KB |
check_format_compatible.sh | -rwxr-xr-x | 6.5 KB |
db_bench.cc | -rw-r--r-- | 813 bytes |
db_bench_tool.cc | -rw-r--r-- | 220.4 KB |
db_bench_tool_test.cc | -rw-r--r-- | 9.6 KB |
db_crashtest.py | -rw-r--r-- | 14.3 KB |
db_repl_stress.cc | -rw-r--r-- | 4.5 KB |
db_sanity_test.cc | -rw-r--r-- | 8.4 KB |
db_stress.cc | -rw-r--r-- | 144.3 KB |
dbench_monitor | -rwxr-xr-x | 2.6 KB |
generate_random_db.sh | -rwxr-xr-x | 734 bytes |
ingest_external_sst.sh | -rwxr-xr-x | 331 bytes |
ldb.cc | -rw-r--r-- | 572 bytes |
ldb_cmd.cc | -rw-r--r-- | 104.7 KB |
ldb_cmd_impl.h | -rw-r--r-- | 15.6 KB |
ldb_cmd_test.cc | -rw-r--r-- | 3.9 KB |
ldb_test.py | -rw-r--r-- | 26.1 KB |
ldb_tool.cc | -rw-r--r-- | 4.7 KB |
pflag | -rwxr-xr-x | 4.0 KB |
reduce_levels_test.cc | -rw-r--r-- | 5.2 KB |
regression_test.sh | -rwxr-xr-x | 15.8 KB |
report_lite_binary_size.sh | -rwxr-xr-x | 1.2 KB |
rocksdb_dump_test.sh | -rwxr-xr-x | 364 bytes |
run_flash_bench.sh | -rwxr-xr-x | 13.5 KB |
run_leveldb.sh | -rwxr-xr-x | 6.2 KB |
sample-dump.dmp | -rw-r--r-- | 100 bytes |
sst_dump.cc | -rw-r--r-- | 581 bytes |
sst_dump_test.cc | -rw-r--r-- | 7.0 KB |
sst_dump_tool.cc | -rw-r--r-- | 23.3 KB |
sst_dump_tool_imp.h | -rw-r--r-- | 2.9 KB |
trace_analyzer.cc | -rw-r--r-- | 709 bytes |
trace_analyzer_test.cc | -rw-r--r-- | 27.5 KB |
trace_analyzer_tool.cc | -rw-r--r-- | 68.1 KB |
trace_analyzer_tool.h | -rw-r--r-- | 9.8 KB |
verify_random_db.sh | -rwxr-xr-x | 1.0 KB |
write_external_sst.sh | -rwxr-xr-x | 457 bytes |
write_stress.cc | -rw-r--r-- | 10.9 KB |
write_stress_runner.py | -rw-r--r-- | 2.3 KB |
![swh spinner](/static/img/swh-spinner.gif)
Computing file changes ...