| 92df749 | sdong | 31 October 2019, 22:06:10 UTC | An odd fix for GCC 7 | 31 October 2019, 22:09:55 UTC |
| 09040bf | sdong | 31 October 2019, 21:42:34 UTC | Disable warning as error | 31 October 2019, 22:09:49 UTC |
| 2591096 | sdong | 31 October 2019, 21:17:26 UTC | Add some include<functional> | 31 October 2019, 22:09:36 UTC |
| c381701 | sdong | 31 October 2019, 21:10:18 UTC | [FB Internal] Point to the latest tool chain. | 31 October 2019, 22:09:36 UTC |
| da2ae09 | sdong | 17 July 2017, 22:39:37 UTC | [FB Only] use gcc-5 | 17 July 2017, 22:50:03 UTC |
| 1f303c1 | sdong | 12 October 2016, 22:47:53 UTC | fb internal: Should also use GCC 4.8.1 for CentOS 7 | 13 October 2016, 20:38:23 UTC |
| a51fbf5 | Yueh-Hsuan Chiang | 10 July 2014, 20:35:04 UTC | Update HISTORY.md for release 3.3 Summary: Update HISTORY.md for release 3.3 Test Plan: n/a Reviewers: sdong, ljin, igor Reviewed By: igor Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19635 | 10 July 2014, 20:35:04 UTC |
| 3f53d27 | Igor Canadi | 10 July 2014, 20:29:40 UTC | Merge pull request #200 from rdallman/missing_cf C API: create missing cf's, cleanup | 10 July 2014, 20:29:40 UTC |
| 5ff6633 | Igor Canadi | 10 July 2014, 20:19:28 UTC | Fix mac compile | 10 July 2014, 20:19:47 UTC |
| 1fc71a4 | Reed Allman | 10 July 2014, 19:53:46 UTC | C API: create missing cf's, cleanup | 10 July 2014, 19:55:53 UTC |
| 105c1e0 | Tomislav Novak | 10 July 2014, 00:46:18 UTC | ForwardIterator::status() checks all child iterators Summary: Forward iterator only checked `status_` and `mutable_iter_->status()`, which is not sufficient. For example, when reading exclusively from cache (kBlockCacheTier), `mutable_iter_->status()` may return kOk (e.g. there's nothing in the memtable), but one of immutable iterators could be in kIncomplete. In this case, `ForwardIterator::status()` ought to return that status instead of kOk. This diff changes `status()` to also check `imm_iters_`, `l0_iters_`, and `level_iters_`. Test Plan: ROCKSDB_TESTS=TailingIteratorIncomplete ./db_test Reviewers: ljin, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D19581 | 10 July 2014, 19:43:12 UTC |
| 01700b6 | sdong | 10 July 2014, 18:37:53 UTC | Update master to version 3.3 Summary: As tittle Test Plan: no need Reviewers: igor, yhchiang, ljin Reviewed By: ljin Subscribers: haobo, dhruba, xjin, leveldb Differential Revision: https://reviews.facebook.net/D19629 | 10 July 2014, 18:59:35 UTC |
| 36de0e5 | sdong | 10 July 2014, 18:35:07 UTC | Add a function to return current perf level Summary: Add a function to return the perf level. It is to allow a wrapper of DB to increase the perf level and restore the original perf level after finishing the function call. Test Plan: Add a verification in db_test Reviewers: yhchiang, igor, ljin Reviewed By: ljin Subscribers: xjin, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D19551 | 10 July 2014, 18:35:48 UTC |
| 30c81e7 | Stanislau Hlebik | 10 July 2014, 18:32:04 UTC | Removing NewTotalOrderPlainTableFactory Summary: Seems like NewTotalOrderPlainTableFactory is useless and is semantically incorrect. Total order mode indicator is prefix_extractor == nullptr, but NewTotalOrderPlainTableFactory doesn't set it to be nullptr. That's why some tests in plain_table_db_tests is incorrect. Test Plan: make all check Reviewers: sdong Reviewed By: sdong Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19587 | 10 July 2014, 18:32:04 UTC |
| 536f4b3 | Igor Canadi | 10 July 2014, 16:52:46 UTC | Add missing DB functions to DocumentDB | 10 July 2014, 16:52:46 UTC |
| f0a8be2 | Igor Canadi | 10 July 2014, 16:31:42 UTC | JSON (Document) API sketch Summary: This is a rough sketch of our new document API. Would like to get some thoughts and comments about the high-level architecture and API. I didn't optimize for performance at all. Leaving some low-hanging fruit so that we can be happy when we fix them! :) Currently, bunch of features are not supported at all. Indexes can be only specified when creating database. There is no query planner whatsoever. This will all be added in due time. Test Plan: Added a simple unit test Reviewers: haobo, yhchiang, dhruba, sdong, ljin Reviewed By: ljin Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D18747 | 10 July 2014, 16:31:42 UTC |
| 222cf25 | Feng Zhu | 10 July 2014, 06:40:03 UTC | change the init parameter for FileDescriptor Summary: fix a bug in improve_file_key_search, change the parameter for FileDescriptor Test Plan: make all check Reviewers: sdong Reviewed By: sdong Differential Revision: https://reviews.facebook.net/D19611 | 10 July 2014, 06:40:03 UTC |
| 8a7d1fe | Lei Jin | 10 July 2014, 05:46:15 UTC | disable rate limiter test Summary: The test is not stable because it relies on disk and only runs for a short period of time. So misisng a compaction/flush would greatly affect the rate. I am disabling it for now. What do you guys think? Test Plan: make Reviewers: yhchiang, igor, sdong Reviewed By: sdong Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19599 | 10 July 2014, 05:46:15 UTC |
| f697cad | Feng Zhu | 10 July 2014, 05:14:39 UTC | create compressed_levels_ in Version, allocate its space using arena. Make Version::Get, Version::FindFile faster Summary: Define CompressedFileMetaData that just contains fd, smallest_slice, largest_slice. Create compressed_levels_ in Version, the space is allocated using arena Thus increase the file meta data locality, speed up "Get" and "FindFile" benchmark with in-memory tmpfs, could have 4% improvement under "random read" and 2% improvement under "read while writing" benchmark command: ./db_bench --db=/mnt/db/rocksdb --num_levels=6 --key_size=20 --prefix_size=20 --keys_per_prefix=0 --value_size=100 --block_size=4096 --cache_size=17179869184 --cache_numshardbits=6 --compression_type=none --compression_ratio=1 --min_level_to_compress=-1 --disable_seek_compaction=1 --hard_rate_limit=2 --write_buffer_size=134217728 --max_write_buffer_number=2 --level0_file_num_compaction_trigger=8 --target_file_size_base=33554432 --max_bytes_for_level_base=1073741824 --disable_wal=0 --sync=0 --disable_data_sync=1 --verify_checksum=1 --delete_obsolete_files_period_micros=314572800 --max_grandparent_overlap_factor=10 --max_background_compactions=4 --max_background_flushes=0 --level0_slowdown_writes_trigger=16 --level0_stop_writes_trigger=24 --statistics=0 --stats_per_interval=0 --stats_interval=1048576 --histogram=0 --use_plain_table=1 --open_files=-1 --mmap_read=1 --mmap_write=0 --memtablerep=prefix_hash --bloom_bits=10 --bloom_locality=1 --perf_level=0 --benchmarks=readwhilewriting,readwhilewriting,readwhilewriting --use_existing_db=1 --num=52428800 --threads=1 —writes_per_second=81920 Read Random: From 1.8363 ms/op, improve to 1.7587 ms/op. Read while writing: From 2.985 ms/op, improve to 2.924 ms/op. Test Plan: make all check Reviewers: ljin, haobo, yhchiang, sdong Reviewed By: sdong Subscribers: dhruba, igor Differential Revision: https://reviews.facebook.net/D19419 | 10 July 2014, 05:14:39 UTC |
| cae316d | Yueh-Hsuan Chiang | 09 July 2014, 20:54:11 UTC | Merge pull request #195 from ankgup87/master [Java] Package .so file with JNI jar | 09 July 2014, 20:54:11 UTC |
| 7082855 | Yueh-Hsuan Chiang | 09 July 2014, 19:46:08 UTC | Some fixes on size compensation logic for deletion entry in compaction Summary: This patch include two fixes: 1. newly created Version will now takes the aggregated stats for average-value-size from the latest Version. 2. compensated size of a file is now computed only for newly created / loaded file, this addresses the issue where files are already sorted by their compensated file size but might sometimes observe some out-of-order due to later update on compensated file size. Test Plan: export ROCKSDB_TESTS=CompactionDele ./db_test Reviewers: ljin, igor, sdong Reviewed By: sdong Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19557 | 09 July 2014, 19:46:08 UTC |
| 0a4d930 | Ankit Gupta | 09 July 2014, 16:09:08 UTC | Caching methodId and fieldId is fine: v2 | 09 July 2014, 16:09:08 UTC |
| b6caaea | Ankit Gupta | 09 July 2014, 16:06:40 UTC | Caching methodId and fieldId is fine | 09 July 2014, 16:06:40 UTC |
| 21e5226 | Ankit Gupta | 09 July 2014, 16:06:20 UTC | Revert wrong commit | 09 July 2014, 16:06:20 UTC |
| 05bd545 | Ankit Gupta | 09 July 2014, 16:02:05 UTC | Caching methodId and fieldId is fine | 09 July 2014, 16:02:05 UTC |
| ef1aad9 | Lei Jin | 08 July 2014, 22:29:13 UTC | fix one more internal_stats issue Summary: stall count is wrong Test Plan: make release Reviewers: sdong, yhchiang, igor Reviewed By: igor Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19539 | 08 July 2014, 22:29:13 UTC |
| 73d7147 | Lei Jin | 08 July 2014, 22:15:00 UTC | make rate limiter test more reliable Summary: Randomize keys so that compaction actually happens. Change the config so that compaction happens more aggressively. The test takes longer time, but the results are more stable shown by iostat Test Plan: ran it Reviewers: igor, yhchiang Reviewed By: yhchiang Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19533 | 08 July 2014, 22:15:00 UTC |
| 8a9cc78 | Lei Jin | 08 July 2014, 19:48:10 UTC | report correct interval amplification Summary: as title Test Plan: make release Reviewers: sdong, yhchiang, igor Reviewed By: igor Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19515 | 08 July 2014, 19:48:10 UTC |
| 534357c | Lei Jin | 08 July 2014, 19:31:49 UTC | integrate rate limiter into rocksdb Summary: Add option and plugin rate limiter for PosixWritableFile. The rate limiter only applies to flush and compaction. WAL and MANIFEST are excluded from this enforcement. Test Plan: db_test Reviewers: igor, yhchiang, sdong Reviewed By: sdong Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19425 | 08 July 2014, 19:31:49 UTC |
| 5ef1ba7 | Lei Jin | 08 July 2014, 18:41:57 UTC | generic rate limiter Summary: A generic rate limiter that can be shared by threads and rocksdb instances. Will use this to smooth out write traffic generated by compaction and flush. This will help us get better p99 behavior on flash storage. Test Plan: unit test output ==== Test RateLimiterTest.Rate request size [1 - 1023], limit 10 KB/sec, actual rate: 10.374969 KB/sec, elapsed 2002265 request size [1 - 2047], limit 20 KB/sec, actual rate: 20.771242 KB/sec, elapsed 2002139 request size [1 - 4095], limit 40 KB/sec, actual rate: 41.285299 KB/sec, elapsed 2202424 request size [1 - 8191], limit 80 KB/sec, actual rate: 81.371605 KB/sec, elapsed 2402558 request size [1 - 16383], limit 160 KB/sec, actual rate: 162.541268 KB/sec, elapsed 3303500 Reviewers: yhchiang, igor, sdong Reviewed By: sdong Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19359 | 08 July 2014, 18:41:57 UTC |
| b278ae8 | Lei Jin | 08 July 2014, 18:40:42 UTC | Apply fractional cascading in ForwardIterator::Seek() Summary: Use search hint to reduce FindFile range thus avoid comparison For a small DB with 50M keys, perf_context counter shows it reduces comparison from 2B to 1.3B for a 15-minute run. No perf change was observed for 1 seek thread, but quite good improvement was seen for 32 seek threads, when CPU was busy. will post detail results when ready Test Plan: db_bench and db_test Reviewers: haobo, sdong, dhruba, igor Reviewed By: igor Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D18879 | 08 July 2014, 18:40:42 UTC |
| 1b95bf7 | Igor Canadi | 08 July 2014, 18:40:11 UTC | Merge pull request #197 from rdallman/update-options C API: update options w/ convenience funcs & fifo compaction | 08 July 2014, 18:40:11 UTC |
| fd3fb4b | Reed Allman | 08 July 2014, 04:12:25 UTC | C API: update options w/ convenience funcs & fifo compaction | 08 July 2014, 17:57:45 UTC |
| db1a10e | Igor Canadi | 08 July 2014, 15:51:41 UTC | Merge pull request #198 from rdallman/cf-compact-range C API: bugfix column_family_compact_range | 08 July 2014, 15:51:41 UTC |
| 5491c48 | Igor Canadi | 08 July 2014, 15:49:49 UTC | Merge pull request #199 from jrobeson/patch-1 facebook accounts are not required for CLA signers | 08 July 2014, 15:49:49 UTC |
| 058421d | Johnny Robeson | 08 July 2014, 09:57:54 UTC | facebook accounts are not required for CLA signers slightly reword the sentence to avoid mentioning it. | 08 July 2014, 09:57:54 UTC |
| e9b18b6 | Reed Allman | 08 July 2014, 04:48:49 UTC | C API: bugfix column_family_comact_range | 08 July 2014, 04:48:49 UTC |
| 4216ca3 | Ankit Gupta | 08 July 2014, 04:45:02 UTC | Class IDs and method IDs should not be cached | 08 July 2014, 04:45:02 UTC |
| 637744c | Ankit Gupta | 08 July 2014, 03:50:21 UTC | Package .so file with JNI jar | 08 July 2014, 03:50:21 UTC |
| 01159aa | sdong | 07 July 2014, 22:50:09 UTC | stackable_db: add default function for GetLiveFilesMetaData() Summary: stackable_db doesn't have GetLiveFilesMetaData() implemented. Add it Test Plan: make all check Reviewers: yhchiang, ljin, igor Reviewed By: igor Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19491 | 08 July 2014, 00:02:57 UTC |
| 4adf64e | Igor Canadi | 07 July 2014, 21:54:11 UTC | Fix compile issue | 07 July 2014, 21:54:11 UTC |
| 8622277 | Igor Canadi | 07 July 2014, 21:48:48 UTC | Merge pull request #194 from ankgup87/master Add compression type to options | 07 July 2014, 21:48:48 UTC |
| 8a03935 | Igor Canadi | 07 July 2014, 21:41:54 UTC | Fix valgrind error in c_test Summary: External contribution caused some valgrind errors: https://github.com/facebook/rocksdb/commit/1a34aaaef0900785c2de7e55b55d8c48d1201300 This diff fixes them Test Plan: ran valgrind Reviewers: sdong, yhchiang, ljin Reviewed By: ljin Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19485 | 07 July 2014, 21:41:54 UTC |
| 8f6e9ab | Igor Canadi | 07 July 2014, 20:42:46 UTC | Merge pull request #191 from edsrzf/c-api-compaction-filter-factory C API: Add support for compaction filter factories (v1) | 07 July 2014, 20:42:46 UTC |
| 13a130c | Evan Shaw | 04 July 2014, 22:59:47 UTC | C API: Add test for compaction filter factories Also refactored the compaction filter tests to share some code and ensure that options were getting reset so future test results aren't confused. | 07 July 2014, 20:12:36 UTC |
| 3f7104d | Evan Shaw | 04 July 2014, 04:04:55 UTC | C API: Allow setting compaction filter factory | 07 July 2014, 20:12:36 UTC |
| 91bede7 | Evan Shaw | 03 July 2014, 23:52:22 UTC | C API: Add support for compaction filter factories (v1) | 07 July 2014, 20:12:36 UTC |
| 0bf5589 | Evan Shaw | 03 July 2014, 22:46:55 UTC | Fix compaction_filter.h typos | 07 July 2014, 20:11:01 UTC |
| 97bfcd6 | Ankit Gupta | 07 July 2014, 19:26:06 UTC | Update doc of options.cc | 07 July 2014, 19:26:06 UTC |
| c9cfb88 | Ankit Gupta | 07 July 2014, 19:21:15 UTC | Remove doc from options class | 07 July 2014, 19:21:15 UTC |
| 4efca96 | Ankit Gupta | 07 July 2014, 19:18:55 UTC | Change enum type from int to byte | 07 July 2014, 19:18:55 UTC |
| f0660d5 | Radheshyam Balasundaram | 07 July 2014, 17:53:31 UTC | Adding NUMA support to db_bench tests Summary: Changes: - Adding numa_aware flag to db_bench.cc - Using numa.h library to bind memory and cpu of threads to a fixed NUMA node Result: There seems to be no significant change in the micros/op time with numa_aware enabled. I also tried this with other implementations, including a combination of pthread_setaffinity_np, sched_setaffinity and set_mempolicy methods. It'd be great if someone could point out where I'm going wrong and if we can achieve a better micors/op. Test Plan: Ran db_bench tests using following command: ./db_bench --db=/mnt/tmp --num_levels=6 --key_size=20 --prefix_size=20 --keys_per_prefix=0 --value_size=100 --block_size=4096 --cache_size=17179869184 --cache_numshardbits=6 --compression_type=none --compression_ratio=1 --min_level_to_compress=-1 --disable_seek_compaction=1 --hard_rate_limit=2 --write_buffer_size=134217728 --max_write_buffer_number=2 --level0_file_num_compaction_trigger=8 --target_file_size_base=134217728 --max_bytes_for_level_base=1073741824 --disable_wal=0 --wal_dir=/mnt/tmp --sync=0 --disable_data_sync=1 --verify_checksum=1 --delete_obsolete_files_period_micros=314572800 --max_grandparent_overlap_factor=10 --max_background_compactions=4 --max_background_flushes=0 --level0_slowdown_writes_trigger=16 --level0_stop_writes_trigger=24 --statistics=0 --stats_per_interval=0 --stats_interval=1048576 --histogram=0 --use_plain_table=1 --open_files=-1 --mmap_read=1 --mmap_write=0 --memtablerep=prefix_hash --bloom_bits=10 --bloom_locality=1 --perf_level=0 --duration=300 --benchmarks=readwhilewriting --use_existing_db=1 --num=157286400 --threads=24 --writes_per_second=10240 --numa_aware=[False/True] The tests were run in private devserver with 24 cores and the db was prepopulated using filluniquerandom test. The tests resulted in 0.145 us/op with numa_aware=False and 0.161 us/op with numa_aware=True. Reviewers: sdong, yhchiang, ljin, igor Reviewed By: ljin, igor Subscribers: igor, leveldb Differential Revision: https://reviews.facebook.net/D19353 | 07 July 2014, 17:53:31 UTC |
| e8c592c | Ankit Gupta | 07 July 2014, 17:25:27 UTC | Add newline | 07 July 2014, 17:25:27 UTC |
| 0bc5fa9 | Igor Canadi | 07 July 2014, 17:17:43 UTC | Merge pull request #190 from edsrzf/c-api-writebatch-serialized C API: support constructing write batch from serialized representation | 07 July 2014, 17:17:43 UTC |
| 47f0cf6 | Ankit Gupta | 07 July 2014, 17:06:28 UTC | Add docs | 07 July 2014, 17:06:28 UTC |
| da12f9e | Ankit Gupta | 07 July 2014, 17:01:28 UTC | Remove .so package with JNI | 07 July 2014, 17:01:28 UTC |
| 54d7a2c | Ankit Gupta | 07 July 2014, 16:58:54 UTC | Add compression type to options | 07 July 2014, 16:58:54 UTC |
| efbb4bd | Igor Canadi | 07 July 2014, 16:47:30 UTC | Merge pull request #193 from rdallman/columnfamilies C API: column family support | 07 July 2014, 16:47:30 UTC |
| bc708e0 | Ankit Gupta | 07 July 2014, 16:14:32 UTC | Merge branch 'master' of https://github.com/facebook/rocksdb | 07 July 2014, 16:14:32 UTC |
| 1a34aaa | Reed Allman | 07 July 2014, 08:18:52 UTC | C API: column family support | 07 July 2014, 08:41:01 UTC |
| 9fc23d0 | Evan Shaw | 05 July 2014, 22:35:13 UTC | C API: support constructing write batch from serialized representation | 05 July 2014, 22:36:33 UTC |
| 7b85c1e | Yueh-Hsuan Chiang | 04 July 2014, 07:02:12 UTC | Improve SimpleWriteTimeoutTest to avoid false alarm. Summary: SimpleWriteTimeoutTest has two parts: 1) insert two large key/values to make memtable full and expect both of them are successful; 2) insert another key / value and expect it to be timed-out. Previously we also set a timeout in the first step, but this might sometimes cause false alarm. This diff makes the first two writes run without timeout setting. Test Plan: export ROCKSDB_TESTS=Time make db_test Reviewers: sdong, ljin Reviewed By: ljin Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19461 | 04 July 2014, 07:02:12 UTC |
| 8d9a46f | Lei Jin | 04 July 2014, 06:13:08 UTC | initialize decoded_internal_key_valid Summary: ReadInternalKey() will assign correct value anyway. Initialize it to true to suppress compiler error reported https://github.com/facebook/rocksdb/issues/186 Test Plan: I cannot reproduce it but this is obvious Reviewers: sdong, yhchiang Reviewed By: yhchiang Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19467 | 04 July 2014, 06:13:08 UTC |
| d33657a | Yueh-Hsuan Chiang | 04 July 2014, 00:19:17 UTC | Fixed a warning in release mode. Summary: Removed a variable that is only used in assertion check. Test Plan: make release Reviewers: ljin, sdong Reviewed By: sdong Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19455 | 04 July 2014, 00:19:17 UTC |
| 90a6aca | Yueh-Hsuan Chiang | 03 July 2014, 23:28:03 UTC | Finer report I/O stats about Flush and Compaction. Summary: This diff allows the I/O stats about Flush and Compaction to be reported in a more accurate way. Instead of measuring the size of a file, it measure I/O cost in per read / write basis. Test Plan: make all check Reviewers: sdong, igor, ljin Reviewed By: ljin Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19383 | 03 July 2014, 23:28:03 UTC |
| a1df6c1 | Yueh-Hsuan Chiang | 03 July 2014, 22:56:03 UTC | Update HISTORY.md to include TimeOut write API and compaction update. Summary: Update HISTORY.md to include TimeOut write API and compaction update. Test Plan: n/a Reviewers: ljin, sdong Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19449 | 03 July 2014, 22:58:27 UTC |
| d4d338d | Yueh-Hsuan Chiang | 03 July 2014, 22:47:02 UTC | Add timeout_hint_us to WriteOptions and introduce Status::TimeOut. Summary: This diff adds timeout_hint_us to WriteOptions. If it's non-zero, then 1) writes associated with this options MAY be aborted when it has been waiting for longer than the specified time. If an abortion happens, associated writes will return Status::TimeOut. 2) the stall time of the associated write caused by flush or compaction will be limited by timeout_hint_us. The default value of timeout_hint_us is 0 (i.e., OFF.) The statistics of timeout writes will be recorded in WRITE_TIMEDOUT. Test Plan: export ROCKSDB_TESTS=WriteTimeoutAndDelayTest make db_test ./db_test Reviewers: igor, ljin, haobo, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D18837 | 03 July 2014, 22:47:02 UTC |
| 6b835c6 | Ankit Gupta | 03 July 2014, 21:13:46 UTC | Merge branch 'master' of https://github.com/facebook/rocksdb | 03 July 2014, 21:13:46 UTC |
| 70aa243 | Ankit Gupta | 03 July 2014, 21:13:05 UTC | Package .so file with .jar | 03 July 2014, 21:13:05 UTC |
| 4203431 | Igor Canadi | 03 July 2014, 21:03:24 UTC | Fix mac os compile error | 03 July 2014, 21:03:24 UTC |
| 6580685 | Yueh-Hsuan Chiang | 03 July 2014, 17:22:08 UTC | Add TimedWait() API to CondVar. Summary: Add TimedWait() API to CondVar, which will be used in the future to support TimedOut Write API and Rate limiter. Test Plan: make db_test -j32 Reviewers: sdong, ljin Reviewed By: ljin Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19431 | 03 July 2014, 17:22:08 UTC |
| 2459f7e | sdong | 02 July 2014, 16:54:20 UTC | Support Multiple DB paths (without having an interface to expose to users) Summary: In this patch, we allow RocksDB to support multiple DB paths internally. No user interface is supported yet so this patch is silent to users. Test Plan: make all check Reviewers: igor, haobo, ljin, yhchiang Reviewed By: yhchiang Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D18921 | 03 July 2014, 04:14:44 UTC |
| f146cab | Igor Canadi | 02 July 2014, 18:40:57 UTC | Centralize compression decision to compaction picker Summary: Before this diff, we're deciding enable_compression in CompactionPicker and then we're deciding final compression type in DBImpl. This is kind of confusing. After the diff, the final compression type will be decided in CompactionPicker. The reason for this is that I want CompactFiles() to specify output compression type, so that people can mix and match compression styles in their compaction algorithms. This diff makes it much easier to do that. Test Plan: make check Reviewers: dhruba, haobo, sdong, yhchiang, ljin Reviewed By: ljin Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19137 | 02 July 2014, 18:40:57 UTC |
| d3f63f0 | Igor Canadi | 02 July 2014, 09:40:16 UTC | Fix 32-bit errors Summary: https://www.facebook.com/groups/rocksdb.dev/permalink/590438347721350/ Test Plan: compiles Reviewers: sdong, ljin, yhchiang Reviewed By: yhchiang Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19197 | 02 July 2014, 09:40:16 UTC |
| 1d05006 | sdong | 02 July 2014, 01:54:50 UTC | Re-commit the correct part (WalDir) of the revision: Commit 6634844dba962b9a150646382f4d6531d1f2440b by sdong Two small fixes in db_test Summary: Two fixes: (1) WalDir to pick a directory under TmpDir to allow two tests running in parallel without impacting each other (2) kBlockBasedTableWithWholeKeyHashIndex is disabled by mistake (I assume). Enable it. Test Plan: ./db_test Reviewers: yhchiang, ljin Reviewed By: ljin Subscribers: nkg-, igor, dhruba, haobo, leveldb Differential Revision: https://reviews.facebook.net/D19389 | 02 July 2014, 01:54:50 UTC |
| 30b2060 | sdong | 02 July 2014, 00:41:38 UTC | Revert "Two small fixes in db_test" This reverts commit 6634844dba962b9a150646382f4d6531d1f2440b. | 02 July 2014, 00:41:38 UTC |
| 9c332aa | sdong | 01 July 2014, 18:05:05 UTC | HashLinkList memtable switches a bucket to a skip list to reduce performance outliers Summary: In this patch, we enhance HashLinkList memtable to reduce performance outliers when a bucket contains too many entries. We switch to skip list for this case to enable binary search. Add threshold_use_skiplist parameter to determine when a bucket needs to switch to skip list. The new data structure is documented in comments in the codes. Test Plan: make all check set threshold_use_skiplist in several tests Reviewers: yhchiang, haobo, ljin Reviewed By: yhchiang, ljin Subscribers: nkg-, xjin, dhruba, yhchiang, leveldb Differential Revision: https://reviews.facebook.net/D19299 | 02 July 2014, 00:14:15 UTC |
| 6634844 | sdong | 01 July 2014, 23:27:00 UTC | Two small fixes in db_test Summary: Two fixes: (1) WalDir to pick a directory under TmpDir to allow two tests running in parallel without impacting each other (2) kBlockBasedTableWithWholeKeyHashIndex is disabled by mistake (I assume). Enable it. Test Plan: ./db_test Reviewers: yhchiang, ljin Reviewed By: ljin Subscribers: nkg-, igor, dhruba, haobo, leveldb Differential Revision: https://reviews.facebook.net/D19389 | 01 July 2014, 23:58:03 UTC |
| c4018e7 | Feng Zhu | 01 July 2014, 18:02:42 UTC | In tools/db_stress.cc, set proper value in NewHashSkipListRepFactory's bucket_size Summary: Now that the arena is used to allocate space for hashskiplist's bucket. The bucket size need to be set small enough to avoid "should_flush_" failure in memtable's assertion. Test Plan: make all check Reviewers: sdong Reviewed By: sdong Subscribers: igor Differential Revision: https://reviews.facebook.net/D19371 | 01 July 2014, 18:02:42 UTC |
| f5d4df1 | Igor Canadi | 01 July 2014, 08:55:03 UTC | Fix compile error | 01 July 2014, 08:55:03 UTC |
| a2e0d89 | Igor Canadi | 01 July 2014, 06:55:04 UTC | No need for files_by_size_ in universal compaction Summary: files_by_size_ is sorted by time in case of universal compaction. However, Version::files_ is also sorted by time. So no need for files_by_size_ Test Plan: 1) make check with the change 2) make check with `assert(last_index == c->input_version_->files_[level].size() - 1);` in compaction picker Reviewers: dhruba, haobo, yhchiang, sdong, ljin Reviewed By: ljin Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19125 | 01 July 2014, 06:55:04 UTC |
| 5656367 | Feng Zhu | 30 June 2014, 22:54:31 UTC | use arena to allocate memtable's bloomfilter and hashskiplist's buckets_ Summary: Bloomfilter and hashskiplist's buckets_ allocated by memtable's arena DynamicBloom: pass arena via constructor, allocate space in SetTotalBits HashSkipListRep: allocate space of buckets_ using arena. do not delete it in deconstructor because arena would take care of it. Several test files are changed. Test Plan: make all check Reviewers: ljin, haobo, yhchiang, sdong Reviewed By: sdong Subscribers: igor, dhruba Differential Revision: https://reviews.facebook.net/D19335 | 30 June 2014, 22:54:31 UTC |
| f799c8b | Yueh-Hsuan Chiang | 29 June 2014, 19:35:47 UTC | [Java] Correct the library loading for zlib in RocksJava. Summary: Correct the library loading for zlib in RocksJava: zlib should be loaded by loadLibrary("z") instead of loadLibrary("zlib"). Test Plan: make rocksdbjava cd java make db_bench ./jdb_bench.sh --compression_type=zlib Reviewers: sdong, ljin, ankgup87 Reviewed By: ankgup87 Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19341 | 29 June 2014, 19:35:47 UTC |
| 918a355 | Yueh-Hsuan Chiang | 29 June 2014, 17:29:26 UTC | Merge pull request #183 from ankgup87/master [Java] Merge pull request #183 --- Add statistics collector utility Summary: 1. Add statistics collector which collects statistics periodically at a period specified by caller. 2. Added unit-test for StatsCollector make rocksdbjava make test Reviewers: haobo, yhchiang, sdong, dhruba, rsumbaly, zzbennett, swapnilghike Reviewed By: yhchiang CC: leveldb Differential Revision: https://reviews.facebook.net/19215 | 29 June 2014, 17:29:26 UTC |
| b2625a4 | Ankit Gupta | 29 June 2014, 14:52:27 UTC | Add java docs for StatsCollector constructor and StatsCollectorCallback thread safety | 29 June 2014, 14:52:27 UTC |
| abf9920 | Ankit Gupta | 28 June 2014, 18:22:33 UTC | Merge branch 'master' of https://github.com/facebook/rocksdb | 28 June 2014, 18:22:33 UTC |
| dd337bc | sdong | 27 June 2014, 23:01:59 UTC | In logging format, use PRIu64 instead of casting Summary: Code cleaning up, since we are already using __STDC_FORMAT_MACROS in printing uint64_t, change other places. Only logging is changed. Test Plan: make all check Reviewers: ljin Reviewed By: ljin Subscribers: dhruba, yhchiang, haobo, leveldb Differential Revision: https://reviews.facebook.net/D19113 | 27 June 2014, 23:34:15 UTC |
| a359486 | Stanislau Hlebik | 26 June 2014, 23:45:27 UTC | Cache some conditions for DBImpl::MakeRoomForWrite Summary: Task 4580155. Some conditions in DBImpl::MakeRoomForWrite can be cached in ColumnFamilyData, because theirs value can be changed only during compaction, adding new memtable and/or add recalculation of compaction score. These conditions are: cfd->imm()->size() == cfd->options()->max_write_buffer_number - 1 cfd->current()->NumLevelFiles(0) >= cfd->options()->level0_stop_writes_trigger cfd->options()->soft_rate_limit > 0.0 && (score = cfd->current()->MaxCompactionScore()) > cfd->options()->soft_rate_limit cfd->options()->hard_rate_limit > 1.0 && (score = cfd->current()->MaxCompactionScore()) > cfd->options()->hard_rate_limit P.S. As it's my first diff, Siying suggested to add everybody as a reviewers for this diff. Sorry, if I forgot someone or add someone by mistake. Test Plan: make all check Reviewers: haobo, xjin, dhruba, yhchiang, zagfox, ljin, sdong Reviewed By: sdong Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19311 | 26 June 2014, 23:45:27 UTC |
| 81c5d98 | Yueh-Hsuan Chiang | 25 June 2014, 21:31:30 UTC | Fixed a comparison between signed and unsigned integers in options.cc Summary: Fixed the following warning: util/options.cc: In constructor ‘rocksdb::ColumnFamilyOptions::ColumnFamilyOptions(const rocksdb::Options&)’: util/options.cc:157:58: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare] if (max_bytes_for_level_multiplier_additional.size() < num_levels) { ^ Test Plan: make all check Reviewers: haobo, sdong, ljin Reviewed By: ljin Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19293 | 25 June 2014, 21:31:30 UTC |
| 19de6a7 | sdong | 25 June 2014, 00:52:30 UTC | Remove MemTableRep::GetIterator(const Slice& slice) Summary: It seems to me that when ever function MemTableRep::GetIterator(const Slice& slice) is used, we can use MemTableRep::GetDynamicPrefixIterator() instead. Just delete it to simplify the codes. Test Plan: make all check Reviewers: yhchiang, ljin Reviewed By: ljin Subscribers: xjin, dhruba, haobo, leveldb Differential Revision: https://reviews.facebook.net/D19281 | 25 June 2014, 21:09:29 UTC |
| 55531fd | Yueh-Hsuan Chiang | 25 June 2014, 20:11:12 UTC | Fixed heap-buffer-overflow issue when Options.num_levels > 7. Summary: Currently, when num_levels has been changed to > 7, internally it will not resize max_bytes_for_level_multiplier_additional. As a result, max_bytes_for_level_multiplier_additional.size() will be smaller than num_levels, which causes heap-buffer-overflow. Test Plan: make all check Reviewers: haobo, sdong, ljin Reviewed By: ljin Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19275 | 25 June 2014, 20:11:12 UTC |
| 8898a0a | Yueh-Hsuan Chiang | 25 June 2014, 01:22:11 UTC | Reorder the member variables of FileMetaData to improve cache locality. Summary: Move stats related member variables of FileMetaData to the bottom to improve cache locality of normal DB operations. Test Plan: make Reviewers: haobo, ljin, sdong Reviewed By: sdong Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19287 | 25 June 2014, 01:22:11 UTC |
| e813f5b | Yueh-Hsuan Chiang | 24 June 2014, 22:37:06 UTC | Allow compaction to reclaim storage more effectively. Summary: This diff allows compaction to reclaim storage more effectively. In the current design, compactions are mainly triggered based on the file sizes. However, since deletion entries does not have value, files which have many deletion entries are less likely to be compacted. As a result, it may took a while to make deletion entries to be compacted. This diff address issue by compensating the size of deletion entries during compaction process: the size of each deletion entry in the compaction process is augmented by 2x average value size. The diff applies to both leveled and universal compacitons. Test Plan: develop CompactionDeletionTrigger make db_test ./db_test Reviewers: haobo, igor, ljin, sdong Reviewed By: sdong Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19029 | 24 June 2014, 22:37:06 UTC |
| faa8d21 | Yueh-Hsuan Chiang | 24 June 2014, 21:29:28 UTC | Improve an assertion in RandomGenerator::Generate() in db_bench. Summary: RandomGenerator::Generate() currently has an assertion len < data_.size(). However, it is actually fine to have len == data_.size(). This diff change the assertion to len <= data_.size(). Test Plan: make db_bench ./db_bench Reviewers: haobo, sdong, ljin Reviewed By: ljin Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19269 | 24 June 2014, 21:29:28 UTC |
| 85f9bb4 | Yueh-Hsuan Chiang | 24 June 2014, 21:12:36 UTC | [Java] Enable compression_ratio option in DbBenchmark.java Summary: Enable the random values in Java DB Bench to be generated based on the compression_ratio specified in the command-line arguments. Test Plan: make rocksdbjava java/jdb_bench.sh Reviewers: sdong, ankgup87, haobo Reviewed By: haobo Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19101 | 24 June 2014, 21:12:36 UTC |
| e7c9412 | Yueh-Hsuan Chiang | 24 June 2014, 05:55:24 UTC | Merge pull request #181 from nanwu/master Corrected the output message in mac-install-gflags.sh | 24 June 2014, 05:55:24 UTC |
| fb54eef | nawu | 24 June 2014, 05:14:02 UTC | escaped the special characters and added a dot | 24 June 2014, 05:14:02 UTC |
| c9ad282 | nawu | 24 June 2014, 05:05:03 UTC | added a dot | 24 June 2014, 05:05:03 UTC |
| 82c3179 | Yueh-Hsuan Chiang | 24 June 2014, 00:51:00 UTC | Revert the default setting of InitFromCmdLineArgs(). Summary: Revert the default setting of InitFromCmdLineArgs() as all the callers currently provide full set of arguments. Test Plan: make reduce_levels_test ./reduce_levels_test Reviewers: haobo, ljin Reviewed By: ljin Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19257 | 24 June 2014, 00:51:00 UTC |
| bf4b152 | Yueh-Hsuan Chiang | 23 June 2014, 23:48:20 UTC | Fix compile error in reduce_levels_test. Summary: Fixed the following compile error. tools/reduce_levels_test.cc:89:31: error: no matching function for call to 'InitFromCmdLineArgs' LDBCommand* level_reducer = LDBCommand::InitFromCmdLineArgs(args); ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ./util/ldb_cmd.h:56:22: note: candidate function not viable: requires 3 arguments, but 1 was provided static LDBCommand* InitFromCmdLineArgs( ^ ./util/ldb_cmd.h:62:22: note: candidate function not viable: requires 4 arguments, but 1 was provided static LDBCommand* InitFromCmdLineArgs( ^ 1 error generated. Test Plan: make reduce_levels_test ./reduce_levels_test Reviewers: haobo, sdong, ljin Reviewed By: ljin Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19251 | 23 June 2014, 23:48:20 UTC |
