| c464aa2 | 赵星宇 | 24 April 2017, 13:52:11 UTC | fix log err | 24 April 2017, 13:52:11 UTC |
| 04abb2b | Willem Jan Withagen | 09 April 2017, 18:15:35 UTC | FreeBSD only requires WITH_JEMALLOC, not the rest Summary: The compiler error: ``` /home/jenkins/workspace/ceph-master/src/rocksdb/db/db_impl.cc:20:10: fatal error: 'jemalloc/jemalloc.h' file not found ^ 1 error generated. ``` But is does compile with the `WITH_JEMALLOC` set. So ignore all the other settings. Closes https://github.com/facebook/rocksdb/pull/2118 Differential Revision: D4858387 Pulled By: yiwu-arbug fbshipit-source-id: 05b982969dcab53669a73a903641e71641c714e7 | 09 April 2017, 18:26:50 UTC |
| 6173018 | Islam AbdelRahman | 07 April 2017, 23:58:07 UTC | dummy diff Summary: Closes https://github.com/facebook/rocksdb/pull/2114 Differential Revision: D4854860 Pulled By: IslamAbdelRahman fbshipit-source-id: b871c5b9ccc52d20f5ceacdd172dc70b1dbf9110 | 08 April 2017, 00:07:37 UTC |
| 360e996 | Islam AbdelRahman | 07 April 2017, 21:58:58 UTC | Summary: Remove .arcconfig fbshipit-source-id: fe9f000138776a14c507b24c952ecf50b48d71a9 | 07 April 2017, 21:58:58 UTC |
| 69a5e64 | Islam AbdelRahman | 07 April 2017, 21:40:30 UTC | Deprecate .arcconfig fbshipit-source-id: 2e88ce2a16d16620aa6ef36aabbf3e91be2fc779 | 07 April 2017, 21:40:30 UTC |
| 9690653 | Maysam Yabandeh | 07 April 2017, 18:25:42 UTC | Add a verify phase to benchmarks Summary: Check the result of the benchmark againt a specified truth_db, which is expected to be produced using the same benchmark but perhaps on a different commit or with different configs. The verification is simple and assumes that key/values are generated deterministically. This assumption would break if db_bench using rand variable differently from the benchmark that produced truth_db. Currently it is checked to work on fillrandom and readwhilewriting. A param finish_after_writes is added to ensure that the background writing thread will write the same number of entries between two benchmarks. Example: $ TEST_TMPDIR=/dev/shm/truth_db ./db_bench --benchmarks="fillrandom,readwhilewriting" --num=200000 --finish_after_writes=true $ TEST_TMPDIR=/dev/shm/tmpdb ./db_bench --benchmarks="fillrandom,readwhilewriting,verify" --truth_db /dev/shm/truth_db/dbbench --num=200000 --finish_after_writes=true Verifying db <= truth_db... Verifying db >= truth_db... ...Verified Closes https://github.com/facebook/rocksdb/pull/2098 Differential Revision: D4839233 Pulled By: maysamyabandeh fbshipit-source-id: 2f4ed31 | 07 April 2017, 18:39:12 UTC |
| dd8f9e3 | Ayappan | 07 April 2017, 17:41:38 UTC | Fix compilation for GCC-5 Summary: Fixes this issue https://github.com/facebook/rocksdb/issues/2108 Closes https://github.com/facebook/rocksdb/pull/2109 Differential Revision: D4851965 Pulled By: yiwu-arbug fbshipit-source-id: 6ee807b | 07 April 2017, 17:54:12 UTC |
| c2954f9 | Volker Mische | 07 April 2017, 17:36:09 UTC | Add Travis job to build examples Summary: Currently the examples fail, having a job building them would prevent that from happening. Closes https://github.com/facebook/rocksdb/pull/2007 Differential Revision: D4831570 Pulled By: yiwu-arbug fbshipit-source-id: e38adfa | 07 April 2017, 17:39:15 UTC |
| 72fc1e9 | Nikhil Benesch | 07 April 2017, 17:26:19 UTC | avoid non-existent O_DIRECT on OpenBSD Summary: OpenBSD doesn't have `O_DIRECT`, so avoid it. (RocksDB compiles successfully on OpenBSD with this patch.) Closes https://github.com/facebook/rocksdb/pull/2106 Differential Revision: D4847833 Pulled By: siying fbshipit-source-id: 214b785 | 07 April 2017, 17:39:15 UTC |
| ff97287 | Siying Dong | 07 April 2017, 03:06:34 UTC | Refactor compaction picker code Summary: 1. Move universal compaction picker to separate files compaction_picker_universal.cc and compaction_picker_universal.h. 2. Rename some functions to make the code easier to understand. 3. Move leveled compaction picking code to a dedicated class, so that we we don't need to pass some common variable around when calling functions. It also allowed us to break down LevelCompactionPicker::PickCompaction() to smaller functions. Closes https://github.com/facebook/rocksdb/pull/2100 Differential Revision: D4845948 Pulled By: siying fbshipit-source-id: efa0ab4 | 07 April 2017, 03:09:34 UTC |
| 9e72939 | Aaron Gao | 07 April 2017, 01:08:53 UTC | only FALLOC_FL_PUNCH_HOLE when ftruncate is buggy Summary: In RocksDB, we sometimes preallocate the estimated space for a file to have better perf with fallocate (if supported). Usually it is a little bit bigger than the real resulting file size. At this time, we have to let the Filesystem reclaim the space not used. Ideally, calling ftruncate to truncate the file to its real size should be enough. HOWEVER, it isn't on tmpfs, which we witness in our case, with some buggy kernel version. ftruncate a file with preallocated space doesn't change number of the blocks used by the file, which means the space not used by the file is not returned to the filesystems. So in this case we need fallocate with FALLOC_FL_PUNCH_HOLE to explicitly reclaim the used blocks. It is a hack to cope with the kernel bug and usually we should not need it. Closes https://github.com/facebook/rocksdb/pull/2102 Differential Revision: D4848934 Pulled By: lightmark fbshipit-source-id: f1b40b5 | 07 April 2017, 01:25:03 UTC |
| 343b59d | Sagar Vemuri | 06 April 2017, 21:49:13 UTC | Move various string utility functions into string_util Summary: This is an effort to club all string related utility functions into one common place, in string_util, so that it is easier for everyone to know what string processing functions are available. Right now they seem to be spread out across multiple modules, like logging and options_helper. Check the sub-commits for easier reviewing. Closes https://github.com/facebook/rocksdb/pull/2094 Differential Revision: D4837730 Pulled By: sagar0 fbshipit-source-id: 344278a | 06 April 2017, 21:54:12 UTC |
| 1d068f6 | Ayappan | 06 April 2017, 20:59:24 UTC | Fix CompactRange incorrect buffer release Summary: While running `make jtest` using IBM Java, it fails at compactRangeToLevel with the below error. ``` Run: org.rocksdb.RocksDBTest testing now -> compactRangeToLevel JVMJNCK056E JNI error in ReleaseByteArrayElements: Got memory 0x00003FFF94AA8908 from object 0x00000000000C7F78, releasing from 0x00000000000C7F68 JVMJNCK077E Error detected in org/rocksdb/RocksDB.compactRange0(J[BI[BIZII)V JVMJNCK024E JNI error detected. Aborting. JVMJNCK025I Use -Xcheck:jni:nonfatal to continue running when errors are detected. Fatal error: JNI error Makefile:205: recipe for target 'run_test' failed make[1]: *** [run_test] Error 87 make[1]: Leaving directory '/home/ubuntu/rocksdb/java' Makefile:1542: recipe for target 'jtest' failed make: *** [jtest] Error 2 ``` After checking the code, it is vivid that we are messing up the `ReleaseByteArrayElements` args in `rocksdb_compactrange_helper`. ``` ................. 1959 s = db->CompactRange(compact_options, &begin_slice, &end_slice); 1960 } Closes https://github.com/facebook/rocksdb/pull/2060 Differential Revision: D4831427 Pulled By: yiwu-arbug fbshipit-source-id: dd02037 | 06 April 2017, 21:09:13 UTC |
| df6f5a3 | Yi Wu | 06 April 2017, 20:59:31 UTC | Move memtable related files into memtable directory Summary: Move memtable related files into memtable directory. Closes https://github.com/facebook/rocksdb/pull/2087 Differential Revision: D4829242 Pulled By: yiwu-arbug fbshipit-source-id: ca70ab6 | 06 April 2017, 21:09:13 UTC |
| 107c5f6 | Tamir Duberstein | 06 April 2017, 20:54:49 UTC | CMake: more MinGW fixes Summary: siying this is a resubmission of #2081 with the 4th commit fixed. From that commit message: > Note that the previous use of quotes in PLATFORM_{CC,CXX}FLAGS was incorrect and caused GCC to produce the incorrect define: > > #define ROCKSDB_JEMALLOC -DJEMALLOC_NO_DEMANGLE 1 > > This was the cause of the Linux build failure on the previous version of this change. I've tested this locally, and the Linux build succeeds now. Closes https://github.com/facebook/rocksdb/pull/2097 Differential Revision: D4839964 Pulled By: siying fbshipit-source-id: cc51322 | 06 April 2017, 21:09:13 UTC |
| d2dce56 | Siying Dong | 06 April 2017, 02:02:00 UTC | Move some files under util/ to separate dirs Summary: Move some files under util/ to new directories env/, monitoring/ options/ and cache/ Closes https://github.com/facebook/rocksdb/pull/2090 Differential Revision: D4833681 Pulled By: siying fbshipit-source-id: 2fd8bef | 06 April 2017, 02:09:16 UTC |
| c50e375 | Islam AbdelRahman | 06 April 2017, 00:18:21 UTC | Use a human readable size for level report Summary: Current ``` ** Compaction Stats [default] ** Level Files Size(MB} Score Read(GB} Rn(GB} Rnp1(GB} Write(GB} Wnew(GB} Moved(GB} W-Amp Rd(MB/s} Wr(MB/s} Comp(sec} Comp(cnt} Avg(sec} KeyIn KeyDrop ---------------------------------------------------------------------------------------------------------------------------------------------------------- L0 2/0 49.02 0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 76.1 1 2 0.322 0 0 Sum 2/0 49.02 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 76.1 1 2 0.322 0 0 Int 0/0 0.00 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 76.1 1 2 0.322 0 0 ``` New ``` ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) Comp(cnt) Avg(sec) KeyIn Key Closes https://github.com/facebook/rocksdb/pull/2055 Differential Revision: D4804576 Pulled By: IslamAbdelRahman fbshipit-source-id: 719be6a | 06 April 2017, 00:24:19 UTC |
| ce64b8b | Siying Dong | 06 April 2017, 00:14:05 UTC | Divide db/db_impl.cc Summary: db_impl.cc is too large to manage. Divide db_impl.cc into db/db_impl.cc, db/db_impl_compaction_flush.cc, db/db_impl_files.cc, db/db_impl_open.cc and db/db_impl_write.cc. Closes https://github.com/facebook/rocksdb/pull/2095 Differential Revision: D4838188 Pulled By: siying fbshipit-source-id: c5f3059 | 06 April 2017, 00:24:19 UTC |
| 02799ad | Aaron Gao | 05 April 2017, 22:56:45 UTC | Revert "delete fallocate with punch_hole" Summary: This reverts commit 0fd574926cc9be7309c2247092d6b337fb022a5d. It breaks tmpfs on kernel 4.0 or earlier. We will wait for the fix before remove this part Closes https://github.com/facebook/rocksdb/pull/2096 Differential Revision: D4839661 Pulled By: lightmark fbshipit-source-id: 574a51f | 05 April 2017, 23:10:09 UTC |
| e2a7b20 | Maysam Yabandeh | 05 April 2017, 19:16:13 UTC | Release note for partition filters Summary: I tagged it experimental since the configuring the filter partitions by size will not be ready for this release. Closes https://github.com/facebook/rocksdb/pull/2049 Differential Revision: D4831575 Pulled By: maysamyabandeh fbshipit-source-id: fcc2d25 | 05 April 2017, 19:24:22 UTC |
| af256eb | Aaron Gao | 05 April 2017, 19:10:01 UTC | build db every monday Summary: Rebuilding regression db at the first run every monday. Closes https://github.com/facebook/rocksdb/pull/2093 Differential Revision: D4836961 Pulled By: lightmark fbshipit-source-id: 22d6c25 | 05 April 2017, 19:24:21 UTC |
| e5a1372 | Dmitri Smirnov | 05 April 2017, 18:22:36 UTC | Rework test running script. Summary: Rework test running script. New options SuiteRun - runs specified executables as google suite test cases in parallel. Run - this option now runs executables in parallel the same as 'tests' RunAll - scans for test executables and attempts to run them all as suites except those mentiones in RunOnly (hardcoded in the script) or specified either in $ExcludeTestCases $ExcludeTestExe Closes https://github.com/facebook/rocksdb/pull/2089 Differential Revision: D4832212 Pulled By: yiwu-arbug fbshipit-source-id: 954990c | 05 April 2017, 18:39:20 UTC |
| d659faa | Andrew Kryczka | 05 April 2017, 00:57:27 UTC | Level-based L0->L0 compaction Summary: Level-based L0->L0 compaction operates on spans of files that aren't currently being compacted. It reduces the number of L0 files, thus making write stall conditions harder to reach. - L0->L0 is triggered when base level is unavailable due to pending compactions - L0->L0 always outputs one file of at most `max_level0_burst_file_size` bytes. - Subcompactions are disabled for L0->L0 since we want to output one file. - Input files are chosen as the longest span of available files that will fit within the size limit. This minimizes number of files in L0. Closes https://github.com/facebook/rocksdb/pull/2027 Differential Revision: D4760318 Pulled By: ajkr fbshipit-source-id: 9d07183 | 05 April 2017, 01:09:11 UTC |
| a12306f | Jevon Qiao | 04 April 2017, 23:24:08 UTC | Add a notice on gflags installation in INSTALL.md Summary: Add a notice on gflags installation to help people build and install RocksDB from source code correctly. Please see https://github.com/facebook/rocksdb/issues/1775 Closes https://github.com/facebook/rocksdb/pull/2061 Differential Revision: D4831323 Pulled By: yiwu-arbug fbshipit-source-id: 0df1f0e | 04 April 2017, 23:39:25 UTC |
| 43010a9 | Siying Dong | 04 April 2017, 23:23:00 UTC | Revert "[rocksdb][PR] CMake: more MinGW fixes" fbshipit-source-id: 43b4529 | 04 April 2017, 23:24:26 UTC |
| a30b75c | Islam AbdelRahman | 04 April 2017, 23:09:31 UTC | Add buckifier script to github repo Summary: Add buckifier script and TARGETS file to github repo Closes https://github.com/facebook/rocksdb/pull/2083 Differential Revision: D4825822 Pulled By: IslamAbdelRahman fbshipit-source-id: 205f4a7 | 04 April 2017, 23:24:26 UTC |
| 3450ac8 | Tamir Duberstein | 04 April 2017, 21:53:30 UTC | CMake: more MinGW fixes Summary: See individual commits. yuslepukhin siying Closes https://github.com/facebook/rocksdb/pull/2081 Differential Revision: D4824639 Pulled By: IslamAbdelRahman fbshipit-source-id: 2fc2b00 | 04 April 2017, 22:09:17 UTC |
| 90cfd46 | Aaron Gao | 04 April 2017, 21:17:16 UTC | update IterKey that can get user key and internal key explicitly Summary: to void future bug that caused by the mix of userkey/internalkey Closes https://github.com/facebook/rocksdb/pull/2084 Differential Revision: D4825889 Pulled By: lightmark fbshipit-source-id: 28411db | 04 April 2017, 21:24:20 UTC |
| e2c6c06 | Andrew Kryczka | 04 April 2017, 18:12:47 UTC | add TimedEnv Summary: I've needed Env timing measurements a few times now, so finally built something for it. Closes https://github.com/facebook/rocksdb/pull/2073 Differential Revision: D4811231 Pulled By: ajkr fbshipit-source-id: 218a249 | 04 April 2017, 18:24:12 UTC |
| 9e44531 | Yi Wu | 04 April 2017, 17:19:33 UTC | Refactor WriteImpl (pipeline write part 1) Summary: Refactor WriteImpl() so when I plug-in the pipeline write code (which is an alternative approach for WriteThread), some of the logic can be reuse. I split out the following methods from WriteImpl(): * PreprocessWrite() * HandleWALFull() (previous MaybeFlushColumnFamilies()) * HandleWriteBufferFull() * WriteToWAL() Also adding a constructor to WriteThread::Writer, and move WriteContext into db_impl.h. No real logic change in this patch. Closes https://github.com/facebook/rocksdb/pull/2042 Differential Revision: D4781014 Pulled By: yiwu-arbug fbshipit-source-id: d45ca18 | 04 April 2017, 17:24:32 UTC |
| 6ef8c62 | Siying Dong | 04 April 2017, 01:27:24 UTC | Move auto_roll_logger and filename out of db/ Summary: It is confusing to have auto_roll_logger to stay under db/, which has nothing to do with database. Move filename together as it is a dependency. Closes https://github.com/facebook/rocksdb/pull/2080 Differential Revision: D4821141 Pulled By: siying fbshipit-source-id: ca7d768 | 04 April 2017, 01:39:14 UTC |
| a1c469d | Maysam Yabandeh | 03 April 2017, 21:58:10 UTC | Add release notes for PinnableSlice Summary: Closes https://github.com/facebook/rocksdb/pull/2037 Differential Revision: D4822085 Pulled By: maysamyabandeh fbshipit-source-id: 9d5a986 | 03 April 2017, 22:09:19 UTC |
| 0537f51 | Aaron Gao | 03 April 2017, 20:01:06 UTC | fix run_remote with strong quoting Summary: add \' pair to avoid bash expanding before run remotely Closes https://github.com/facebook/rocksdb/pull/2079 Differential Revision: D4821342 Pulled By: lightmark fbshipit-source-id: 418ba41 | 03 April 2017, 20:09:12 UTC |
| 72e6000 | Doogie Lee | 03 April 2017, 18:58:00 UTC | fixed misses on Centos library installation instructions Summary: Following the instructions on INSTALL.md, I found some errors while installing gflags and zstandard libraries on Centos and fixed them. Thanks :) Closes https://github.com/facebook/rocksdb/pull/2077 Differential Revision: D4820804 Pulled By: ajkr fbshipit-source-id: db4abb3 | 03 April 2017, 19:09:14 UTC |
| 88cc81d | Siying Dong | 03 April 2017, 18:24:39 UTC | auto_roll_logger_test to move away from real sleep Summary: auto_roll_logger_test relies on timing conditon that some operations finish within 1 seconds. This caused flaky tests. Move away from real timing and sleep and use fake time to verify the time-based rolling. Closes https://github.com/facebook/rocksdb/pull/2066 Differential Revision: D4810647 Pulled By: siying fbshipit-source-id: c54d994 | 03 April 2017, 18:39:09 UTC |
| d25e28d | Nikhil Benesch | 03 April 2017, 18:19:29 UTC | replace sometimes-undefined uint type with unsigned int Summary: `uint` is nonstandard and not a built-in type on all compilers; replace it with the always-valid `unsigned int`. I assume this went unnoticed because it's inside an `#ifdef ROCKDB_JEMALLOC`. Closes https://github.com/facebook/rocksdb/pull/2075 Differential Revision: D4820427 Pulled By: ajkr fbshipit-source-id: 0876561 | 03 April 2017, 18:39:09 UTC |
| a1d7e48 | Andrew Kryczka | 03 April 2017, 18:10:34 UTC | Add L0 write-amp to compaction level stats Summary: Previously it always showed 0.0 for L0 write-amp because we were dividing by bytes read from non-output level. For L0, we should instead divide by bytes ingested to the DB. Note the numerator (bytes written to L0) includes flush bytes. Closes https://github.com/facebook/rocksdb/pull/2078 Differential Revision: D4816902 Pulled By: ajkr fbshipit-source-id: 7dca31a | 03 April 2017, 18:24:10 UTC |
| b6d6090 | Tamir Duberstein | 03 April 2017, 16:58:35 UTC | CMake: support AVX2 in MinGW Summary: yuslepukhin See individual commits. Closes https://github.com/facebook/rocksdb/pull/2051 Differential Revision: D4819697 Pulled By: siying fbshipit-source-id: 75c1de4 | 03 April 2017, 17:11:46 UTC |
| bd7d138 | Aaron Gao | 01 April 2017, 01:40:45 UTC | test remote instead run remote in regression test Summary: avoid exit when $? != 0 Closes https://github.com/facebook/rocksdb/pull/2076 Differential Revision: D4815655 Pulled By: lightmark fbshipit-source-id: 829299e | 01 April 2017, 01:54:11 UTC |
| c81a805 | Aaron Gao | 31 March 2017, 21:51:19 UTC | test db existence on the remote host Summary: Fix the bug that previous test existence locally Closes https://github.com/facebook/rocksdb/pull/2074 Differential Revision: D4813631 Pulled By: lightmark fbshipit-source-id: eceb0d9 | 31 March 2017, 21:54:17 UTC |
| 5fc1e67 | Aaron Gao | 31 March 2017, 18:58:51 UTC | add -rf when remove db in regression test Summary: force remove db dir when rebuilding Closes https://github.com/facebook/rocksdb/pull/2067 Differential Revision: D4811926 Pulled By: lightmark fbshipit-source-id: ab068a2 | 31 March 2017, 19:10:23 UTC |
| 4ab4049 | Adam Retter | 31 March 2017, 16:59:02 UTC | gflags has moved to GitHub Summary: Closes https://github.com/facebook/rocksdb/issues/2068 Closes https://github.com/facebook/rocksdb/pull/2072 Differential Revision: D4810855 Pulled By: ajkr fbshipit-source-id: 288ddb9 | 31 March 2017, 17:09:21 UTC |
| 4e00650 | Andrew Kryczka | 31 March 2017, 06:27:42 UTC | make all DB::Get overloads virtual Summary: some fbcode services override it, we need to keep it virtual. original change: #1756 Closes https://github.com/facebook/rocksdb/pull/2065 Differential Revision: D4808123 Pulled By: ajkr fbshipit-source-id: 5eaeea7 | 31 March 2017, 06:39:14 UTC |
| 6401a8b | Orgad Shaneh | 30 March 2017, 23:47:19 UTC | Fix build with MinGW Summary: There still are many warnings (most of them about invalid printf format for long long), but it builds if FAIL_ON_WARNINGS is disabled. Closes https://github.com/facebook/rocksdb/pull/2052 Differential Revision: D4807355 Pulled By: siying fbshipit-source-id: ef03786 | 30 March 2017, 23:54:52 UTC |
| 80fe5b3 | Andrew Kryczka | 30 March 2017, 23:45:39 UTC | disable test: DeleteSchedulerTest.DynamicRateLimiting1 Summary: temporarily disable since it isn't working on travis. Closes https://github.com/facebook/rocksdb/pull/2064 Differential Revision: D4807373 Pulled By: ajkr fbshipit-source-id: f2bb2b0 | 30 March 2017, 23:54:52 UTC |
| a9c86f5 | Andrew Kryczka | 30 March 2017, 21:44:56 UTC | backup garbage collect shared_checksum tmp files Summary: previously we only cleaned up .tmp files under "shared/" and "private/" directories in case the previous backup failed. we need to do the same for "shared_checksum/"; otherwise, the subsequent backup will fail if it tries to backup at least one of the same files. Closes https://github.com/facebook/rocksdb/pull/2062 Differential Revision: D4805599 Pulled By: ajkr fbshipit-source-id: eaa6088 | 30 March 2017, 21:54:12 UTC |
| da175f7 | Aaron Gao | 30 March 2017, 20:56:52 UTC | exit with code 2 when there is already a db_bench running in regression test Summary: Closes https://github.com/facebook/rocksdb/pull/2063 Differential Revision: D4805507 Pulled By: lightmark fbshipit-source-id: 59ccb18 | 30 March 2017, 21:09:11 UTC |
| 0ee7f04 | Adam Retter | 30 March 2017, 19:04:09 UTC | Added missing options to RocksJava Summary: This adds almost all missing options to RocksJava Closes https://github.com/facebook/rocksdb/pull/2039 Differential Revision: D4779991 Pulled By: siying fbshipit-source-id: 4a1bf28 | 30 March 2017, 19:09:21 UTC |
| c6d04f2 | Sagar Vemuri | 30 March 2017, 18:57:05 UTC | Option to fail a request as incomplete when skipping too many internal keys Summary: Operations like Seek/Next/Prev sometimes take too long to complete when there are many internal keys to be skipped. Adding an option, max_skippable_internal_keys -- which could be used to set a threshold for the maximum number of keys that can be skipped, will help to address these cases where it is much better to fail a request (as incomplete) than to wait for a considerable time for the request to complete. This feature -- to fail an iterator seek request as incomplete, is disabled by default when max_skippable_internal_keys = 0. It is enabled only when max_skippable_internal_keys > 0. This feature is based on the discussion mentioned in the PR https://github.com/facebook/rocksdb/pull/1084. Closes https://github.com/facebook/rocksdb/pull/2000 Differential Revision: D4753223 Pulled By: sagar0 fbshipit-source-id: 1c973f7 | 30 March 2017, 19:09:21 UTC |
| 58179ec | Herman Lee | 30 March 2017, 17:31:26 UTC | Cleanup of ThreadStatusUtil structures should use the DB's reference Summary: instead of thread_local The cleanup path for the rocksdb database might not have the thread_updater_local_cache_ pointer initialized because the thread executing the cleanup is likely not a rocksdb thread. This results in a memory leak detected by Valgrind. The cleanup code path should use the thread_status_updater pointer obtained from the DB object instead of a thread local one. Closes https://github.com/facebook/rocksdb/pull/2059 Differential Revision: D4801611 Pulled By: hermanlee fbshipit-source-id: 407d7de | 30 March 2017, 17:39:13 UTC |
| f360764 | Aaron Gao | 30 March 2017, 01:06:24 UTC | add ldb build to regression test Summary: add ldb to regression test in order to enable reuse db by creating checkpoint Closes https://github.com/facebook/rocksdb/pull/2030 Differential Revision: D4800549 Pulled By: lightmark fbshipit-source-id: d3a7325 | 30 March 2017, 01:24:11 UTC |
| 8a8c967 | Min Wei | 30 March 2017, 00:34:36 UTC | Enable Fast CRC32 for Win64 Summary: Currently the fast crc32 path is not enabled on Windows. I am trying to enable it here, hopefully, with the minimum impact to the existing code structure. Closes https://github.com/facebook/rocksdb/pull/2033 Differential Revision: D4770635 Pulled By: siying fbshipit-source-id: 676f8b8 | 30 March 2017, 00:39:19 UTC |
| f9813b8 | Mikhail Antonov | 30 March 2017, 00:13:43 UTC | Added SstFileWriter construtor without explicit comparator to JNI api Summary: Adding API missing after https://github.com/facebook/rocksdb/commit/1ffbdfd9a7637b6517053842386d71df2cd00d9b#diff-b94146418eed4a9c1bf324041b95b279. adamretter IslamAbdelRahman Tested locally. Closes https://github.com/facebook/rocksdb/pull/2028 Differential Revision: D4762817 Pulled By: IslamAbdelRahman fbshipit-source-id: 833f478 | 30 March 2017, 00:24:14 UTC |
| 8d3cb4f | Sharan Suryanarayanan | 29 March 2017, 23:58:45 UTC | Added naming of backup engine threads Summary: Changed the naming of backup engine threads from "ldb" to "backup_engine" Closes https://github.com/facebook/rocksdb/pull/2053 Differential Revision: D4799325 Pulled By: ajkr fbshipit-source-id: 046893f | 30 March 2017, 00:10:46 UTC |
| 67d7623 | Siying Dong | 29 March 2017, 18:42:56 UTC | Expose the stalling information through DB::GetProperty() Summary: Add two DB properties: rocksdb.actual_delayed_write_rate and rocksdb.is_write_stooped, for people to know whether current writes are being throttled. Closes https://github.com/facebook/rocksdb/pull/2043 Differential Revision: D4782975 Pulled By: siying fbshipit-source-id: 6b2f5cf | 29 March 2017, 18:54:20 UTC |
| 0fd5749 | Aaron Gao | 28 March 2017, 22:47:54 UTC | delete fallocate with punch_hole Summary: As discuss in this thread: https://www.facebook.com/groups/rocksdb.dev/permalink/1218043868294125/ We remove fallocate with FALLOC_FL_PUNCH_HOLE because the recent bug on xfs in kernel 4.x+ that align file size to page size even with FALLOC_FL_KEEP_SIZE | FALLOC_FL_PUNCH_HOLE. Closes https://github.com/facebook/rocksdb/pull/2038 Differential Revision: D4779974 Pulled By: siying fbshipit-source-id: 5f54625 | 28 March 2017, 22:54:12 UTC |
| 41fe9ad | Yi Wu | 28 March 2017, 20:26:53 UTC | Hide usage of compaction_options_fifo from lite build Summary: ...to fix lite build error. Closes https://github.com/facebook/rocksdb/pull/2046 Differential Revision: D4785910 Pulled By: yiwu-arbug fbshipit-source-id: b591f27 | 28 March 2017, 20:39:13 UTC |
| e7731d1 | Maysam Yabandeh | 28 March 2017, 18:56:56 UTC | Configure index partition size Summary: Allow the users to specify the target index partition size. With this patch an index partition is cut before its estimated in-memory size goes above the configured value for metadata_block_size. The filter partitions are still cut right after an index partition is cut. Closes https://github.com/facebook/rocksdb/pull/2041 Differential Revision: D4780216 Pulled By: maysamyabandeh fbshipit-source-id: 95a0831 | 28 March 2017, 19:09:12 UTC |
| 69c8d52 | Ayappan | 27 March 2017, 20:57:07 UTC | Fix jni library name for PowerPC Architecture Summary: Right now, building rocksdbjava in PowerPC is broken due to JNI library name. I figured it out that "uname -m" and java's os.arch matches in PowerPC architecture. I made use of this advantage to fix the issue. More info can found from this issue --> https://github.com/facebook/rocksdb/issues/1317 Closes https://github.com/facebook/rocksdb/pull/2040 Differential Revision: D4779967 Pulled By: siying fbshipit-source-id: 259f939 | 27 March 2017, 21:09:11 UTC |
| 34a7085 | Maysam Yabandeh | 25 March 2017, 00:07:38 UTC | Fix segmentation fault caused by #1961 Summary: Fixes #1961 which causes a segfault when filter_policy is nullptr and both pin_l0_filter_and_index_blocks_in_cache/cache_index_and_filter_blocks are set. Closes https://github.com/facebook/rocksdb/pull/2029 Differential Revision: D4764862 Pulled By: maysamyabandeh fbshipit-source-id: 05bd695 | 25 March 2017, 00:24:11 UTC |
| 8dee8ca | Shu Zhang | 24 March 2017, 23:58:51 UTC | Enable fifo compaction benchmark to db_bench Summary: Added fifo benchmark to db_bench. One thing i am not sure is that i am using CompactRange() instead of CompactFiles(). (may cause performance skew because CompactionRange() is not happening in current thread?) For CompactFiles(), for some reason FIFO compaction doesn't work as expected. More insight is welcomed. I guess FIFO compaction doesn't work with file names? igorcanadi test cmd: ./db_bench --compaction_style=2 --benchmarks=fillseqdeterministic --disable_auto_compactions --num_levels=1 --fifo_compaction_max_table_files_size_mb=10 ---------------------- DB 0 LSM --------------------- Level[0]: /000014.sst(size: 4211014 bytes) fillseqdeterministic : 4.731 micros/op 211381 ops/sec; 23.4 MB/s Closes https://github.com/facebook/rocksdb/pull/1734 Differential Revision: D4774964 Pulled By: siying fbshipit-source-id: 9d08df6 | 25 March 2017, 00:09:15 UTC |
| a5c8b54 | Andrew Kryczka | 24 March 2017, 02:20:04 UTC | fix a header include Summary: need to consistently include "rocksdb/persistent_cache.h" to fix internal build Closes https://github.com/facebook/rocksdb/pull/2034 Differential Revision: D4768101 Pulled By: ajkr fbshipit-source-id: 2ecb07f | 24 March 2017, 02:24:22 UTC |
| 91b5feb | Siying Dong | 24 March 2017, 01:06:44 UTC | Fix Windows Build broken by a recent commit Summary: Closes https://github.com/facebook/rocksdb/pull/2032 Differential Revision: D4766260 Pulled By: siying fbshipit-source-id: 415daa4 | 24 March 2017, 01:09:57 UTC |
| 41ccae6 | Warren Falk | 23 March 2017, 22:50:51 UTC | Add C API functions (and tests) for WriteBatchWithIndex Summary: I've added functions to the C API to support WriteBatchWithIndex as requested in #1833. I've also added unit tests to c_test I've implemented the WriteBatchWithIndex variation of every function available for regular WriteBatch. And added additional functions unique to WriteBatchWithIndex. For now, the following is omitted: 1. The ability to create WriteBatchWithIndex's custom batch-only iterator as I'm not sure what its purpose is. It should be possible to add later if anyone wants it. 2. The ability to create the batch with a fallback comparator, since it appears to be unnecessary. I believe the column family comparator will be used for this, meaning those using a custom comparator can just use the column family variations. Closes https://github.com/facebook/rocksdb/pull/1985 Differential Revision: D4760039 Pulled By: siying fbshipit-source-id: 393227e | 23 March 2017, 22:54:13 UTC |
| 88bb6f6 | Siying Dong | 23 March 2017, 22:09:15 UTC | non_shm CI should run tests on /tmp Summary: Since non_shn CI was made to run in parallel, /dev/shm is automatically used. It defeated the purpose of the test to cover a non-ramfs file system. Closes https://github.com/facebook/rocksdb/pull/2031 Differential Revision: D4764804 Pulled By: siying fbshipit-source-id: 5666bda | 23 March 2017, 22:24:12 UTC |
| 8888de2 | Sagar Vemuri | 23 March 2017, 18:20:47 UTC | Update .gitignore file in examples Summary: options_file_example should be added in .gitignore so that it does not show up as an untracked file in `git status`. Closes https://github.com/facebook/rocksdb/pull/2026 Differential Revision: D4759402 Pulled By: sagar0 fbshipit-source-id: d7fe133 | 23 March 2017, 18:24:14 UTC |
| 203136e | Ayappan | 23 March 2017, 01:05:39 UTC | Fix Compilation errors when using IBM Java Summary: PR to fix this issue -> https://github.com/facebook/rocksdb/issues/1926 Closes https://github.com/facebook/rocksdb/pull/1965 Differential Revision: D4682411 Pulled By: siying fbshipit-source-id: a519be1 | 23 March 2017, 01:09:10 UTC |
| f4fce47 | Daniel Black | 23 March 2017, 01:03:58 UTC | Fix clang compile error - [-Werror,-Wunused-lambda-capture] Summary: Errors where: db/version_set.cc:1535:20: error: lambda capture 'this' is not used [-Werror,-Wunused-lambda-capture] [this](const Fsize& f1, const Fsize& f2) -> bool { ^ db/version_set.cc:1541:20: error: lambda capture 'this' is not used [-Werror,-Wunused-lambda-capture] [this](const Fsize& f1, const Fsize& f2) -> bool { ^ db/db_test.cc:2983:27: error: lambda capture 'kNumPutsBeforeWaitForFlush' is not required to be captured for this use [-Werror,-Wunused-lambda-capture] auto gen_l0_kb = [this, kNumPutsBeforeWaitForFlush](int size) { ^ Closes https://github.com/facebook/rocksdb/pull/1972 Differential Revision: D4685991 Pulled By: siying fbshipit-source-id: 9125379 | 23 March 2017, 01:09:10 UTC |
| a084b26 | Siying Dong | 23 March 2017, 00:45:15 UTC | Blog post for releasing 5.2.1 Summary: Closes https://github.com/facebook/rocksdb/pull/2025 Differential Revision: D4759957 Pulled By: siying fbshipit-source-id: 5b5b625 | 23 March 2017, 00:54:18 UTC |
| 15950fe | Siying Dong | 22 March 2017, 22:45:15 UTC | Remove ASSERT_EQ(boolean, ...) Summary: Closes https://github.com/facebook/rocksdb/pull/2024 Differential Revision: D4755420 Pulled By: siying fbshipit-source-id: 7332ab1 | 22 March 2017, 22:54:12 UTC |
| 3e56c7e | Aaron Gao | 22 March 2017, 18:50:25 UTC | make total_log_size_ atomic Summary: make total_log_size_ atomic to avoid overflow caused by data race. Closes https://github.com/facebook/rocksdb/pull/2019 Differential Revision: D4751391 Pulled By: siying fbshipit-source-id: fac01dd | 22 March 2017, 18:54:40 UTC |
| 909028e | Siying Dong | 22 March 2017, 18:13:55 UTC | HISTORY.md for log_size_for_flush in CreateCheckpoint() Summary: Closes https://github.com/facebook/rocksdb/pull/2021 Differential Revision: D4755324 Pulled By: siying fbshipit-source-id: c8d7955 | 22 March 2017, 18:24:12 UTC |
| be723a8 | Dmitri Smirnov | 22 March 2017, 18:07:52 UTC | Optionally construct Post Processing Info map in MemTableInserter Summary: MemTableInserter default constructs Post processing info std::map. However, on Windows with 2015 STL the default constructed map still dynamically allocates one node which shows up on a profiler and we loose ~40% throughput on fillrandom benchmark. Solution: declare a map as std::aligned storage and optionally construct. This addresses https://github.com/facebook/rocksdb/issues/1976 Before: ------------------------------------------------------------------- Initializing RocksDB Options from command-line flags DB path: [k:\data\BulkLoadRandom_10M_fillonly] fillrandom : 2.775 micros/op 360334 ops/sec; 280.4 MB/s Microseconds per write: Count: 10000000 Average: 2.7749 StdDev: 39.92 Min: 1 Median: 2.0826 Max: 26051 Percentiles: P50: 2.08 P75: 2.55 P99: 3.55 P99.9: 9.58 P99.99: 51.5**6 ------------------------------------------------------ After: Initializing RocksDB Options from command-line flags DB path: [k:\data\BulkLoadRandom_10M_fillon Closes https://github.com/facebook/rocksdb/pull/2011 Differential Revision: D4740823 Pulled By: siying fbshipit-source-id: 1daaa2c | 22 March 2017, 18:24:12 UTC |
| e474df9 | Siying Dong | 22 March 2017, 18:05:55 UTC | db_bench: not need to check mmap for PlainTable Summary: PlainTable now supports non-mmap mode. We don't need to check it anymore. Closes https://github.com/facebook/rocksdb/pull/1882 Differential Revision: D4751643 Pulled By: siying fbshipit-source-id: ab14540 | 22 March 2017, 18:09:13 UTC |
| 8b0097b | Maysam Yabandeh | 22 March 2017, 16:11:23 UTC | Readers for partition filter Summary: This is the last split of this pull request: https://github.com/facebook/rocksdb/pull/1891 which includes the reader part as well as the tests. Closes https://github.com/facebook/rocksdb/pull/1961 Differential Revision: D4672216 Pulled By: maysamyabandeh fbshipit-source-id: 6a2b829 | 22 March 2017, 16:24:15 UTC |
| 9ef3627 | Siying Dong | 22 March 2017, 00:53:21 UTC | Allow checkpointing without flushing Summary: Add a parameter to Checkpoint::CreateCheckpoint() so that flush can be skipped if total log file size is within a threshold. Closes https://github.com/facebook/rocksdb/pull/1993 Differential Revision: D4719842 Pulled By: siying fbshipit-source-id: 4f9d9e1 | 22 March 2017, 01:09:13 UTC |
| 17866ec | Siying Dong | 22 March 2017, 00:22:10 UTC | Allow Users to change customized ldb tools' header in help printing Summary: Closes https://github.com/facebook/rocksdb/pull/2018 Differential Revision: D4748448 Pulled By: siying fbshipit-source-id: a54c2f9 | 22 March 2017, 00:39:12 UTC |
| a2a8833 | Leonidas Galanis | 21 March 2017, 19:47:36 UTC | remove deleted option from benchmark.sh Summary: Removed max_grandparent_overlap_factor from benchmark.sh since it is not a valid option anymore. Closes https://github.com/facebook/rocksdb/pull/2015 Differential Revision: D4748229 Pulled By: lgalanis fbshipit-source-id: c3869ea | 21 March 2017, 19:54:13 UTC |
| 78cb195 | Aaron Gao | 21 March 2017, 18:49:08 UTC | add checkpoint to ldb Summary: Closes https://github.com/facebook/rocksdb/pull/2017 Differential Revision: D4747656 Pulled By: lightmark fbshipit-source-id: c52f160 | 21 March 2017, 18:54:11 UTC |
| 4b04add | Raza Hussain | 21 March 2017, 18:04:06 UTC | updated solution if "make format" command fails Summary: Closes https://github.com/facebook/rocksdb/pull/2016 Differential Revision: D4747003 Pulled By: yiwu-arbug fbshipit-source-id: c1e2c5a | 21 March 2017, 18:09:10 UTC |
| 8f5bf04 | Siying Dong | 21 March 2017, 17:59:57 UTC | Flush triggered by DB write buffer size picks the oldest unflushed CF Summary: Previously, when DB write buffer size triggers, we always pick the CF with most data in its memtable to flush. This approach can minimize total flush happens. Change the behavior to always pick the oldest unflushed CF, which makes it the same behavior when max_total_wal_size hits. This approach will minimize size used by max_total_wal_size. Closes https://github.com/facebook/rocksdb/pull/1987 Differential Revision: D4703214 Pulled By: siying fbshipit-source-id: 9ff8b09 | 21 March 2017, 18:09:10 UTC |
| 6908e24 | Raza Hussain | 21 March 2017, 05:50:56 UTC | dynamic setting of stats_dump_period_sec through SetDBOption() Summary: Resolved the following issue: https://github.com/facebook/rocksdb/issues/1930 Closes https://github.com/facebook/rocksdb/pull/2004 Differential Revision: D4736764 Pulled By: yiwu-arbug fbshipit-source-id: 64fe0b7 | 21 March 2017, 05:54:13 UTC |
| 93c68b6 | Aaron Gao | 21 March 2017, 00:24:20 UTC | change regression bash file with debug mode Summary: add debug mode for better debugging refactor some regex change micros/op to ops/sec Closes https://github.com/facebook/rocksdb/pull/1999 Differential Revision: D4742806 Pulled By: lightmark fbshipit-source-id: 0fe3ae6 | 21 March 2017, 00:39:17 UTC |
| 21d8c31 | Maysam Yabandeh | 20 March 2017, 19:52:20 UTC | remove LIKELY from public headers Summary: Fixes #2008 Closes https://github.com/facebook/rocksdb/pull/2009 Differential Revision: D4739885 Pulled By: maysamyabandeh fbshipit-source-id: 36fba5d | 20 March 2017, 19:54:14 UTC |
| 36ad757 | Tamir Duberstein | 17 March 2017, 18:39:00 UTC | INSTALL: document USE_SSE Summary: Fixes #1790. adamretter siying Closes https://github.com/facebook/rocksdb/pull/1996 Differential Revision: D4730449 Pulled By: IslamAbdelRahman fbshipit-source-id: 19944fa | 17 March 2017, 18:54:11 UTC |
| 9272e12 | Aaron Gao | 17 March 2017, 18:25:01 UTC | avoid ftruncate twice in buffered io Summary: in buffered io, the filesize_ is the real size. Closes https://github.com/facebook/rocksdb/pull/1991 Differential Revision: D4711433 Pulled By: lightmark fbshipit-source-id: ad604b9 | 17 March 2017, 18:39:13 UTC |
| d52f334 | Islam AbdelRahman | 17 March 2017, 01:10:37 UTC | Break stalls when no bg work is happening Summary: Current stall will keep sleeping even if there is no Flush/Compactions to wait for, I changed the logic to break the stall if we are not flushing or compacting db_bench command used ``` # fillrandom # memtable size = 10MB # value size = 1 MB # num = 1000 # use /dev/shm ./db_bench --benchmarks="fillrandom,stats" --value_size=1048576 --write_buffer_size=10485760 --num=1000 --delayed_write_rate=XXXXX --db="/dev/shm/new_stall" | grep "Cumulative stall" ``` ``` Current results # delayed_write_rate = 1000 Kb/sec Cumulative stall: 00:00:9.031 H:M:S # delayed_write_rate = 200 Kb/sec Cumulative stall: 00:00:22.314 H:M:S # delayed_write_rate = 100 Kb/sec Cumulative stall: 00:00:42.784 H:M:S # delayed_write_rate = 50 Kb/sec Cumulative stall: 00:01:23.785 H:M:S # delayed_write_rate = 25 Kb/sec Cumulative stall: 00:02:45.702 H:M:S ``` ``` New results # delayed_write_rate = 1000 Kb/sec Cumulative stall: 00:00:9.017 H:M:S # delayed_write_rate = 200 Kb/sec Cumulative stall: 00 Closes https://github.com/facebook/rocksdb/pull/1884 Differential Revision: D4585439 Pulled By: IslamAbdelRahman fbshipit-source-id: aed2198 | 17 March 2017, 01:24:17 UTC |
| e66221a | Andrew Kryczka | 17 March 2017, 00:45:28 UTC | fix db_bench rate limiter callsites Summary: pass nullptr as stats object for db_bench-specific rate limiters since its stats are intended to capture background write activity only. Closes https://github.com/facebook/rocksdb/pull/1997 Differential Revision: D4726806 Pulled By: ajkr fbshipit-source-id: 8e4b225 | 17 March 2017, 00:54:12 UTC |
| dbae438 | Siying Dong | 16 March 2017, 20:42:13 UTC | Replace "DEPRECATED" comment to "not supported" Summary: "DEPRECATED" is ambiguous. Make it clear that those options not supported won't take effect. Closes https://github.com/facebook/rocksdb/pull/1995 Differential Revision: D4724241 Pulled By: siying fbshipit-source-id: 1e812b8 | 16 March 2017, 20:54:11 UTC |
| 995618a | Islam AbdelRahman | 16 March 2017, 19:06:04 UTC | Support SstFileManager::SetDeleteRateBytesPerSecond() Summary: Update DeleteScheduler component to support changing delete rate in runtime by introducing SstFileManager::SetDeleteRateBytesPerSecond() Closes https://github.com/facebook/rocksdb/pull/1994 Differential Revision: D4719906 Pulled By: IslamAbdelRahman fbshipit-source-id: e6b8d9e | 16 March 2017, 19:09:15 UTC |
| e191636 | Islam AbdelRahman | 16 March 2017, 02:22:52 UTC | Add macros to include file name and line number during Logging Summary: current logging ``` 2017/03/14-14:20:30.393432 7fedde9f5700 (Original Log Time 2017/03/14-14:20:30.393414) [default] Level summary: base level 1 max bytes base 268435456 files[1 0 0 0 0 0 0] max score 0.25 2017/03/14-14:20:30.393438 7fedde9f5700 [JOB 2] Try to delete WAL files size 61417909, prev total WAL file size 73820858, number of live WAL files 2. 2017/03/14-14:20:30.393464 7fedde9f5700 [DEBUG] [JOB 2] Delete /dev/shm/old_logging//MANIFEST-000001 type=3 #1 -- OK 2017/03/14-14:20:30.393472 7fedde9f5700 [DEBUG] [JOB 2] Delete /dev/shm/old_logging//000003.log type=0 #3 -- OK 2017/03/14-14:20:31.427103 7fedd49f1700 [default] New memtable created with log file: #9. Immutable memtables: 0. 2017/03/14-14:20:31.427179 7fedde9f5700 [JOB 3] Syncing log #6 2017/03/14-14:20:31.427190 7fedde9f5700 (Original Log Time 2017/03/14-14:20:31.427170) Calling FlushMemTableToOutputFile with column family [default], flush slots available 1, compaction slots allowed 1, compaction slots scheduled 1 2017/03/14-14:20:31. Closes https://github.com/facebook/rocksdb/pull/1990 Differential Revision: D4708695 Pulled By: IslamAbdelRahman fbshipit-source-id: cb8968f | 16 March 2017, 02:39:12 UTC |
| d525718 | Aaron Gao | 15 March 2017, 05:23:21 UTC | cleanup direct io flag in WritableFileWriter Summary: remove unnecessary field `direct_io_`, use `use_direct_io()` instead. Closes https://github.com/facebook/rocksdb/pull/1992 Differential Revision: D4712195 Pulled By: lightmark fbshipit-source-id: 57d34f9 | 15 March 2017, 05:39:09 UTC |
| 5fa927a | slyrz | 14 March 2017, 19:00:38 UTC | Add Xpress and ZSTD CompressionType values to C header Summary: This PR copies the missing CompressionType values ```cpp enum CompressionType : unsigned char { ... kXpressCompression = 0x6, kZSTD = 0x7, ... }; ``` into the C header. Closes https://github.com/facebook/rocksdb/pull/1989 Differential Revision: D4706265 Pulled By: siying fbshipit-source-id: e65e62a | 14 March 2017, 19:09:21 UTC |
| 1152625 | Maysam Yabandeh | 13 March 2017, 18:44:50 UTC | Pinnableslice (2nd attempt) Summary: PinnableSlice Summary: Currently the point lookup values are copied to a string provided by the user. This incures an extra memcpy cost. This patch allows doing point lookup via a PinnableSlice which pins the source memory location (instead of copying their content) and releases them after the content is consumed by the user. The old API of Get(string) is translated to the new API underneath. Here is the summary for improvements: value 100 byte: 1.8% regular, 1.2% merge values value 1k byte: 11.5% regular, 7.5% merge values value 10k byte: 26% regular, 29.9% merge values The improvement for merge could be more if we extend this approach to pin the merge output and delay the full merge operation until the user actually needs it. We have put that for future work. PS: Sometimes we observe a small decrease in performance when switching from t5452014 to this patch but with the old Get(string) API. The d Closes https://github.com/facebook/rocksdb/pull/1756 Differential Revision: D4391738 Pulled By: maysamyabandeh fbshipit-source-id: 6f3edd3 | 13 March 2017, 18:54:10 UTC |
| e5bd8de | Aaron Gao | 13 March 2017, 18:35:50 UTC | update history.md for fixing the bug that skips keys Summary: Closes https://github.com/facebook/rocksdb/pull/1986 Differential Revision: D4699152 Pulled By: siying fbshipit-source-id: b18c32c | 13 March 2017, 18:39:13 UTC |
| 1ffbdfd | Sagar Vemuri | 13 March 2017, 18:17:19 UTC | Add a new SstFileWriter constructor without explicit comparator Summary: The comparator param in SstFileWriter constructor is redundant as it already exists as a field in options. So the current SstFileWriter constructor should be deprecated in favor of a new one which does not take a comparator. Note that the jni/java apis have not been touched yet. Closes https://github.com/facebook/rocksdb/pull/1978 Differential Revision: D4685629 Pulled By: sagar0 fbshipit-source-id: 372ce96 | 13 March 2017, 18:39:13 UTC |
| ebd5639 | Reid Horuff | 13 March 2017, 17:24:52 UTC | Add ability to search for key prefix in sst_dump tool Summary: Add the flag --prefix to the sst_dump tool This flag is similar to, and exclusive from, the --from flag. --prefix=0x00FF will return all rows prefixed with 0x00FF. The --to flag may also be specified and will work as expected. These changes were used to help in debugging the power cycle corruption issue and theses changes were tested by scanning through a udb. Closes https://github.com/facebook/rocksdb/pull/1984 Differential Revision: D4691814 Pulled By: reidHoruff fbshipit-source-id: 027f261 | 13 March 2017, 17:39:12 UTC |
| e6725e8 | Maysam Yabandeh | 13 March 2017, 16:41:30 UTC | Fix some bugs in MockEnv Summary: Fixing some bugs in MockEnv so it be actually used. Closes https://github.com/facebook/rocksdb/pull/1914 Differential Revision: D4609923 Pulled By: maysamyabandeh fbshipit-source-id: ca25735 | 13 March 2017, 16:54:11 UTC |
| 900c62b | Min Wei | 10 March 2017, 19:07:33 UTC | fix compile for VS2015 Summary: Without the cast, the build will break on Windows. Closes https://github.com/facebook/rocksdb/pull/1982 Differential Revision: D4690462 Pulled By: ajkr fbshipit-source-id: c493b6c | 10 March 2017, 19:24:09 UTC |
| fe18356 | Andrew Kryczka | 09 March 2017, 20:33:48 UTC | release 5.3 Summary: Closes https://github.com/facebook/rocksdb/pull/1971 Differential Revision: D4683851 Pulled By: ajkr fbshipit-source-id: 967116e | 09 March 2017, 20:39:10 UTC |
