Skip to main content

Browse the archive

swh:1:snp:5115096b921df712aeb2a08114fede57fb3331fb
sort by:
Revision Author Date Message Commit Date
52a2cc5 sdong 05 November 2019, 18:57:56 UTC Disable error as warning 05 November 2019, 19:04:14 UTC
d60acee sdong 05 November 2019, 17:25:25 UTC Add one more #include<functional> 05 November 2019, 19:04:14 UTC
84b27f4 sdong 31 October 2019, 21:17:26 UTC Add some include<functional> 31 October 2019, 21:28:16 UTC
92bf1f5 sdong 31 October 2019, 21:10:18 UTC [FB Internal] Point to the latest tool chain. 31 October 2019, 21:28:08 UTC
6672cd0 sdong 18 July 2017, 05:17:47 UTC [fb only] revert unintended change of USE_SSE The previuos change that use gcc-5 set USE_SSE to wrong flag by mistake. Fix it. 18 July 2017, 05:22:16 UTC
1d0562e sdong 17 July 2017, 22:39:37 UTC [FB Only] use gcc-5 18 July 2017, 04:53:13 UTC
3f96ed1 Islam AbdelRahman 07 June 2016, 21:22:52 UTC backport fbcode gcc path fix 08 June 2016, 20:08:36 UTC
0645e8b Yi Wu 09 May 2016, 18:52:28 UTC Fix win build Summary: Fixing error with win build where we compare int64_t with size_t. Test Plan: make check Reviewers: andrewkr Reviewed By: andrewkr Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D57885 09 May 2016, 20:43:05 UTC
6504c31 Islam AbdelRahman 05 May 2016, 18:24:08 UTC Fix valgrind (DBIteratorTest.ReadAhead) Summary: This test is failing under valgrind because we dont delete the Env that we allocated Test Plan: run the test under valgrind Reviewers: andrewkr, yhchiang, yiwu, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D57693 09 May 2016, 20:42:56 UTC
53a86bf Andrew Kryczka 06 May 2016, 18:27:28 UTC Make max_dict_bytes optional in options string Summary: For backwards compatibility with older option strings, the parser needs to treat this argument as optional. Test Plan: Updated unit test to cover case where compression_opts is present but max_dict_bytes is omitted. Reviewers: MarkCallaghan, sdong, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D57759 06 May 2016, 18:39:26 UTC
8a5ec0e sdong 05 May 2016, 23:50:32 UTC OptimizeForSmallDb(): revert some options whose defaults were just changed Summary: We changed default options of max_open_files and max_file_opening_threads but didn't revert it in OptimizeForSmallDb(). Test Plan: Add a unit test Reviewers: igor, yhchiang, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D57675 06 May 2016, 17:18:34 UTC
b7dbbdf sdong 06 May 2016, 00:20:22 UTC BlockBasedTable::Get() not to use prefix bloom if read_options.total_order_seek = true Summary: This is to provide a way for users to skip prefix bloom in point look-up. Test Plan: Add a new unit test scenario. Reviewers: IslamAbdelRahman Subscribers: leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D57747 06 May 2016, 17:18:03 UTC
ec64b24 Yi Wu 05 May 2016, 00:20:52 UTC Fixing lite build Summary: Fixing lite build broke in unit test. `FilesPerLevel()` depends on `DB::GetProperty()`, which lite build doesn't support. Test Plan: OPT=-DROCKSDB_LITE make check -j64 Reviewers: sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D57651 05 May 2016, 17:09:16 UTC
2182bf2 Yi Wu 04 May 2016, 22:25:58 UTC Enable configurable readahead for iterators Summary: Add an option `iterator_readahead_size` to `ReadOptions` to enable configurable readahead for iterators similar to the corresponding option for compaction. Test Plan: ``` make commit_prereq ``` Reviewers: kumar.rangarajan, ott, igor, sdong Reviewed By: sdong Subscribers: yiwu, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D55419 05 May 2016, 17:09:16 UTC
2137377 Islam AbdelRahman 03 May 2016, 23:50:01 UTC Fix Iterator::Prev memory pinning bug Summary: We should not use IterKey::SetKey with copy = false except if we are pinning the iterator thru it's life time, otherwise we may release the temporarily pinned blocks and in this case the IterKey will be pointing to freed memory Test Plan: added a new test Reviewers: sdong, andrewkr Reviewed By: andrewkr Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D57561 05 May 2016, 17:09:16 UTC
a9f71f1 Patrick Chan 03 May 2016, 15:46:24 UTC sst_dump won't print size for unsupported compression type 05 May 2016, 17:09:16 UTC
1f95089 Islam AbdelRahman 03 May 2016, 04:46:30 UTC Eliminate memcpy in Iterator::Prev() by pinning blocks for keys spanning multiple blocks Summary: This diff is stacked on top of this diff https://reviews.facebook.net/D56493 The current Iterator::Prev() implementation need to copy every value since the underlying Iterator may move after reading the value. This can be optimized by making sure that the block containing the value is pinned until the Iterator move. which will improve the throughput by up to 1.5X master ``` ==> 1000000_Keys_100Byte.txt <== readreverse : 0.449 micros/op 2225887 ops/sec; 246.2 MB/s readreverse : 0.433 micros/op 2311508 ops/sec; 255.7 MB/s readreverse : 0.436 micros/op 2294335 ops/sec; 253.8 MB/s readreverse : 0.471 micros/op 2121295 ops/sec; 234.7 MB/s readreverse : 0.465 micros/op 2152227 ops/sec; 238.1 MB/s readreverse : 0.454 micros/op 2203011 ops/sec; 243.7 MB/s readreverse : 0.451 micros/op 2216095 ops/sec; 245.2 MB/s readreverse : 0.462 micros/op 2162447 ops/sec; 239.2 MB/s readreverse : 0.476 micros/op 2099151 ops/sec; 232.2 MB/s readreverse : 0.472 micros/op 2120710 ops/sec; 234.6 MB/s avg : 242.34 MB/s ==> 1000000_Keys_1KB.txt <== readreverse : 1.013 micros/op 986793 ops/sec; 978.7 MB/s readreverse : 0.942 micros/op 1061136 ops/sec; 1052.5 MB/s readreverse : 0.951 micros/op 1051901 ops/sec; 1043.3 MB/s readreverse : 0.932 micros/op 1072894 ops/sec; 1064.1 MB/s readreverse : 1.024 micros/op 976720 ops/sec; 968.7 MB/s readreverse : 0.935 micros/op 1069169 ops/sec; 1060.4 MB/s readreverse : 1.012 micros/op 988132 ops/sec; 980.1 MB/s readreverse : 0.962 micros/op 1039579 ops/sec; 1031.1 MB/s readreverse : 0.991 micros/op 1008924 ops/sec; 1000.7 MB/s readreverse : 1.004 micros/op 996144 ops/sec; 988.0 MB/s avg : 1016.76 MB/s ==> 1000000_Keys_10KB.txt <== readreverse : 4.167 micros/op 239952 ops/sec; 2346.9 MB/s readreverse : 4.070 micros/op 245713 ops/sec; 2403.3 MB/s readreverse : 4.572 micros/op 218733 ops/sec; 2139.4 MB/s readreverse : 4.497 micros/op 222388 ops/sec; 2175.2 MB/s readreverse : 4.203 micros/op 237920 ops/sec; 2327.1 MB/s readreverse : 4.206 micros/op 237756 ops/sec; 2325.5 MB/s readreverse : 4.181 micros/op 239149 ops/sec; 2339.1 MB/s readreverse : 4.157 micros/op 240552 ops/sec; 2352.8 MB/s readreverse : 4.187 micros/op 238848 ops/sec; 2336.1 MB/s readreverse : 4.106 micros/op 243575 ops/sec; 2382.4 MB/s avg : 2312.78 MB/s ==> 100000_Keys_100KB.txt <== readreverse : 41.281 micros/op 24224 ops/sec; 2366.0 MB/s readreverse : 39.722 micros/op 25175 ops/sec; 2458.9 MB/s readreverse : 40.319 micros/op 24802 ops/sec; 2422.5 MB/s readreverse : 39.762 micros/op 25149 ops/sec; 2456.4 MB/s readreverse : 40.916 micros/op 24440 ops/sec; 2387.1 MB/s readreverse : 41.188 micros/op 24278 ops/sec; 2371.4 MB/s readreverse : 40.061 micros/op 24962 ops/sec; 2438.1 MB/s readreverse : 40.221 micros/op 24862 ops/sec; 2428.4 MB/s readreverse : 40.084 micros/op 24947 ops/sec; 2436.7 MB/s readreverse : 40.655 micros/op 24597 ops/sec; 2402.4 MB/s avg : 2416.79 MB/s ==> 10000_Keys_1MB.txt <== readreverse : 298.038 micros/op 3355 ops/sec; 3355.3 MB/s readreverse : 335.001 micros/op 2985 ops/sec; 2985.1 MB/s readreverse : 286.956 micros/op 3484 ops/sec; 3484.9 MB/s readreverse : 329.954 micros/op 3030 ops/sec; 3030.8 MB/s readreverse : 306.428 micros/op 3263 ops/sec; 3263.5 MB/s readreverse : 330.749 micros/op 3023 ops/sec; 3023.5 MB/s readreverse : 328.903 micros/op 3040 ops/sec; 3040.5 MB/s readreverse : 324.853 micros/op 3078 ops/sec; 3078.4 MB/s readreverse : 320.488 micros/op 3120 ops/sec; 3120.3 MB/s readreverse : 320.536 micros/op 3119 ops/sec; 3119.8 MB/s avg : 3150.21 MB/s ``` After memcpy elimination ``` ==> 1000000_Keys_100Byte.txt <== readreverse : 0.395 micros/op 2529890 ops/sec; 279.9 MB/s readreverse : 0.368 micros/op 2715922 ops/sec; 300.5 MB/s readreverse : 0.384 micros/op 2603929 ops/sec; 288.1 MB/s readreverse : 0.375 micros/op 2663286 ops/sec; 294.6 MB/s readreverse : 0.357 micros/op 2802180 ops/sec; 310.0 MB/s readreverse : 0.363 micros/op 2757684 ops/sec; 305.1 MB/s readreverse : 0.372 micros/op 2689603 ops/sec; 297.5 MB/s readreverse : 0.379 micros/op 2638599 ops/sec; 291.9 MB/s readreverse : 0.375 micros/op 2663803 ops/sec; 294.7 MB/s readreverse : 0.375 micros/op 2665579 ops/sec; 294.9 MB/s avg: 295.72 MB/s (1.22 X) ==> 1000000_Keys_1KB.txt <== readreverse : 0.879 micros/op 1138112 ops/sec; 1128.8 MB/s readreverse : 0.842 micros/op 1187998 ops/sec; 1178.3 MB/s readreverse : 0.837 micros/op 1194915 ops/sec; 1185.1 MB/s readreverse : 0.845 micros/op 1182983 ops/sec; 1173.3 MB/s readreverse : 0.877 micros/op 1140308 ops/sec; 1131.0 MB/s readreverse : 0.849 micros/op 1177581 ops/sec; 1168.0 MB/s readreverse : 0.915 micros/op 1093284 ops/sec; 1084.3 MB/s readreverse : 0.863 micros/op 1159418 ops/sec; 1149.9 MB/s readreverse : 0.895 micros/op 1117670 ops/sec; 1108.5 MB/s readreverse : 0.852 micros/op 1174116 ops/sec; 1164.5 MB/s avg: 1147.17 MB/s (1.12 X) ==> 1000000_Keys_10KB.txt <== readreverse : 3.870 micros/op 258386 ops/sec; 2527.2 MB/s readreverse : 3.568 micros/op 280296 ops/sec; 2741.5 MB/s readreverse : 4.005 micros/op 249694 ops/sec; 2442.2 MB/s readreverse : 3.550 micros/op 281719 ops/sec; 2755.5 MB/s readreverse : 3.562 micros/op 280758 ops/sec; 2746.1 MB/s readreverse : 3.507 micros/op 285125 ops/sec; 2788.8 MB/s readreverse : 3.463 micros/op 288739 ops/sec; 2824.1 MB/s readreverse : 3.428 micros/op 291734 ops/sec; 2853.4 MB/s readreverse : 3.553 micros/op 281491 ops/sec; 2753.2 MB/s readreverse : 3.535 micros/op 282885 ops/sec; 2766.9 MB/s avg : 2719.89 MB/s (1.17 X) ==> 100000_Keys_100KB.txt <== readreverse : 22.815 micros/op 43830 ops/sec; 4281.0 MB/s readreverse : 29.957 micros/op 33381 ops/sec; 3260.4 MB/s readreverse : 25.334 micros/op 39473 ops/sec; 3855.4 MB/s readreverse : 23.037 micros/op 43409 ops/sec; 4239.8 MB/s readreverse : 27.810 micros/op 35958 ops/sec; 3512.1 MB/s readreverse : 30.327 micros/op 32973 ops/sec; 3220.6 MB/s readreverse : 29.704 micros/op 33665 ops/sec; 3288.2 MB/s readreverse : 29.423 micros/op 33987 ops/sec; 3319.6 MB/s readreverse : 23.334 micros/op 42856 ops/sec; 4185.9 MB/s readreverse : 29.969 micros/op 33368 ops/sec; 3259.1 MB/s avg : 3642.21 MB/s (1.5 X) ==> 10000_Keys_1MB.txt <== readreverse : 244.748 micros/op 4085 ops/sec; 4085.9 MB/s readreverse : 230.208 micros/op 4343 ops/sec; 4344.0 MB/s readreverse : 235.655 micros/op 4243 ops/sec; 4243.6 MB/s readreverse : 235.730 micros/op 4242 ops/sec; 4242.2 MB/s readreverse : 237.346 micros/op 4213 ops/sec; 4213.3 MB/s readreverse : 227.306 micros/op 4399 ops/sec; 4399.4 MB/s readreverse : 194.957 micros/op 5129 ops/sec; 5129.4 MB/s readreverse : 238.359 micros/op 4195 ops/sec; 4195.4 MB/s readreverse : 221.588 micros/op 4512 ops/sec; 4513.0 MB/s readreverse : 235.911 micros/op 4238 ops/sec; 4239.0 MB/s avg : 4360.52 MB/s (1.38 X) ``` Test Plan: COMPILE_WITH_ASAN=1 make check -j64 Reviewers: andrewkr, yhchiang, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D56511 05 May 2016, 17:09:16 UTC
39d156b Yi Wu 02 May 2016, 21:38:04 UTC Release RocksDB 4.8.0 Summary: Release RocksDB 4.8.0 Test Plan: N/A Reviewers: sdong, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D57525 02 May 2016, 22:09:38 UTC
cb08423 Warren Falk 02 May 2016, 17:04:37 UTC Fix #1110, 32-bit build failure on Mac OSX (#1112) Using explicit 64-bit type in conditional in platforms above 32-bits This appears to be necessary on Mac OSX as std::conditional does not appear to short circuit and evaluates the third template arg Making the third template arg be 64 bits explicitly works around this problem and will work on both 32 bit and 64+ bit platforms. 02 May 2016, 22:09:38 UTC
d9a7d8a Islam AbdelRahman 30 April 2016, 00:00:50 UTC Fix calling GetCurrentMutableCFOptions in CompactionJob::ProcessKeyValueCompaction() Summary: GetCurrentMutableCFOptions() can only be called when DB mutex is held so we cannot call it in CompactionJob::ProcessKeyValueCompaction() since it's not holding the db mutex Test Plan: make check -j64 Reviewers: sdong, andrewkr Reviewed By: andrewkr Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D57471 02 May 2016, 22:09:38 UTC
5407b7c Dmitri Smirnov 29 April 2016, 23:43:13 UTC Fix multiple issues with WinMmapFile fo sequential writing (#1108) make preallocation inline with other writable files make sure that we map no more than pre-allocated size. 02 May 2016, 22:09:38 UTC
1d2e4ef Reid Horuff 29 April 2016, 01:23:29 UTC ldb support new WAL records 29 April 2016, 18:47:24 UTC
a92049e Yi Wu 29 April 2016, 18:35:00 UTC Added EventListener::OnTableFileCreationStarted() callback Summary: Added EventListener::OnTableFileCreationStarted. EventListener::OnTableFileCreated will be called on failure case. User can check creation status via TableFileCreationInfo::status. Test Plan: unit test. Reviewers: dhruba, yhchiang, ott, sdong Reviewed By: sdong Subscribers: sdong, kradhakrishnan, IslamAbdelRahman, andrewkr, yhchiang, leveldb, ott, dhruba Differential Revision: https://reviews.facebook.net/D56337 29 April 2016, 18:35:00 UTC
e8115ce PraveenSinghRao 29 April 2016, 05:50:26 UTC Revert "Use async file handle for better parallelism (#1049)" (#1105) This reverts commit b54c34742412af0001a69c2f7d909bc05e1ea71f. Revert async file handle change as it causes failures with appveyor 29 April 2016, 05:50:26 UTC
6a14f7a sdong 08 April 2016, 00:40:42 UTC Change several option defaults Summary: Changing several option defaults: options.max_open_files changes from 5000 to -1 options.base_background_compactions changes from max_background_compactions to 1 options.wal_recovery_mode changes from kTolerateCorruptedTailRecords to kTolerateCorruptedTailRecords options.compaction_pri changes from kByCompensatedSize to kByCompensatedSize Test Plan: Write unit tests to see OldDefaults() works as expected. Reviewers: IslamAbdelRahman, yhchiang, igor Reviewed By: igor Subscribers: MarkCallaghan, yiwu, kradhakrishnan, leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D56427 29 April 2016, 00:50:58 UTC
c6c770a Peter Mattis 28 April 2016, 23:48:03 UTC Use prefix_same_as_start to avoid iteration in FindNextUserEntryInternal. (#1102) This avoids excessive iteration in tombstone fields. 28 April 2016, 23:48:03 UTC
992a8f8 sdong 28 April 2016, 23:04:31 UTC Not enable jemalloc status printing if USE_CLANG=1 Summary: Warning is printed out with USE_CLANG=1 when including jemalloc.h. Disable it in that case. Test Plan: Run db_bench with USE_CLANG=1 and not. Make sure they can all build and jemalloc status is printed out in the case where USE_CLANG is not set. Reviewers: andrewkr, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D57399 28 April 2016, 23:16:14 UTC
029022b Islam AbdelRahman 28 April 2016, 22:59:33 UTC Fix crash_test Summary: crash_test grep for 'fail' string in the output and if found it consider that we failed. Update the output to use something else Test Plan: make crash_test (still running) Reviewers: yhchiang, sdong, andrewkr Reviewed By: andrewkr Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D57381 28 April 2016, 22:59:33 UTC
a06faa6 Andrew Kryczka 28 April 2016, 22:11:28 UTC Skip PresetCompressionDict test for lite Summary: This test relies on "rocksdb.num-files-at-levelN" property that isn't implemented in rocksdb lite. So we will compile it only for non-lite builds. Test Plan: $ make -j40 check 'OPT=-g -DROCKSDB_LITE' Reviewers: sdong, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D57387 28 April 2016, 22:11:28 UTC
e7899c6 Dmitri Smirnov 28 April 2016, 18:39:12 UTC Fix build issue. (#1103) 28 April 2016, 18:39:12 UTC
0f428c5 Andrew Kryczka 28 April 2016, 17:42:10 UTC Fix compression dictionary clang osx error Summary: There was one narrowing conversion in D52287 that only showed up with clang on osx. Test Plan: $ make clean && USE_CLANG=1 DISABLE_JEMALLOC=1 TEST_TMPDIR=/dev/shm/rocksdb OPT=-g make -j32 check Reviewers: sdong, lightmark, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D57357 28 April 2016, 17:42:10 UTC
6d4832a Li Peng 28 April 2016, 09:30:44 UTC Merge pull request #1101 from flyd1005/wip-fix-typo fix typos and remove duplicated words 28 April 2016, 09:30:44 UTC
af70f9a Islam AbdelRahman 28 April 2016, 04:00:22 UTC Fix typo in build_tools/fbcode_config.sh Summary: Fix typo in build_tools/fbcode_config.sh Test Plan: none Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D57363 28 April 2016, 04:00:22 UTC
54de13a Andrew Kryczka 28 April 2016, 01:30:04 UTC Fix compression dictionary clang errors Summary: There were a few narrowing conversions that clang didn't like. Test Plan: $ make clean && USE_CLANG=1 DISABLE_JEMALLOC=1 TEST_TMPDIR=/dev/shm/rocksdb OPT=-g make -j32 check Reviewers: IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D57351 28 April 2016, 01:30:04 UTC
0850bc5 Islam AbdelRahman 28 April 2016, 01:25:19 UTC Fix build on machines without jemalloc Summary: It looks like we mistakenly enable JEMALLOC even if it's not available on the machine, that's why travis is failing Test Plan: check on my devserver check on my mac Reviewers: sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D57345 28 April 2016, 01:25:19 UTC
4032145 Andrew Kryczka 28 April 2016, 00:39:18 UTC Configurable compression in db_bench Summary: Made compression type and dictionary size configurable via environment variables. Depends on D52287. Test Plan: check these options are passed to the db. $ COMPRESSION_MAX_DICT_BYTES=65536 COMPRESSION_TYPE=LZ4 NUM_KEYS=10000000 DB_DIR=./tmp/ WAL_DIR=./tmp/ ./tools/benchmark.sh filluniquerandom ... $ grep Options.compression tmp/LOG 2016/04/22-19:11:30.397829 7f5f263a2980 Options.compression: LZ4 ... 2016/04/22-19:11:30.397837 7f5f263a2980 Options.compression_opts.max_dict_bytes: 65536 Reviewers: IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D57141 28 April 2016, 00:39:18 UTC
843d2e3 Andrew Kryczka 28 April 2016, 00:36:03 UTC Shared dictionary compression using reference block Summary: This adds a new metablock containing a shared dictionary that is used to compress all data blocks in the SST file. The size of the shared dictionary is configurable in CompressionOptions and defaults to 0. It's currently only used for zlib/lz4/lz4hc, but the block will be stored in the SST regardless of the compression type if the user chooses a nonzero dictionary size. During compaction, computes the dictionary by randomly sampling the first output file in each subcompaction. It pre-computes the intervals to sample by assuming the output file will have the maximum allowable length. In case the file is smaller, some of the pre-computed sampling intervals can be beyond end-of-file, in which case we skip over those samples and the dictionary will be a bit smaller. After the dictionary is generated using the first file in a subcompaction, it is loaded into the compression library before writing each block in each subsequent file of that subcompaction. On the read path, gets the dictionary from the metablock, if it exists. Then, loads that dictionary into the compression library before reading each block. Test Plan: new unit test Reviewers: yhchiang, IslamAbdelRahman, cyan, sdong Reviewed By: sdong Subscribers: andrewkr, yoshinorim, kradhakrishnan, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D52287 28 April 2016, 00:36:03 UTC
ad573b9 Yueh-Hsuan Chiang 27 April 2016, 23:50:51 UTC Temporarily disable CompactFiles in db_stress in its default setting Summary: As db_stress with CompactFiles possibly catches a previous bug currently, temporarily disable CompactFiles in db_stress in its default setting to allows new bug to be detected while investigating the bug in CompactFiles. Test Plan: crash test Reviewers: sdong, kradhakrishnan, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D57333 27 April 2016, 23:50:51 UTC
1c80dfa Sergey Makarenko 27 April 2016, 23:23:33 UTC Print memory allocation counters Summary: Introduced option to dump malloc statistics using new option flag. Added new command line option to db_bench tool to enable this funtionality. Also extended build to support environments with/without jemalloc. Test Plan: 1) Build rocksdb using `make` command. Launch the following command `./db_bench --benchmarks=fillrandom --dump_malloc_stats=true --num=10000000` end verified that jemalloc dump is present in LOG file. 2) Build rocksdb using `DISABLE_JEMALLOC=1 make db_bench -j32` and ran the same db_bench tool and found the following message in LOG file: "Please compile with jemalloc to enable malloc dump". 3) Also built rocksdb using `make` command on MacOS to verify behavior in non-FB environment. Also to debug build configuration change temporary changed AM_DEFAULT_VERBOSITY = 1 in Makefile to see compiler and build tools output. For case 1) -DROCKSDB_JEMALLOC was present in compiler command line. For both 2) and 3) this flag was not present. Reviewers: sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D57321 27 April 2016, 23:23:33 UTC
eb73980 Islam AbdelRahman 26 April 2016, 23:07:04 UTC Fix BackupableDBTest Summary: Fix BackupableDBTest.NoDoubleCopy and BackupableDBTest.DifferentEnvs by mocking the db files in db_env instead of backup_env_ Test Plan: make check -j64 Reviewers: sdong, andrewkr Reviewed By: andrewkr Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D57273 26 April 2016, 23:07:04 UTC
ac0e54b sdong 21 April 2016, 22:32:06 UTC CompactedDB should not be used if there is outstanding WAL files Summary: CompactedDB skips memtable. So we shouldn't use compacted DB if there is outstanding WAL files. Test Plan: Change to options.max_open_files = -1 perf context test to create a compacted DB, which we shouldn't do. Reviewers: yhchiang, kradhakrishnan, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D57057 26 April 2016, 21:22:07 UTC
d719b09 Islam AbdelRahman 26 April 2016, 19:41:07 UTC Introduce PinnedIteratorsManager (Reduce PinData() overhead / Refactor PinData) Summary: While trying to reuse PinData() / ReleasePinnedData() .. to optimize away some memcpys I realized that there is a significant overhead for using PinData() / ReleasePinnedData if they were called many times. This diff refactor the pinning logic by introducing PinnedIteratorsManager a centralized component that will be created once and will be notified whenever we need to Pin an Iterator. This implementation have much less overhead than the original implementation Test Plan: make check -j64 COMPILE_WITH_ASAN=1 make check -j64 Reviewers: yhchiang, sdong, andrewkr Reviewed By: andrewkr Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D56493 26 April 2016, 19:41:07 UTC
1995e34 Andrew Kryczka 26 April 2016, 19:33:30 UTC Retrieve file size from proper Env Summary: When db_env_ != backup_env_, InsertPathnameToSizeBytes() would use the wrong Env during backup creation. This happened because this function used backup_env_ instead of db_env_ to get WAL/data file sizes. This diff adds an argument to InsertPathnameToSizeBytes() indicating which Env to use. Test Plan: ran @anirbanb's BackupTestTool Reviewers: sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D57159 26 April 2016, 19:33:30 UTC
7c14abf Islam AbdelRahman 26 April 2016, 06:02:14 UTC Improve BytewiseComparatorImpl::FindShortestSeparator Summary: The current implementation find the first different byte and try to increment it, if it cannot it return the original key we can improve this by keep going after the first different byte to find the first non 0xFF byte and increment it After trying this patch on some logdevice sst files I see decrease in there index block size by 8.5% Test Plan: existing tests and updated test Reviewers: yhchiang, andrewkr, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D56241 26 April 2016, 06:02:14 UTC
f3eb0b5 Islam AbdelRahman 26 April 2016, 01:18:35 UTC Make EventListenerTest.CompactionReasonLevel more deterministic Summary: In this test some times automatic compactions do everything and Manual compaction become a no-op. Update the test to make sure manual compaction is not a no-op Test Plan: run the test Reviewers: andrewkr, yhchiang, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D57189 26 April 2016, 01:18:35 UTC
7b78d62 sdong 21 April 2016, 23:58:38 UTC Shouldn't report default column family's compaction stats as DB compaction stats Summary: Now we collect compaction stats per column family, but report default colum family's stat as compaction stats for DB. Fix it by reporting compaction stats per column family instead. Test Plan: Run db_bench with --num_column_families=4 and see the number fixed. Reviewers: IslamAbdelRahman, yhchiang Reviewed By: yhchiang Subscribers: leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D57063 25 April 2016, 20:56:59 UTC
995353e Dhruba Borthakur 25 April 2016, 13:27:27 UTC Fix null-pointer-dereference detected by Infer (https://github.com/facebook/infer) Test Plan: make check Reviewers: leveldb, sdong Reviewed By: sdong Subscribers: leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D57165 25 April 2016, 19:09:36 UTC
24110ce Yueh-Hsuan Chiang 25 April 2016, 19:01:01 UTC Correct Statistics FLUSH_WRITE_BYTES Summary: In https://reviews.facebook.net/D56271, we fixed an issue where we consider flush as compaction. However, that makes us mistakenly count FLUSH_WRITE_BYTES twice (one in flush_job and one in db_impl.) This patch removes the one incremented in db_impl. Test Plan: db_test Reviewers: yiwu, andrewkr, IslamAbdelRahman, kradhakrishnan, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D57111 25 April 2016, 19:01:01 UTC
b71c4e6 dx9 22 April 2016, 23:49:12 UTC Alpine Linux Build (#990) * Musl libc does not provide adaptive mutex. Added feature test for PTHREAD_MUTEX_ADAPTIVE_NP. * Musl libc does not provide backtrace(3). Added a feature check for backtrace(3). * Fixed compiler error. * Musl libc does not implement backtrace(3). Added platform check for libexecinfo. * Alpine does not appear to support gcc -pg option. By default (gcc has PIE option enabled) it fails with: gcc: error: -pie and -pg|p|profile are incompatible when linking When -fno-PIE and -nopie are used it fails with: /usr/lib/gcc/x86_64-alpine-linux-musl/5.3.0/../../../../x86_64-alpine-linux-musl/bin/ld: cannot find gcrt1.o: No such file or directory Added gcc -pg platform test and output PROFILING_FLAGS accordingly. Replaced pg var in Makefile with PROFILING_FLAGS. * fix segfault when TEST_IOCTL_FRIENDLY_TMPDIR is undefined and default candidates are not suitable * use ASSERT_DOUBLE_EQ instead of ASSERT_EQ * When compiled with ROCKSDB_MALLOC_USABLE_SIZE UniversalCompactionFourPaths and UniversalCompactionSecondPathRatio tests fail due to premature memtable flushes on systems with 16-byte alignment. Arena runs out of block space before GenerateNewFile() completes. Increased options.write_buffer_size. 22 April 2016, 23:49:12 UTC
90ffed1 Igor Canadi 22 April 2016, 22:53:32 UTC Update USERS.md with link to LinkedIn blog post (#1088) 22 April 2016, 22:53:32 UTC
99a3bf8 Naitik Shah 22 April 2016, 20:49:59 UTC Merge pull request #1068 from daaku/c-purge-old-backups rocksdb_backup_engine_purge_old_backups for C libraries 22 April 2016, 20:49:59 UTC
b54c347 PraveenSinghRao 22 April 2016, 20:27:33 UTC Use async file handle for better parallelism (#1049) 22 April 2016, 20:27:33 UTC
c146c9b Naitik Shah 22 April 2016, 20:25:05 UTC rocksdb_create_mem_env to allow C libraries to create mem env (#1066) 22 April 2016, 20:25:05 UTC
6da70c5 Naitik Shah 22 April 2016, 20:24:09 UTC expose more options in the c api (#1067) 22 April 2016, 20:24:09 UTC
6f01687 Naitik Shah 22 April 2016, 20:22:21 UTC C rocksdb_create_iterators to expose NewIterators (#1069) 22 April 2016, 20:22:21 UTC
644f978 Yueh-Hsuan Chiang 21 April 2016, 21:47:23 UTC Fix RocksDB Lite build in db_stress Summary: Fix RocksDB Lite build in db_stress Test Plan: OPT=-DROCKSDB_LITE db_stress Reviewers: IslamAbdelRahman, kradhakrishnan, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D57045 21 April 2016, 21:47:23 UTC
5bd4022 Islam AbdelRahman 21 April 2016, 17:16:28 UTC Add comparator, merge operator, property collectors to SST file properties (again) Summary: This is the original diff that I have landed and reverted and now I want to land again https://reviews.facebook.net/D34269 For old SST files we will show ``` comparator name: N/A merge operator name: N/A property collectors names: N/A ``` For new SST files with no merge operator name and with no property collectors ``` comparator name: leveldb.BytewiseComparator merge operator name: nullptr property collectors names: [] ``` for new SST files with these properties ``` comparator name: leveldb.BytewiseComparator merge operator name: UInt64AddOperator property collectors names: [DummyPropertiesCollector1,DummyPropertiesCollector2] ``` Test Plan: unittests Reviewers: andrewkr, yhchiang, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D56487 21 April 2016, 17:16:28 UTC
7a6045a Andrew Kryczka 21 April 2016, 01:53:37 UTC fix typo in HISTORY.md 21 April 2016, 01:53:37 UTC
73a847e Andrew Kryczka 21 April 2016, 01:46:54 UTC Add per-level compression ratio property Summary: This is needed so we can measure compression ratio improvements achieved by D52287. The property compares raw data size against the total file size for a given level. If the level is empty it should return 0.0. Test Plan: new unit test Reviewers: IslamAbdelRahman, yhchiang, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D56967 21 April 2016, 01:46:54 UTC
ee221d2 Dmitri Smirnov 20 April 2016, 05:54:24 UTC Introduce XPRESS compresssion on Windows. (#1081) Comparable with Snappy on comp ratio. Implemented using Windows API, does not require external package. Avaiable since Windows 8 and server 2012. Use -DXPRESS=1 with CMake to enable. 20 April 2016, 05:54:24 UTC
874c96a flabby 19 April 2016, 21:51:24 UTC Merge pull request #1083 from flabby/master fix typo in comment of options.h 19 April 2016, 21:51:24 UTC
6cbffd5 Yueh-Hsuan Chiang 19 April 2016, 21:36:09 UTC Enable testing CompactFiles in db_stress Summary: Enable testing CompactFiles in db_stress by adding flag test_compact_files to db_stress. Test Plan: ./db_stress --test_compact_files=1 --compaction_style=0 --allow_concurrent_memtable_write=false --ops_per_thread=100000 ./db_stress --test_compact_files=1 --compaction_style=1 --allow_concurrent_memtable_write=false --ops_per_thread=100000 Sample output (note that it's normal to have some CompactFiles() failed): Stress Test : 491.891 micros/op 65054 ops/sec : Wrote 21.98 MB (0.45 MB/sec) (45% of 3200352 ops) : Wrote 1440728 times : Deleted 441616 times : Single deleted 38181 times : 319251 read and 19025 found the key : Prefix scanned 640520 times : Iterator size sum is 9691415 : Iterated 319704 times : Got errors 0 times : 1323 CompactFiles() succeed : 32 CompactFiles() failed 2016/04/11-15:50:58 Verification successful Reviewers: sdong, IslamAbdelRahman, kradhakrishnan, yiwu, andrewkr Reviewed By: andrewkr Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D56565 19 April 2016, 21:36:09 UTC
b95510d Islam AbdelRahman 19 April 2016, 21:05:48 UTC Fix DBTest.RateLimitedDelete flakiness Summary: We need to enable sync_point processing before creating the SstFileManager to ensure that we are holding the bg delete scheduler thread from running Test Plan: run the test debug using printf Reviewers: sdong, yhchiang, yiwu, andrewkr Reviewed By: andrewkr Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D56871 19 April 2016, 21:05:48 UTC
6356b4d Islam AbdelRahman 19 April 2016, 20:57:05 UTC Fix nullptr dereference in adaptive_table Summary: @dulmarod Ran infer on RocksDB and found that we dereference nullptr in adaptive_table https://fb.facebook.com/groups/rocksdb/permalink/1046374415411173/ Test Plan: make check -j64 Reviewers: sdong, yhchiang, andrewkr Reviewed By: andrewkr Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D56973 19 April 2016, 20:57:05 UTC
9385fd7 Andrew Kryczka 19 April 2016, 17:32:31 UTC Delete deprecated backup classes in Java Summary: Corresponding change to D56331. Test Plan: Now build succeeds: $ make jclean && make rocksdbjava Reviewers: yhchiang, IslamAbdelRahman, adamretter Reviewed By: adamretter Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D56913 19 April 2016, 17:32:31 UTC
a2466c8 Yueh-Hsuan Chiang 18 April 2016, 21:43:33 UTC [db_stress] Make subcompaction random in crash_test Summary: Make subcompaction random in crash_test Test Plan: make crash_test and verify whether subcompaction changes randomly Reviewers: IslamAbdelRahman, kradhakrishnan, yiwu, sdong, andrewkr Reviewed By: andrewkr Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D56571 18 April 2016, 21:43:33 UTC
c3c389d Andrew Kryczka 18 April 2016, 21:34:45 UTC Fix column label for L0 write sum Summary: This is taken from the "Write(GB)" column in compaction stats, so the units should be GB, not MB. Test Plan: none Reviewers: sdong, yhchiang, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D56889 18 April 2016, 21:34:45 UTC
ec84bef Andrew Kryczka 18 April 2016, 20:01:10 UTC New legocastle output parsing Summary: Added a python script to parse combined stdout/stderr of legocastle steps. Previously we just matched words like 'Failure', which didn't work since even our test names matched that pattern. I went through all the legocastle steps to come up with strict failure regexes for the common failure cases. There is also some more complex logic to present gtest failures, since the test name and failure message are not on the same line. There will definitely be error cases that don't match any of these patterns, so we can iterate on it over time. Test Plan: no end-to-end test. I ran the legocastle steps locally and piped to my script, then verified output, e.g., $ set -o pipefail && TEST_TMPDIR=/dev/shm/rocksdb COMPILE_WITH_ASAN=1 OPT=-g make J=1 asan_check |& /usr/facebook/ops/scripts/asan_symbolize.py -d |& python build_tools/error_filter.py asan ==2058029==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60200000a414 at pc 0x4c12f6 bp 0x7ffcfb7a0520 sp 0x7ffcfb7a0518 Reviewers: kradhakrishnan Reviewed By: kradhakrishnan Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D56691 18 April 2016, 20:01:10 UTC
725184b Yi Wu 18 April 2016, 18:34:11 UTC Fix db_block_cache_test in lite build Summary: D56715 move some of the tests from db_test to db_block_cache_test. Some of them should be disabled in lite build. Test Plan: make check -j32 OPT='-DROCKSDB_LITE' make check -j32 Reviewers: sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D56907 18 April 2016, 18:34:11 UTC
290883d Yi Wu 18 April 2016, 17:47:10 UTC Fix lite build Summary: Fix rocksdb lite build after D56715. Test Plan: make -j40 'OPT=-g -DROCKSDB_LITE' Reviewers: sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D56895 18 April 2016, 17:47:10 UTC
23089fd sdong 16 April 2016, 00:13:12 UTC write_callback_test: clean test directory before running tests Summary: write_callback_test fails if previous run didn't finish cleanly. Clean the DB before runing the test. Test Plan: Run the test that see it doesn't fail any more. Reviewers: andrewkr, yhchiang, yiwu, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: kradhakrishnan, leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D56859 18 April 2016, 17:18:41 UTC
792762c Yi Wu 18 April 2016, 16:42:50 UTC Split db_test.cc Summary: Split db_test.cc into several files. Moving several helper functions into DBTestBase. Test Plan: make check Reviewers: sdong, yhchiang, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: dhruba, andrewkr, kradhakrishnan, yhchiang, leveldb, sdong Differential Revision: https://reviews.facebook.net/D56715 18 April 2016, 16:42:50 UTC
40b840f Andrew Kryczka 18 April 2016, 16:04:14 UTC Delete deprecated *BackupableDB interface for backups Summary: This interface is redundant and has been deprecated for a while. It's also unused internally. Let's delete it. I moved the comments to the corresponding functions in BackupEngine/ BackupEngineReadOnly. This caused the diff tool to not work cleanly. Test Plan: unit tests $ ./backupable_db_test Reviewers: yhchiang, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D56331 18 April 2016, 16:04:14 UTC
6affd45 Yi Wu 18 April 2016, 05:34:56 UTC Make more tests run in parallel Summary: Generate t/run-* scripts to run tests in $PARALLEL_TEST separately, then make check_0 rule execute all of them. Run `time make check` after running `make all`. master: 71 sec with this diff: 63 sec. It seems moving more tests to $PARALLEL_TEST doesn't help improve test time though. Test Plan: Run the following make check J=16 make check J=1 make check make valgrind_check J=1 make valgrind_check J=16 make_valgrind_check Reviewers: IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: leveldb, kradhakrishnan, dhruba, andrewkr, yhchiang Differential Revision: https://reviews.facebook.net/D56805 18 April 2016, 05:34:56 UTC
47833e0 Victor Tyutyunov 16 April 2016, 08:48:17 UTC Merge branch 'master' of github.com:facebook/rocksdb 16 April 2016, 08:48:17 UTC
e5c614e Victor Tyutyunov 12 April 2016, 17:35:15 UTC Fixing snapshot 0 assertion Summary: Solution is not to change db sequence number to start from 1 because 0 value is used in multiple other places. Fix covers only compact_iterator::findEarliestVisibleSnapshot with updated logic to support snapshot's numbering starting from 0. Test Plan: run: make all check it should pass all tests Reviewers: IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: lgalanis, mgalushka, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D56601 16 April 2016, 08:47:15 UTC
1b1adeb Victor Tyutyunov 12 April 2016, 17:35:15 UTC Fixing snapshot 0 assertion Test Plan: TBD Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D56601 Cosmetic changes and comment update 16 April 2016, 08:45:31 UTC
6d436a3 sdong 16 April 2016, 00:00:56 UTC DBTest.HardLimit made more deterministic Summary: In DBTest.HardLimit, multiple flushes may merge into one, based on thread scheduling. Avoid it by waiting each flush to finish before generating the next one. Test Plan: Run test in parallel several times and see it doesn't fail any more. Reviewers: yhchiang, kradhakrishnan, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: yiwu, leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D56853 16 April 2016, 00:36:57 UTC
994d9bc Islam AbdelRahman 15 April 2016, 23:21:03 UTC Make parallel valgrind watch-log more readable Summary: Before {F1131675} After {F1131681} This will have no effect on normal make check Test Plan: make watch-log Reviewers: sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D56847 15 April 2016, 23:21:03 UTC
9d35ae6 sdong 15 April 2016, 22:44:42 UTC Make DBTestUniversalCompaction.IncreaseUniversalCompactionNumLevels more deterministic Summary: DBTestUniversalCompaction, IncreaseUniversalCompactionNumLevels fails one in about 30 runs when running in parallel. We wait for compaction after each flush to make the compaction behavior deterministic. Test Plan: Run the test 1000 times in parallel and it still passes. Reviewers: yhchiang, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: kradhakrishnan, yiwu, leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D56841 15 April 2016, 23:16:53 UTC
cea8ed9 sdong 15 April 2016, 21:34:53 UTC Fix backupable_db_test test cases that can't run by itself Summary: Several of backupable_db_test fails if running standalone, because of directory missing. Fix it by: (1) garbage collector skips shared directory if it doesn't exit (2) BackupableDBTest.Issue921Test to create the parent directory of the backup directory fist. Test Plan: Run the tests individually and make sure they pass Subscribers: leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D56829 15 April 2016, 22:48:57 UTC
4b6833a sdong 14 April 2016, 20:56:29 UTC Rename options.compaction_measure_io_stats to options.report_bg_io_stats and include flush too. Summary: It is useful to print out IO stats in flush jobs too. Extend options.compaction_measure_io_stats to flush jobs and raname it. Test Plan: Try db_bench and see the stats are printed out. Reviewers: yhchiang Reviewed By: yhchiang Subscribers: kradhakrishnan, yiwu, IslamAbdelRahman, leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D56769 15 April 2016, 17:22:18 UTC
3894603 sdong 14 April 2016, 02:27:52 UTC Allow valgrind_check to run in parallel Summary: Extend "J=<parallel>" to valgrind_check. For DBTest, modify the script to run valgrind. For other tests, prefix launch command with valgrind. Test Plan: Run valgrind_check with J=1 and J>1 and make sure tests run under valgrind. Manually change codes to introduce memory leak and make sure "make watch-log" correctly report it. Reviewers: yhchiang, yiwu, andrewkr, kradhakrishnan, IslamAbdelRahman Reviewed By: kradhakrishnan, IslamAbdelRahman Subscribers: leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D56727 15 April 2016, 01:33:04 UTC
c9d668c Dmitri Smirnov 15 April 2016, 00:33:53 UTC Fix unit tests issues on Windows (#1078) 15 April 2016, 00:33:53 UTC
083cadc krad 14 April 2016, 22:04:10 UTC Minor fix to Java sandcastle job definition Summary: proxy config tools are not available in sandcastle environment. Hard coding the proxy information in the job. Test Plan: Run manually Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D56775 14 April 2016, 22:05:54 UTC
80b74a6 Sherlock 14 April 2016, 17:50:34 UTC Include ldb tool in the windows build (#914) 14 April 2016, 17:50:34 UTC
7c14d11 krad 14 April 2016, 17:44:15 UTC Minor fix to java build job Summary: Java build is downloading junit when not available. Without the proxy settings exported that would not work. Test Plan: Run manually Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D56751 14 April 2016, 17:46:02 UTC
535af52 sdong 13 April 2016, 20:02:33 UTC BlockBasedTable::PrefixMayMatch() to skip index checking if we can't find a filter block. Summary: In the case where we can't find a filter block, there is not much benefit of doing the binary search and see whether the index key has the prefix. With the change, we blindly return true if we can't get the filter. It also fixes missing row cases for reverse comparator with full bloom. Test Plan: Add a test case that used to fail. Reviewers: yhchiang, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: kradhakrishnan, yiwu, hermanlee4, yoshinorim, leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D56697 14 April 2016, 02:06:48 UTC
09be5ca krad 13 April 2016, 22:56:46 UTC Minor fix to sandcastle java job definition Summary: System was not able to locate javac Test Plan: Run command manually and observe output Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D56709 13 April 2016, 22:59:05 UTC
1aeca97 Islam AbdelRahman 13 April 2016, 22:03:08 UTC Release RocksDB 4.7 Summary: Bump the version and update HISTORY.md Test Plan: none Reviewers: yhchiang, sdong, andrewkr Reviewed By: andrewkr Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D56469 13 April 2016, 22:03:08 UTC
dfc3de8 Islam AbdelRahman 13 April 2016, 21:22:29 UTC Split Travis unittests Job Summary: - Update Makefile check_some command to run a subset of the tests - Update travis config to split the unittests job into 2 jobs -- job testing db_test, db_test2 -- job testing the rest of the unittests Test Plan: Run the new travis.yml on my own branch https://travis-ci.org/facebook/rocksdb/builds/122691453 Reviewers: kradhakrishnan, yhchiang, sdong, andrewkr Reviewed By: andrewkr Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D56673 13 April 2016, 21:22:29 UTC
c2c8fe4 krad 13 April 2016, 18:38:47 UTC Add Java job for sandcastle Summary: Adding java build job definition to sandcastle. Test Plan: ./build_tools/rocksdb-lego-determinator java_build Reviewers: yhchiang, sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D56679 13 April 2016, 18:40:51 UTC
19ef3de Islam AbdelRahman 13 April 2016, 17:38:45 UTC Fix ManualCompactionPartial test flakiness Summary: The reason for this test flakiness is that we try to verify that number of files in L0 is 3 after flushing the 3rd file although we may have a compaction running in the background that may finish before we do the check and the 3 L0 files are converted to 1 L1 file Test Plan: Run a modified version of the test that sleep before doing the check Reviewers: sdong, andrewkr, kradhakrishnan, yhchiang Reviewed By: yhchiang Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D56643 13 April 2016, 17:38:45 UTC
b345b36 Jay Edgar 12 April 2016, 22:09:28 UTC Add a minimum value for the refill bytes per period value Summary: If the user specified a small enough value for the rate limiter's bytes per second, the calculation for the number of refill bytes per period could become zero which would effectively cause the server to hang forever. Test Plan: Existing tests Reviewers: sdong, yhchiang, igor Reviewed By: igor Subscribers: leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D56631 13 April 2016, 16:01:42 UTC
dff4c48 sdong 12 April 2016, 20:56:24 UTC BlockBasedTable::PrefixMayMatch: no need to find data block after full bloom checking Summary: Full block checking should be a good enough indication of prefix existance. No need to further check data block. This also fixes wrong results when using prefix bloom and reverse bitwise comparator. Test Plan: Will add a unit test. Reviewers: yhchiang, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: hermanlee4, yoshinorim, yiwu, kradhakrishnan, leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D56625 12 April 2016, 23:25:54 UTC
0353b85 krad 12 April 2016, 17:37:58 UTC Propagate sandcastle run error to UI Summary: Currently the code does not propagate the sandcastle precommit test run error status to UI. This can confuse the developer when searching for errors. With this change, all success should be in green and all errors should be in red Test Plan: Submit the diff (and hopefully something will fail) Reviewers: andrewkr Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D56595 12 April 2016, 23:18:37 UTC
b885f33 Islam AbdelRahman 12 April 2016, 22:05:56 UTC Parallelize travis jobs Summary: Update travis.yml to split the work into 3 sub jobs - running unittests - Building and testing java - building ROCKSDB_LITE Test Plan: https://travis-ci.org/facebook/rocksdb/builds/122433169 Reviewers: sdong, yhchiang, andrewkr, kradhakrishnan Reviewed By: kradhakrishnan Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D56607 12 April 2016, 22:05:56 UTC
71303e0 Hyunyoung Lee 12 April 2016, 21:05:09 UTC Update db_bench_tool.cc (#1073) * Update db_bench_tool.cc I fixed the wrong letters, LevelDB -> rocksDB, because I thought of LevelDB as the wrong presentation. the following show my fix : fprintf(stderr, "LevelDB: version %d.%d\n", kMajorVersion, kMinorVersion); -----------------> fprintf(stderr, "rocksDB: version %d.%d\n", kMajorVersion, kMinorVersion); * Update db_bench_tool.cc * Update db_bench_tool.cc 12 April 2016, 21:05:09 UTC
63cf15b sdong 11 April 2016, 21:57:27 UTC Fix option settable tests Summary: In option settable tests, bytes for pointers are not all skipped, so that they may be the same as the special character and cause false positive. Test Plan: Run the test. Manually verify the issue is not there any more. Reviewers: IslamAbdelRahman, andrewkr Reviewed By: IslamAbdelRahman Subscribers: kradhakrishnan, yiwu, yhchiang, leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D56553 12 April 2016, 21:03:35 UTC
e208575 Shu Zhang 12 April 2016, 04:08:26 UTC using java7 in runtime for hdfs env (#1072) 12 April 2016, 04:08:26 UTC
back to top