| d788d9a | Islam AbdelRahman | 27 January 2016, 03:07:19 UTC | Release 4.3.1 | 27 January 2016, 03:07:19 UTC |
| 8a294d6 | Islam AbdelRahman | 07 January 2016, 17:48:29 UTC | Fix BlockBasedTableTest.NoopTransformSeek failure Summary: table_test is failing because we are creating a temp InternalComparator 14:27:28 [ RUN ] BlockBasedTableTest.NoopTransformSeek 14:27:28 pure virtual method called 14:27:28 terminate called without an active exception 14:27:28 /bin/sh: line 7: 2346261 Aborted (core dumped) ./$t Test Plan: make table_test -j64 && ./table_test --gtest_filter="BlockBasedTableTest.NoopTransformSeek" Reviewers: igor, sdong, anthony, rven Reviewed By: rven Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D52671 | 27 January 2016, 02:51:31 UTC |
| dde3cdc | Islam AbdelRahman | 26 January 2016, 22:47:42 UTC | Fix issue in Iterator::Seek when using Block based filter block with prefix_extractor Summary: Similar to D53385 we need to check InDomain before checking the filter block. Test Plan: unit tests Reviewers: yhchiang, rven, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D53421 | 27 January 2016, 02:45:03 UTC |
| 3c3020c | Islam AbdelRahman | 26 January 2016, 19:07:08 UTC | Fix bug in block based tables with full filter block and prefix_extractor Summary: Right now when we are creating a BlockBasedTable with fill filter block we add to the filter all the prefixes that are InDomain() based on the prefix_extractor the problem is that when we read a key from the file, we check the filter block for the prefix whether or not it's InDomain() Test Plan: unit tests Reviewers: yhchiang, rven, anthony, kradhakrishnan, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D53385 | 27 January 2016, 02:44:56 UTC |
| 2d8fa53 | Peter Mattis | 06 January 2016, 19:46:32 UTC | Fix index seeking in BlockTableReader::PrefixMayMatch. PrefixMayMatch previously seeked in the prefix index using an internal key with a sequence number of 0. This would cause the prefix index seek to fall off the end if the last key in the index had a user-key greater than or equal to the key being looked for. Falling off the end of the index in turn results in PrefixMayMatch returning false if the index is in memory. | 27 January 2016, 02:44:47 UTC |
| eb965e1 | sdong | 23 December 2015, 17:40:37 UTC | Revert "db_bench: --soft_pending_compaction_bytes_limit should set options.soft_pending_compaction_bytes_limit" This reverts commit c726b0fdc1dd9ed178f0fb740d30f72516b7106a. Bad backporting | 23 December 2015, 17:40:37 UTC |
| c726b0f | sdong | 18 December 2015, 02:10:48 UTC | db_bench: --soft_pending_compaction_bytes_limit should set options.soft_pending_compaction_bytes_limit Summary: Fix a bug that options.soft_pending_compaction_bytes_limit is not actually set with --soft_pending_compaction_bytes_limit Test Plan: Run db_bench with this parameter and make sure the parameter is set correctly. Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D52125 | 18 December 2015, 02:29:23 UTC |
| e15400d | Yueh-Hsuan Chiang | 10 December 2015, 19:53:53 UTC | Fixed the valgrind error in ColumnFamilyTest::CreateAndDropRace Summary: Fixed the valgrind error in ColumnFamilyTest::CreateAndDropRace Test Plan: valgrind --error-exitcode=2 --leak-check=full ./column_family_test Reviewers: kradhakrishnan, rven, anthony, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D51795 | 11 December 2015, 22:14:08 UTC |
| 0642d46 | krad | 09 December 2015, 01:53:20 UTC | Revert "Support marking snapshots for write-conflict checking - Take 2" This reverts commit e5c5f23814a995797d64fa3c2bc65dd0a1d27aaa. | 09 December 2015, 01:53:20 UTC |
| 188170f | krad | 09 December 2015, 01:45:25 UTC | Updating HISTORY.md Summary: Added 4.3.0 version Test Plan: Reviewers: CC: leveldb@ Task ID: #9298965 Blame Rev: | 09 December 2015, 01:46:43 UTC |
| 758dbec | sdong | 09 December 2015, 01:36:59 UTC | Fix fb-only build for gcc 4.8.1 Summary: fb build for gcc 4.8.1 is broken for the ZSTD dependency is not fixed after ea11923550802a3eeebe81f2cc4a9a44cda32121. Fixing it. Test Plan: ROCKSDB_FBCODE_BUILD_WITH_481=1 make -j40 OPT=-g Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D51741 | 09 December 2015, 01:40:59 UTC |
| 774b80e | Yueh-Hsuan Chiang | 09 December 2015, 01:01:02 UTC | Resubmit the fix for a race condition in persisting options Summary: This patch fix a race condition in persisting options which will cause a crash when: * Thread A obtain cf options and start to persist options based on that cf options. * Thread B kicks in and finish DropColumnFamily and delete cf_handle. * Thread A wakes up and tries to finish the persisting options and crashes. Test Plan: Add a test in column_family_test that can reproduce the crash Reviewers: anthony, IslamAbdelRahman, rven, kradhakrishnan, sdong Reviewed By: sdong Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D51717 | 09 December 2015, 01:01:02 UTC |
| afc8473 | sdong | 09 December 2015, 00:39:10 UTC | Include ldb_tools and sst_dump_tools libraries in shared library Summary: Ldb and sst_dump are not included in shared library now. Add it. Test Plan: Build make release make shared_lib Reviewers: igor, kradhakrishnan, rven, yhchiang, IslamAbdelRahman, anthony Reviewed By: IslamAbdelRahman, anthony Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D51735 | 09 December 2015, 00:59:53 UTC |
| e5c5f23 | agiardullo | 08 December 2015, 20:25:48 UTC | Support marking snapshots for write-conflict checking - Take 2 Summary: D51183 was reverted due to breaking the LITE build. This diff is the same as D51183 but with a fix for the LITE BUILD(D51693) Test Plan: run all unit tests Reviewers: sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D51711 | 09 December 2015, 00:47:31 UTC |
| ea11923 | sdong | 09 December 2015, 00:33:10 UTC | Upgrade to ZSTD 0.4.2 Summary: Change to call the new compression function. Test Plan: build and run db_bench with the compression to make sure it compresses. Reviewers: anthony, rven, kradhakrishnan, IslamAbdelRahman, igor, yhchiang Reviewed By: yhchiang Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D51603 | 09 December 2015, 00:33:26 UTC |
| b60cb88 | Yueh-Hsuan Chiang | 09 December 2015, 00:17:06 UTC | Update examples/rocksdb_option_file_example.ini Summary: Update examples/rocksdb_option_file_example.ini to use the options file generated by the db_bench readwhilewriting benchmark. Test Plan: no code change. Reviewers: igor, IslamAbdelRahman, rven, kradhakrishnan, sdong, anthony Reviewed By: anthony Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D51567 | 09 December 2015, 00:17:06 UTC |
| 3d8bb2c | Venkatesh Radhakrishnan | 08 December 2015, 19:45:29 UTC | Fix valgrind failure in IncreaseUniversalCompactionNumLevels Summary: Fixing a valgrind failure in DBTestUniversalCompaction in the IncreaseUniversalCompactionNumLevels test. Using SpecialSkipList with 10 rows per file. Test Plan: Run valgrind and functional tests. Reviewers: anthony, yhchiang, kradhakrishnan, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D51705 | 08 December 2015, 19:45:29 UTC |
| 7af91d4 | Igor Canadi | 08 December 2015, 17:28:02 UTC | Merge pull request #873 from yuslepukhin/make_vs15_build Fix up VS 15 build. | 08 December 2015, 17:28:02 UTC |
| 1d63c3d | sdong | 08 December 2015, 17:27:17 UTC | Revert "Support marking snapshots for write-conflict checking" This reverts commit ec704aafdcfa997330e9c92736d15e17025e3399 for it broke RocksDB LITE build. | 08 December 2015, 17:27:17 UTC |
| 78de0c9 | yuslepukhin | 08 December 2015, 16:38:21 UTC | Fix up VS 15 build. Fix warnings Take advantage of native snprintf on VS 15 | 08 December 2015, 16:38:21 UTC |
| ec704aa | agiardullo | 21 November 2015, 02:23:25 UTC | Support marking snapshots for write-conflict checking Summary: D50475 enables using SST files for transaction write-conflict checking. In order for this to work, we need to make sure not to compact out SingleDeletes when there is an earlier transaction snapshot(D50295). If there is a long-held snapshot, this could reduce the benefit of the SingleDelete optimization. This diff allows Transactions to mark snapshots as being used for write-conflict checking. Then, during compaction, we will be able to optimize SingleDeletes better in the future. This diff adds a flag to SnapshotImpl which is used by Transactions. This diff also passes the earliest write-conflict snapshot's sequence number to CompactionIterator. This diff does not actually change Compaction (after this diff is pushed, D50295 will be able to use this information). Test Plan: no behavior change, ran existing tests Reviewers: rven, kradhakrishnan, yhchiang, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D51183 | 08 December 2015, 03:40:51 UTC |
| 770dea9 | sdong | 08 December 2015, 02:07:01 UTC | Fix occasional failure of DBTest.DynamicCompactionOptions Summary: DBTest.DynamicCompactionOptions ocasionally fails during valgrind run. We sent a sleeping task to block compaction thread pool but we don't wait it to run. Test Plan: Run the test multiple times in an environment which can cause failure. Reviewers: rven, kradhakrishnan, igor, IslamAbdelRahman, anthony, yhchiang Reviewed By: yhchiang Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D51687 | 08 December 2015, 02:38:39 UTC |
| f307036 | sdong | 08 December 2015, 01:09:12 UTC | Revert "Fix a race condition in persisting options" This reverts commit 2fa3ed5180340e485a1caf6fa71cc400ea599278. It breaks RocksDB lite build | 08 December 2015, 01:09:12 UTC |
| 2fa3ed5 | Yueh-Hsuan Chiang | 07 December 2015, 23:25:12 UTC | Fix a race condition in persisting options Summary: This patch fix a race condition in persisting options which will cause a crash when: * Thread A obtain cf options and start to persist options based on that cf options. * Thread B kicks in and finish DropColumnFamily and delete cf_handle. * Thread A wakes up and tries to finish the persisting options and crashes. Test Plan: Add a test in column_family_test that can reproduce the crash Reviewers: anthony, IslamAbdelRahman, rven, kradhakrishnan, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D51609 | 07 December 2015, 23:25:12 UTC |
| f276c3a | Venkatesh Radhakrishnan | 07 December 2015, 19:57:00 UTC | Fix valgrind failures in 3 tests in db_compaction_test due to new skiplist changes Summary: Several tests in db_compaction_test are failing with aborts in valgrind. These are LevelCompactionThirdPath, LevelCompactionPathUse and CompressLevelCompaction. We now use the SpecialSkipListFactory to make them more deterministic Test Plan: valgrind Reviewers: anthony, yhchiang, kradhakrishnan, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D51663 | 07 December 2015, 19:57:00 UTC |
| 291088a | sdong | 07 December 2015, 18:13:58 UTC | Fix undeterministic failure of ColumnFamilyTest.DifferentWriteBufferSizes Summary: After the skip list optimization, ColumnFamilyTest.DifferentWriteBufferSizes can occasionally fail with flush triggering of column family 3. Insert more data to it to make sure flush will trigger. Test Plan: Run it multiple times with both of jemaloc on and off and see it always passes. (Without thd commit the run with jemalloc fails with chance of about one in two) Reviewers: rven, yhchiang, IslamAbdelRahman, anthony, kradhakrishnan, igor Reviewed By: igor Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D51645 | 07 December 2015, 18:53:29 UTC |
| a9ca910 | Islam AbdelRahman | 04 December 2015, 21:27:56 UTC | Fix db_universal_compaction_test Summary: db_universal_compaction_test is still failing because of UniversalCompactionNumLevels/DBTestUniversalCompaction.UniversalCompactionSecondPathRatio/0 https://travis-ci.org/facebook/rocksdb/jobs/94949919 Use same approach to fix other tests to fix this test Test Plan: Run ./db_universal_compaction_test on mac and make sure all the tests pass Reviewers: kradhakrishnan, yhchiang, rven, anthony, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D51591 | 04 December 2015, 21:27:56 UTC |
| d3bb572 | krad | 04 December 2015, 03:05:13 UTC | Build break fix. Summary: Skip list now cannot estimate memory across allocators consistently and hence triggers flush at different time. This breaks certain unit tests. The fix is to adopt key count instead of size for flush. Test Plan: Ran test on dev box and mac (where it used to fail) Reviewers: sdong CC: leveldb@ Task ID: #9273334 Blame Rev: | 04 December 2015, 19:45:51 UTC |
| b28b7c6 | Jay Edgar | 04 December 2015, 18:12:27 UTC | Added callback notification when a snapshot is created Summary: When SetSnapshot() is used the caller immediately knows a snapshot has been created, but when SetSnapshotOnNextOperation() is used the caller needs a way to get notified when that snapshot has been generated. This creates an interface that the client can implement that will be called at the time the snapshot is created. Test Plan: Added a new SetSnapshotOnNextOperationWithNotification test into the transaction_test. Reviewers: sdong, anthony Reviewed By: anthony Subscribers: yoshinorim, leveldb, dhruba Differential Revision: https://reviews.facebook.net/D51177 | 04 December 2015, 18:20:36 UTC |
| e8180f9 | Alex Yang | 20 November 2015, 09:07:24 UTC | added public api to schedule flush/compaction, code to prevent race with db::open Summary: Fixes T8781168. Added a new function EnableAutoCompactions in db.h to be publicly avialable. This allows compaction to be re-enabled after disabling it via SetOptions Refactored code to set the dbptr earlier on in TransactionDB::Open and DB::Open Temporarily disable auto_compaction in TransactionDB::Open until dbptr is set to prevent race condition. Test Plan: Ran make all check verified fix on myrocks side: was able to reproduce the seg fault with ../tools/mysqltest.sh --mem --force rocksdb.drop_table method was to manually sleep the thread after DB::Open but before TransactionDB ptr was assigned in transaction_db_impl.cc: DB::Open(db_options, dbname, column_families_copy, handles, &db); clock_t goal = (60000 * 10) + clock(); while (goal > clock()); ...dbptr(aka rdb) gets assigned below verified my changes fixed the issue. Also added unit test 'ToggleAutoCompaction' in transaction_test.cc Reviewers: hermanlee4, anthony Reviewed By: anthony Subscribers: alex, dhruba Differential Revision: https://reviews.facebook.net/D51147 | 04 December 2015, 06:59:44 UTC |
| 19b1201 | Islam AbdelRahman | 04 December 2015, 01:32:20 UTC | Merge pull request #865 from yuslepukhin/fix_db_table_properties_test Avoid empty ranges vector with subsequent zero element access | 04 December 2015, 01:32:20 UTC |
| e0de7ef | yuslepukhin | 02 December 2015, 22:50:33 UTC | Avoid empty ranges vector with subsequent zero element access | 02 December 2015, 22:50:33 UTC |
| a330f0b | Yueh-Hsuan Chiang | 02 December 2015, 22:08:12 UTC | Fix incorrect merge in db/db_compaction_test.cc Summary: Fix incorrect merge in db/db_compaction_test.cc Test Plan: db_compaction_test Reviewers: igor, sdong, anthony, IslamAbdelRahman, rven, kradhakrishnan Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D51531 | 02 December 2015, 22:09:09 UTC |
| bd7a49d | Yueh-Hsuan Chiang | 02 December 2015, 22:06:33 UTC | Make DBCompactionTestWithParam::CompactionTrigger more deterministic Summary: Make DBCompactionTestWithParam::CompactionTrigger more deterministic Test Plan: ./db_compaction_test Reviewers: anthony, IslamAbdelRahman, rven, kradhakrishnan, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D51507 | 02 December 2015, 22:06:33 UTC |
| bcd7bd1 | sdong | 02 December 2015, 02:25:50 UTC | Relax verification condition of DBTest.SuggestCompactRangeTest Summary: Verifiction condition of DBTest.SuggestCompactRangeTest is too strict. Based on key distribution, we might have more small files in last level. Not check number of files in the last level. Test Plan: Run DBTest.SuggestCompactRangeTest with both of jemalloc on and off. Reviewers: rven, IslamAbdelRahman, yhchiang, kradhakrishnan, igor, anthony Reviewed By: anthony Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D51501 | 02 December 2015, 05:12:24 UTC |
| f9103d9 | sdong | 25 November 2015, 23:43:11 UTC | DBTest.DynamicCompactionOptions: More deterministic and readable Summary: DBTest.DynamicCompactionOptions sometimes fails the assert but I can't repro it locally. Make it more deterministic and readable and see whether the problem is still there. Test Plan: Run tht test and make sure it passes Reviewers: kradhakrishnan, yhchiang, igor, rven, IslamAbdelRahman, anthony Reviewed By: anthony Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D51309 | 02 December 2015, 00:49:47 UTC |
| 0ad6851 | sdong | 01 December 2015, 06:16:49 UTC | Fix DBCompactionTestWithParam.CompactionTrigger in non-jemalloc build. Summary: DBCompactionTestWithParam.CompactionTrigger fails in non-jemalloc build, after the skip list memtable change. Fix it by making mem table flush trigger by number of entries. Test Plan: Run the test using both of jemalloc and non-jemalloc build. Reviewers: anthony, IslamAbdelRahman, rven, kradhakrishnan, igor, yhchiang Reviewed By: yhchiang Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D51471 | 01 December 2015, 20:25:22 UTC |
| 459c7fb | sdong | 01 December 2015, 05:32:59 UTC | Revert previous behavior of internal_key_skipped_count Summary: With recent commit 33e0c93826fd45640b24324a642ca03d8b9ef123, db iterator skips perf context counter internal_key_skipped_count when blindly issuing internal Next(). Now increment the counter by one when issuing this Next() Test Plan: Run all existing tests Reviewers: rven, yhchiang, IslamAbdelRahman, kradhakrishnan, igor, anthony Reviewed By: anthony Subscribers: yoshinorim, leveldb, dhruba Differential Revision: https://reviews.facebook.net/D51465 | 01 December 2015, 05:55:05 UTC |
| 481f9ed | agiardullo | 01 December 2015, 02:06:06 UTC | Fix CLANG build Summary: fix clang build Test Plan: build Reviewers: IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D51453 | 01 December 2015, 04:02:13 UTC |
| d7421c2 | Yueh-Hsuan Chiang | 01 December 2015, 02:10:00 UTC | Fixed some typos in the comments of rocksdb options file example Summary: Fixed some typos in the comments of rocksdb options file example Test Plan: No code change. Reviewers: igor, anthony, IslamAbdelRahman, rven, kradhakrishnan, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D51393 | 01 December 2015, 02:10:00 UTC |
| ef8ed36 | sdong | 30 November 2015, 19:41:53 UTC | Fix DBTest.SuggestCompactRangeTest for disable jemalloc case Summary: DBTest.SuggestCompactRangeTest fails for the case when jemalloc is disabled, including ASAN and valgrind builds. It is caused by the improvement of skip list, which allocates different size of nodes for a new records. Fix it by using a special mem table that triggers a flush by number of entries. In that way the behavior will be consistent for all allocators. Test Plan: Run the test with both of DISABLE_JEMALLOC=1 and 0 Reviewers: anthony, rven, yhchiang, kradhakrishnan, igor, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D51423 | 01 December 2015, 00:40:47 UTC |
| db320b1 | sdong | 24 November 2015, 23:53:42 UTC | DB to only flush the column family with the largest memtable while option.db_write_buffer_size is hit Summary: When option.db_write_buffer_size is hit, we currently flush all column families. Move to flush the column family with the largest active memt table instead. In this way, we can avoid too many small files in some cases. Test Plan: Modify test DBTest.SharedWriteBuffer to work with the updated behavior Reviewers: kradhakrishnan, yhchiang, rven, anthony, IslamAbdelRahman, igor Reviewed By: igor Subscribers: march, leveldb, dhruba Differential Revision: https://reviews.facebook.net/D51291 | 30 November 2015, 21:36:57 UTC |
| 4a009f9 | Siying Dong | 30 November 2015, 19:18:09 UTC | Merge pull request #860 from SherlockNoMad/BuildFix Fix ms version Appveyor build error | 30 November 2015, 19:18:09 UTC |
| b4efaeb | SherlockNoMad | 30 November 2015, 19:07:47 UTC | Fix ms version Appveyor build error | 30 November 2015, 19:07:47 UTC |
| d27ea4c | sdong | 30 November 2015, 18:30:22 UTC | Initialize options.row_cache Summary: options.row_cache should already been initialized as null by default. Still try to set it following current convention, because one valgrind failure reports a failure related to it. Test Plan: Run all unit tests Reviewers: yhchiang, kradhakrishnan, igor Reviewed By: igor Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D51303 | 30 November 2015, 18:30:35 UTC |
| 7293048 | Islam AbdelRahman | 30 November 2015, 18:03:07 UTC | Fix clang build Summary: Fix clang Test Plan: make check Reviewers: sdong, yhchiang, rven Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D51417 | 30 November 2015, 18:03:07 UTC |
| 6bbfa18 | sdong | 24 November 2015, 00:05:09 UTC | BackupDB to have a mode to use file size in file name Summary: Getting file size from all the backup files can take a long time. In some cases, the sizes are available in file names. We allow a mode to get those sizes from file name. Test Plan: Make some unit tests in backupable_db_test to run in such a mode. Make sure RocksDB Lite builds too. Reviewers: IslamAbdelRahman, rven, yhchiang, kradhakrishnan, anthony, igor Reviewed By: igor Subscribers: muthu, asameet, leveldb, dhruba Differential Revision: https://reviews.facebook.net/D51243 | 25 November 2015, 19:55:37 UTC |
| f3ea00b | Igor Canadi | 25 November 2015, 19:38:09 UTC | Merge pull request #856 from ceph/wip-env EnvWrapper: add ReuseWritableFile | 25 November 2015, 19:38:09 UTC |
| 4cedd6b | Sage Weil | 25 November 2015, 19:30:05 UTC | EnvWrapper: add ReuseWritableFile This was missed when ReuseWritableFile was added to Env in 1bcafb62f4aa7c0b2b3786a958220411347f27b8. Signed-off-by: Sage Weil <sage@redhat.com> | 25 November 2015, 19:30:05 UTC |
| 33e0c93 | sdong | 07 October 2015, 00:48:27 UTC | Reduce extra key comparision in DBIter::Next() Summary: Now DBIter::Next() always compares with current key with itself first, which is unnecessary if the last key is not a merge key. I made the change and didn't see db_iter_test fails. Want to hear whether people have any idea what I miss. Test Plan: Run all unit tests Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D48279 | 25 November 2015, 01:16:18 UTC |
| 9a9d475 | Nathan Bronson | 19 November 2015, 22:24:29 UTC | InlineSkipList part 3/3 - new skiplist type that colocates key and node Summary: This diff completes the creation of InlineSkipList<Cmp>, which is like SkipList<const char*, Cmp> but it always allocates the key contiguously with the node. This allows us to remove the pointer from the node to the key. As a result the memory usage of the skip list is reduced (by 1 to sizeof(void*) bytes depending on the padding required to align the key storage), cache locality is improved, and we halve the number of calls to the allocator. For skip lists whose keys are freshly-allocated const char*, InlineSkipList is stricly preferrable to SkipList. This diff doesn't replace SkipList, however, because some of the use cases of SkipList in RocksDB are either character sequences that are not allocated at the same time as the skip list node allocation (for example hash_linklist_rep) or have different key types (for example write_batch_with_index). Taking advantage of inline allocation for those cases is left to future work. The perf win is biggest for small values. For single-threaded CPU-bound (32M fillrandom operations with no WAL log) with 16 byte keys and 0 byte values, the db_bench perf goes from ~310k ops/sec to ~410k ops/sec. For large values the improvement is less pronounced, but seems to be between 5% and 10% on the same configuration. Test Plan: make check Reviewers: igor, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D51123 | 24 November 2015, 23:16:02 UTC |
| 5201729 | Nathan Bronson | 24 November 2015, 21:29:50 UTC | InlineSkipList - part 2/3 Summary: This diff is 2/3 in a sequence that introduces a skip list optimized for a key that is a freshly-allocated const char*. The change is broken into pieces to make it easier to review. This piece removes the Key template type, introduces the AllocateKey interface, and changes the unit test from using uint64_t as the Key type to using pointers to an 8 byte blob. Test Plan: unit test Reviewers: igor, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D51285 | 24 November 2015, 22:30:56 UTC |
| 78812ec | Nathan Bronson | 24 November 2015, 21:01:09 UTC | InlineSkipList - part 1/3 Summary: This diff is 1/3 in a sequence that introduces a skip list optimized for a key that is a freshly-allocated const char*. The diff is broken into pieces to make it easier to review. This piece only introduces the new type by copying the existing SkipList, with mechanical naming changes and reformatting. Test Plan: new unit test Reviewers: igor, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D51279 | 24 November 2015, 22:30:22 UTC |
| ffb466d | Igor Canadi | 24 November 2015, 20:07:12 UTC | Merge pull request #855 from yuslepukhin/enable_3rdparty_override Enable override to 3rd party linkage | 24 November 2015, 20:07:12 UTC |
| 10d257d | Dmitri Smirnov | 24 November 2015, 19:51:37 UTC | Enable override to 3rd party linkage | 24 November 2015, 19:51:37 UTC |
| c5b4673 | agiardullo | 20 November 2015, 02:01:46 UTC | Fix race condition that causes valgrind failures Summary: DBTest.DynamicLevelCompressionPerLevel2 sometimes fails during valgrind runs. This causes our valgrind tests to fail. Not sure what the best fix is for this test, but hopefully this simple change is sufficient. Test Plan: run test Reviewers: sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D51111 | 21 November 2015, 02:26:48 UTC |
| efb01a0 | Siying Dong | 21 November 2015, 01:57:22 UTC | Merge pull request #850 from yuslepukhin/enable_2015_build Build on Visual Studio 2015 Update 1 | 21 November 2015, 01:57:22 UTC |
| 81be49c | Venkatesh Radhakrishnan | 20 November 2015, 23:57:26 UTC | Have a way for compaction filter to ignore snapshots Summary: Provide an API for compaction filter to specify that it needs to be applied even if there are snapshots. Test Plan: DBTestCompactionFilter.CompactionFilterIgnoreSnapshot Reviewers: yhchiang, IslamAbdelRahman, sdong, anthony Reviewed By: anthony Subscribers: yoshinorim, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D51087 | 20 November 2015, 23:57:26 UTC |
| 047bd22 | yuslepukhin | 20 November 2015, 23:31:47 UTC | Build on Visual Studio 2015 Update 1 | 20 November 2015, 23:31:47 UTC |
| 88e0527 | Islam AbdelRahman | 20 November 2015, 06:26:37 UTC | Reduce moving memory in LDB::ScanCommand Summary: Based on https://github.com/facebook/rocksdb/issues/843 It looks that when the data is hot we spend significant amount of time moving data out of RocksDB blocks. This patch reduce moving memory when possible Original performance ``` $ time ./ldb --db=/home/tec/local/ellina_test/testdb scan > /dev/null real 0m16.736s user 0m11.993s sys 0m4.725s ``` Performance after reducing memcpy ``` $ time ./ldb --db=/home/tec/local/ellina_test/testdb scan > /dev/null real 0m11.590s user 0m6.983s sys 0m4.595s ``` Test Plan: dump the output of the scan into 2 files and verifying the are exactly the same make check Reviewers: sdong, yhchiang, anthony, rven, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D51093 | 20 November 2015, 06:26:37 UTC |
| 890f44f | Siying Dong | 19 November 2015, 02:36:30 UTC | Merge pull request #844 from yuslepukhin/enable_C4804_unsafe_bool Enable MS Warning C4804 : unsafe use of type 'bool' in operation | 19 November 2015, 02:36:30 UTC |
| 9d0b8f1 | sdong | 18 November 2015, 02:29:40 UTC | plain table reader: avoid re-read the same position for index and data in non-mmap mode Summary: In non-mmap mode, plain table reader can issue two pread() for index checking and reading the actual data, although it's for the same location. By reusing the key decoder, we reuse the buffer used for the two to avoid it. Test Plan: Run unit tests. Run table_reader_bench and see from strace the repeat read cases to disappear. Reviewers: anthony, yhchiang, rven, kradhakrishnan, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D50949 | 19 November 2015, 00:49:08 UTC |
| 89bacb7 | Dmitri Smirnov | 19 November 2015, 00:23:19 UTC | Enable MS Warning C4804 : unsafe use of type 'bool' in operation | 19 November 2015, 00:23:19 UTC |
| d5239f8 | sdong | 19 November 2015, 00:00:36 UTC | build_tools/fbcode_config4.8.1.sh: upgrade versions of some dependencies Summary: Upgrade version of some dependencies in build_tools/fbcode_config4.8.1.sh. I didn't upgrade version for CLANG because some warnings are shown. Test Plan: build: ROCKSDB_FBCODE_BUILD_WITH_481=1 USE_CLANG=1 make all -j32 as well as ROCKSDB_FBCODE_BUILD_WITH_481=1 make all -j32 Reviewers: anthony, IslamAbdelRahman, kradhakrishnan, yhchiang, igor Reviewed By: igor Subscribers: igor, kradhakrishnan, leveldb, dhruba Differential Revision: https://reviews.facebook.net/D51003 | 19 November 2015, 00:18:11 UTC |
| c4ebb66 | sdong | 18 November 2015, 23:24:28 UTC | Not to build forward_iterator_bench now Summary: forward_iterator_bench is not stable enough for build. Remove it for now. Test Plan: Build it with both of CLANG and non-CLANG and make sure it builds. Reviewers: rven, kradhakrishnan, anthony, yhchiang, igor, IslamAbdelRahman Reviewed By: igor, IslamAbdelRahman Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D50991 | 18 November 2015, 23:42:06 UTC |
| c342549 | Siying Dong | 18 November 2015, 06:10:54 UTC | Merge pull request #841 from yuslepukhin/fix_test_rerun_logic Fix log names when scheduling runs and reruns | 18 November 2015, 06:10:54 UTC |
| 7cb1293 | Dmitri Smirnov | 18 November 2015, 02:41:29 UTC | Fix log names when scheduling runs and reruns | 18 November 2015, 02:41:29 UTC |
| 51fce92 | sdong | 17 November 2015, 22:45:26 UTC | "ldb compact" should force bottommost level compaction Summary: Now "ldb compact" skips the bottommost level compaction. This is an unintended behavior change. Reverting it now. Maybe we need to add another mode later for it. Test Plan: Run a manual test of 'ldb' to make sure bottom most level is compacted. Reviewers: IslamAbdelRahman, yhchiang, anthony, kradhakrishnan, rven Reviewed By: rven Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D50925 | 18 November 2015, 02:07:11 UTC |
| f831641 | Siying Dong | 18 November 2015, 01:43:38 UTC | Merge pull request #837 from yuslepukhin/rerun_concurrency_value Failed tests must be rerun with concurrency 1 | 18 November 2015, 01:43:38 UTC |
| 4159ab8 | Islam AbdelRahman | 18 November 2015, 01:09:49 UTC | Merge pull request #839 from SherlockNoMad/memtableOption Support Memtable Factory Parse in option_helper.cc | 18 November 2015, 01:09:49 UTC |
| 6170fec | sdong | 18 November 2015, 00:41:54 UTC | Fix build broken by previous commit of "option helper refactor" Summary: The commit of option helper refactor broken the build: (1) a git merge problem (2) some uncaught compiler warning Fix it. Test Plan: Make sure "make all" passes Reviewers: anthony, IslamAbdelRahman, rven, kradhakrishnan, yhchiang Reviewed By: yhchiang Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D50943 | 18 November 2015, 00:52:54 UTC |
| 3a6643c | Siying Dong | 18 November 2015, 00:24:52 UTC | Merge pull request #805 from SherlockNoMad/OptionHelperFix Option Helper Refactoring | 18 November 2015, 00:24:52 UTC |
| 189b3e0 | sdong | 18 November 2015, 00:16:29 UTC | Fix uninitilizeded SpecialEnv::time_elapse_only_sleep_ Summary: SpecialEnv::time_elapse_only_sleep_ is not initialized, which might cause some test failures. Fix it. Test Plan: Run some unit tests. Since tests already broken. Might want to commit it sooner. Reviewers: IslamAbdelRahman, yhchiang, anthony Reviewed By: anthony Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D50937 | 18 November 2015, 00:22:17 UTC |
| d5540e1 | sdong | 17 November 2015, 19:06:15 UTC | DBTest.MergeTestTime to only use fake time to be determinstic Summary: DBTest.MergeTestTime is a test verifying timing counters. Depending on real time may cause non-determinstic results. Change to fake time to be determinsitic. Test Plan: Run the test and make sure it passes Reviewers: yhchiang, anthony, rven, kradhakrishnan, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D50883 | 17 November 2015, 22:40:23 UTC |
| bd7be03 | SherlockNoMad | 17 November 2015, 22:29:01 UTC | Support Memtable Factory Parse in option_helper.cc | 17 November 2015, 22:29:01 UTC |
| 94e39e2 | Dmitri Smirnov | 17 November 2015, 21:54:13 UTC | Exclude DBTest.FileCreationRandomFailure as a long running test Increase concurrency to 18 Fix exclusion but in the ps script | 17 November 2015, 21:54:13 UTC |
| 4189c0f | Yueh-Hsuan Chiang | 17 November 2015, 21:50:54 UTC | Fix Java Makefile Summary: In case rocksdb java package is built using make rocksdbjavastaticrelease, then only those rocksdb binary built under the virtual environments is release build. This patch fix this issue. Test Plan: PORTABLE=1 V=2 make rocksdbjavastaticrelease -j32 and make sure -O2 and -NDEBUG is included when compiling all source files. Reviewers: sdong, anthony, IslamAbdelRahman, rven, kradhakrishnan, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D50895 | 17 November 2015, 21:50:54 UTC |
| 605a24d | Islam AbdelRahman | 17 November 2015, 19:51:37 UTC | Block forward_iterator_bench under MAC and Windows Summary: Travis is now failing because we cannot compile forward_iterator_bench under MAC https://travis-ci.org/facebook/rocksdb/jobs/91524025 In forward_iterator_bench.cc we are using multiple functions that are not available in MAC like htobe64 be64toh Blocking forward_iterator_bench under MAC Test Plan: compile under mac Reviewers: rven, yhchiang, anthony, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D50889 | 17 November 2015, 19:51:37 UTC |
| 2a0510c | Dmitri Smirnov | 17 November 2015, 19:23:09 UTC | Failed tests must be rerun with concurrency 1 | 17 November 2015, 19:23:09 UTC |
| 9b8c9be | Venkatesh Radhakrishnan | 17 November 2015, 18:27:51 UTC | Fix forward_iterator allocation of vector. Summary: db_tailing_iter_test was failing on some platforms because of an incorrect allocation and use. This diff fixes the issue. Test Plan: db_tailing_iter_test Run valgrind for db_tailing_iter_test Reviewers: igor, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D50835 | 17 November 2015, 18:27:51 UTC |
| 5cbb7e4 | sdong | 17 November 2015, 02:21:59 UTC | DBTest.MergeTestTime: relax counter upper bound verification Summary: Timing counters' upper bounds depend on platform. It frequently fails in valgrind runs. Relax the upper bound. Test Plan: Run the same valgrind test and make sure it passes. Reviewers: rven, anthony, kradhakrishnan, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D50829 | 17 November 2015, 03:47:07 UTC |
| 52e04b3 | Islam AbdelRahman | 17 November 2015, 02:06:43 UTC | Merge pull request #833 from yuslepukhin/fix_win_build_after_lint Add necessary headers after cpplint rearranged includes | 17 November 2015, 02:06:43 UTC |
| 314f621 | Dmitri Smirnov | 16 November 2015, 23:08:11 UTC | Remove headers from the cc since they are in the module's header. | 16 November 2015, 23:08:11 UTC |
| 472c740 | Dmitri Smirnov | 16 November 2015, 22:41:11 UTC | Add necessary headers after cpplint rearranged includes | 16 November 2015, 22:41:11 UTC |
| 9bc9c93 | sdong | 16 November 2015, 22:15:50 UTC | Move to version 4.3 Summary: RocksDB 4.2 is already cut. Move to 4.3 Test Plan: Not needed Reviewers: IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D50799 | 16 November 2015, 22:29:08 UTC |
| 3381e2c | Reid Horuff | 16 November 2015, 20:34:55 UTC | Handle multiple calls to DBImpl::PauseBackgroundWork() and DBImpl::ContinueBackgroundWork() Summary: Handle multiple calls to DBImpl::PauseBackgroundWork() and DBImpl::ContinueBackgroundWork() Test Plan: rocksdb.information_schema handles this case. Reviewers: igor Reviewed By: igor Subscribers: hermanlee4, jkedgar, dhruba Differential Revision: https://reviews.facebook.net/D50781 | 16 November 2015, 22:20:18 UTC |
| 65a0429 | Siying Dong | 16 November 2015, 22:15:35 UTC | Merge pull request #831 from yuslepukhin/remove_forward_iter_bench_win Remove forward_iter_bench from Win build. | 16 November 2015, 22:15:35 UTC |
| ca5566d | Islam AbdelRahman | 16 November 2015, 22:14:39 UTC | Fix clang build Summary: Fix clang Test Plan: USE_CLANG=1 make all -j64 Reviewers: sdong, yhchiang, anthony, rven Reviewed By: rven Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D50793 | 16 November 2015, 22:14:39 UTC |
| 4175472 | Igor Canadi | 16 November 2015, 22:07:17 UTC | Merge pull request #832 from yuslepukhin/fix_forward_iter_outofbounds Fix empty vector write in ForwardIterator | 16 November 2015, 22:07:17 UTC |
| cb9459f | Dmitri Smirnov | 16 November 2015, 21:58:10 UTC | Fix empty vector write in ForwardIterator | 16 November 2015, 21:58:10 UTC |
| a163cc2 | Islam AbdelRahman | 16 November 2015, 20:56:21 UTC | Lint everything Summary: ``` arc2 lint --everything ``` run the linter on the whole code repo to fix exisitng lint issues Test Plan: make check -j64 Reviewers: sdong, rven, anthony, kradhakrishnan, yhchiang Reviewed By: yhchiang Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D50769 | 16 November 2015, 20:56:21 UTC |
| 8f01f25 | Dmitri Smirnov | 16 November 2015, 19:10:58 UTC | Remove forward_iter_bench from Win build. | 16 November 2015, 19:10:58 UTC |
| dac5b24 | sdong | 13 November 2015, 20:01:52 UTC | UniversalCompactionPicker::PickCompaction(): avoid to form compactions if there is no file Summary: Currently RocksDB may break in lines like this: for (size_t i = sorted_runs.size() - 1; i >= first_index_after; i--) { if options.level0_file_num_compaction_trigger=0. Fix it by not executing the logic of picking compactions if there is no file (sorted_runs.size() = 0). Also internally set options.level0_file_num_compaction_trigger=1 if users give a 0. 0 is a value makes no sense in RocksDB. Test Plan: Run all tests. Will add a unit test too. Reviewers: yhchiang, IslamAbdelRahman, anthony, kradhakrishnan, rven Reviewed By: rven Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D50727 | 16 November 2015, 18:32:45 UTC |
| d06b63e | Venkatesh Radhakrishnan | 16 November 2015, 17:57:08 UTC | Fix Rocksdb lite build failure in forward_iterator_bench Summary: Fixed Rocksdb lite build failure in forward_iterator_bench by defining main for the ROCKSDB_LITE case Test Plan: build ROCKSDB_LITE Reviewers: anthony, yhchiang, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D50733 | 16 November 2015, 17:57:08 UTC |
| 7824444 | Venkatesh Radhakrishnan | 13 November 2015, 23:50:59 UTC | Reuse file iterators in tailing iterator when memtable is flushed Summary: Under a tailing workload, there were increased block cache misses when a memtable was flushed because we were rebuilding iterators in that case since the version set changed. This was exacerbated in the case of iterate_upper_bound, since file iterators which were over the iterate_upper_bound would have been deleted and are now brought back as part of the Rebuild, only to be deleted again. We now renew the iterators and only build iterators for files which are added and delete file iterators for files which are deleted. Refer to https://reviews.facebook.net/D50463 for previous version Test Plan: DBTestTailingIterator.TailingIteratorTrimSeekToNext Reviewers: anthony, IslamAbdelRahman, igor, tnovak, yhchiang, sdong Reviewed By: sdong Subscribers: yhchiang, march, dhruba, leveldb, lovro Differential Revision: https://reviews.facebook.net/D50679 | 13 November 2015, 23:50:59 UTC |
| 2ae4d7d | Venkatesh Radhakrishnan | 13 November 2015, 20:01:00 UTC | Make sure that CompactFiles does not run two parallel Level 0 compactions Summary: Since level 0 files can overlap, two level 0 compactions cannot run in parallel. Compact files needs to check this before running a compaction. Test Plan: CompactFilesTest.L0ConflictsFiles Reviewers: igor, IslamAbdelRahman, anthony, sdong, yhchiang Reviewed By: yhchiang Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D50079 | 13 November 2015, 20:01:00 UTC |
| d781da8 | Yueh-Hsuan Chiang | 13 November 2015, 00:52:51 UTC | Add CheckOptionsCompatibility() API to options_util Summary: Add CheckOptionsCompatibility() API to options_util that returns Status::OK if the input DBOptions and ColumnFamilyDescriptors are compatible with the latest options stored in the specified DB path. Test Plan: Added tests in options_util_test Reviewers: igor, anthony, IslamAbdelRahman, rven, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D50649 | 13 November 2015, 00:52:51 UTC |
| 2391b45 | Siying Dong | 12 November 2015, 23:52:41 UTC | Merge pull request #824 from yuslepukhin/try_ci_tests_on_daily Try running db_test during integration build | 12 November 2015, 23:52:41 UTC |
| 2ab3e2d | yuslepukhin | 12 November 2015, 22:53:19 UTC | Fix a build break so tests can run | 12 November 2015, 22:53:19 UTC |
| 247c49a | yuslepukhin | 12 November 2015, 22:51:19 UTC | Merge branch 'master' into try_ci_tests_on_daily | 12 November 2015, 22:51:19 UTC |
