| 550af4e | SherlockNoMad | 30 October 2015, 05:41:57 UTC | Fix Travis Build Error | 30 October 2015, 05:41:57 UTC |
| a6dd083 | SherlockNoMad | 30 October 2015, 05:10:25 UTC | Add Option to Skip Flushing in TableBuilder | 30 October 2015, 05:10:25 UTC |
| 2872e0c | Islam AbdelRahman | 30 October 2015, 01:07:37 UTC | Clean and expose CreateLoggerFromOptions Summary: CreateLoggerFromOptions have some parameters like db_log_dir and env, these parameters are redundant since they already exist in DBOptions this patch remove the redundant parameters and expose CreateLoggerFromOptions to users Test Plan: make check Reviewers: igor, anthony, yhchiang, rven, kradhakrishnan, sdong Reviewed By: sdong Subscribers: dhruba, hermanlee4 Differential Revision: https://reviews.facebook.net/D49713 | 30 October 2015, 01:07:37 UTC |
| 296c3a1 | sdong | 29 October 2015, 22:52:32 UTC | "make format" in some recent commits Summary: Run "make format" for some recent commits. Test Plan: Build and run tests Reviewers: IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D49707 | 30 October 2015, 00:11:14 UTC |
| 6388e7f | Siying Dong | 29 October 2015, 22:02:59 UTC | Merge pull request #798 from yuslepukhin/readahead_buffermanagement Implement smart buffer management in Windows Env. | 29 October 2015, 22:02:59 UTC |
| f4cbb90 | Islam AbdelRahman | 29 October 2015, 20:41:36 UTC | Merge pull request #799 from yuslepukhin/fix_random_generator_compile Fix compilation problem on Windows. | 29 October 2015, 20:41:36 UTC |
| 1277a48 | Dmitri Smirnov | 29 October 2015, 18:34:34 UTC | Fix 80 character limit issue. | 29 October 2015, 18:34:34 UTC |
| ee2c323 | Dmitri Smirnov | 29 October 2015, 18:29:18 UTC | Fix compilation problem on Windows. char is not a valid template parameter for std::uniform_int_distribution according to the standard. Replacing with int should be just fine. | 29 October 2015, 18:29:18 UTC |
| c97667d | Igor Canadi | 28 October 2015, 23:37:39 UTC | Fix RocksDB lite build for write_stress Summary: We don't have access to GetLiveFilesMetadata() in RocksDB lite. If compiling write_stress for lite, I skip the check for leaked files, which depends on this function. Test Plan: OPT=-DROCKSDB_LITE m write_stress Reviewers: sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D49647 | 28 October 2015, 23:37:39 UTC |
| 0d720df | Herman Lee | 28 October 2015, 21:07:22 UTC | Use the correct variable when fetching table properties. Summary: An uninitialized parameter was being passed into the call to fetch the table properties during the compaction notification callbacks. Test Plan: Build it with myrocks and verify unit test passed. Run unit tests. Reviewers: rven, yhchiang, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D49635 | 28 October 2015, 23:28:11 UTC |
| 4b66d95 | Igor Canadi | 28 October 2015, 23:15:07 UTC | Write stress test Summary: The goal of this diff is to create a simple stress test with focus on catching: * bugs in compaction/flush processes, especially the ones that cause assertion errors * bugs in the code that deletes obsolete files There are two parts of the test: * write_stress, a binary that writes to the database * write_stress_runner.py, a script that invokes and kills write_stress Here are some interesting parts of write_stress: * Runs with very high concurrency of compactions and flushes (32 threads total) and tries to create a huge amount of small files * The keys written to the database are not uniformly distributed -- there is a 3-character prefix that mutates occasionally (in prefix mutator thread), in such a way that the first character mutates slower than second, which mutates slower than third character. That way, the compaction stress tests some interesting compaction features like trivial moves and bottommost level calculation * There is a thread that creates an iterator, holds it for couple of seconds and then iterates over all keys. This is supposed to test RocksDB's abilities to keep the files alive when there are references to them. * Some writes trigger WAL sync. This is stress testing our WAL sync code. * At the end of the run, we make sure that we didn't leak any of the sst files write_stress_runner.py changes the mode in which we run write_stress and also kills and restarts it. There are some interesting characteristics: * At the beginning we divide the full test runtime into smaller parts -- shorter runtimes (couple of seconds) and longer runtimes (100, 1000) seconds * The first time we run write_stress, we destroy the old DB. Every next time during the test, we use the same DB. * We can run in kill mode or clean-restart mode. Kill mode kills the write_stress violently. * We can run in mode where delete_obsolete_files_with_fullscan is true or false * We can run with low_open_files mode turned on or off. When it's turned on, we configure table cache to only hold a couple of files -- that way we need to reopen files every time we access them. Another goal was to create a stress test without a lot of parameters. So tools/write_stress_runner.py should only take one parameter -- runtime_sec and it should figure out everything else on its own. In a separate diff, I'll add this new test to our nightly legocastle runs. Test Plan: The goal of this test was to retroactively catch the following bugs: D33045, D48201, D46899, D42399. I failed to reproduce D48201, but all others have been caught! When i reverted https://reviews.facebook.net/D33045: ./write_stress --runtime_sec=200 --low_open_files_mode=true Iterator statuts not OK: IO error: /fast-rocksdb-tmp/rocksdb_test/write_stress/089166.sst: No such file or directory When i reverted https://reviews.facebook.net/D42399: python tools/write_stress_runner.py --runtime_sec=5000 Running write_stress, will kill after 5 seconds: ./write_stress --runtime_sec=-1 Running write_stress, will kill after 2 seconds: ./write_stress --runtime_sec=-1 --destroy_db=false --delete_obsolete_files_with_fullscan=true Running write_stress, will kill after 7 seconds: ./write_stress --runtime_sec=-1 --destroy_db=false Running write_stress, will kill after 5 seconds: ./write_stress --runtime_sec=-1 --destroy_db=false Running write_stress, will kill after 8 seconds: ./write_stress --runtime_sec=-1 --destroy_db=false --low_open_files_mode=true Write to DB failed: IO error: /fast-rocksdb-tmp/rocksdb_test/write_stress/019250.sst: No such file or directory ERROR: write_stress died with exitcode=-6 When i reverted https://reviews.facebook.net/D46899: python tools/write_stress_runner.py --runtime_sec=1000 runtime: 1000 Going to execute write stress for [3, 3, 100, 3, 2, 100, 1, 788] Running write_stress for 3 seconds: ./write_stress --runtime_sec=3 --low_open_files_mode=true Running write_stress for 3 seconds: ./write_stress --runtime_sec=3 --destroy_db=false --delete_obsolete_files_with_fullscan=true Running write_stress, will kill after 100 seconds: ./write_stress --runtime_sec=-1 --destroy_db=false --delete_obsolete_files_with_fullscan=true write_stress: db/db_impl.cc:2070: void rocksdb::DBImpl::MarkLogsSynced(uint64_t, bool, const rocksdb::Status&): Assertion `log.getting_synced' failed. ERROR: write_stress died with exitcode=-6 Reviewers: IslamAbdelRahman, yhchiang, rven, kradhakrishnan, sdong, anthony Reviewed By: anthony Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D49533 | 28 October 2015, 23:15:07 UTC |
| 47414c6 | sdong | 28 October 2015, 18:57:14 UTC | Move include/posix/io_posix.h to util/io_posix.h Summary: include/posix/io_posix.h is not a public API. Although include/posix/ is not a public header directory, it is confusing to put non-public headers to under include/. Move it to util/ to be clearer. Test Plan: Run all tests Reviewers: rven, IslamAbdelRahman, anthony, kradhakrishnan, yhchiang, igor Reviewed By: igor Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D49611 | 28 October 2015, 19:15:51 UTC |
| 2889df8 | sdong | 28 October 2015, 18:55:20 UTC | Revert "Avoid to reply on ROCKSDB_FALLOCATE_PRESENT in include/posix/io_posix.h" This reverts commit c37223c0836d1637c628f9cef3acb5b55ad3d51b. | 28 October 2015, 18:55:20 UTC |
| 28c8758 | Siying Dong | 28 October 2015, 18:37:37 UTC | Merge pull request #795 from yuslepukhin/fix_mocktable_id Fix MockTable ID storage | 28 October 2015, 18:37:37 UTC |
| 5c8f2ee | Dmitri Smirnov | 28 October 2015, 17:53:14 UTC | Fix MockTable ID storage On Windows two tests fail that use MockTable: flush_job_test and compaction_job_test with the following message: compaction_job_test_je.exe : Assertion failed: result.size() == 4, file c:\dev\rocksdb\rocksdb\table\mock_table.cc, line 110 Investigation reveals that this failure occurs when a 4 byte ID written to a beginning of the physically open file (main contents remains in a in-memory map) can not be read back. The reason for the failure is that the ID is written directly to a WritableFile bypassing WritableFileWriter. The side effect of that is that pending_sync_ never becomes true so the file is never flushed, however, the direct cause of the failure is that the filesize_ member of the WritableFileWriter remains zero. At Close() the file is truncated to that size and the file becomes empty so the ID can not be read back. | 28 October 2015, 17:53:14 UTC |
| 72d6e75 | Islam AbdelRahman | 28 October 2015, 04:04:00 UTC | Fix WritableFileWriter::Append() return Summary: It looks like WritableFileWriter::Append() was returning OK() even when there is an error Test Plan: make check Reviewers: sdong, yhchiang, anthony, rven, kradhakrishnan, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D49569 | 28 October 2015, 04:04:00 UTC |
| d0a18c2 | Siying Dong | 27 October 2015, 23:51:22 UTC | Merge pull request #786 from aloukissas/unused_param Fix unused parameter warnings in db.h | 27 October 2015, 23:51:22 UTC |
| c37223c | sdong | 27 October 2015, 21:27:48 UTC | Avoid to reply on ROCKSDB_FALLOCATE_PRESENT in include/posix/io_posix.h Summary: include/posix/io_posix.h should not depend on ROCKSDB_FALLOCATE_PRESENT. Remove it. Test Plan: Build it with both of ROCKSDB_FALLOCATE_PRESENT defined and not defined. Reviewers: rven, yhchiang, anthony, kradhakrishnan, IslamAbdelRahman, igor Reviewed By: igor Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D49563 | 27 October 2015, 23:48:40 UTC |
| 6fbc4f9 | Dmitri Smirnov | 27 October 2015, 21:44:16 UTC | Implement smart buffer management. introduce a new DBOption random_access_max_buffer_size to limit the size of the random access buffer used for unbuffered access. Implement read ahead buffering when enabled. To that effect propagate compaction_readahead_size and the new option to the env options to make it available for the implementation. Add Hint() override so SetupForCompaction() call would call Hint() readahead can now be setup from both Hint() and EnableReadAhead() Add new option random_access_max_buffer_size support db_bench, options_helper to make it string parsable and the unit test. | 27 October 2015, 21:44:16 UTC |
| d6219e4 | sdong | 27 October 2015, 19:15:55 UTC | Mac build break caused by include/posix/io_posix.h not declearing errno, Summary: Mac build breaks as include/posix/io_posix.h doesn't include errno. Move the exact function declaration to io_posix.cc Test Plan: Run all test. Will run on Mac Reviewers: rven, anthony, yhchiang, IslamAbdelRahman, igor Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D49551 | 27 October 2015, 21:16:19 UTC |
| beb69d4 | Siying Dong | 27 October 2015, 19:13:01 UTC | Merge pull request #765 from PraveenSinghRao/wal_filter Adding wal filter to inspect and filter wal records on recovery | 27 October 2015, 19:13:01 UTC |
| ab0f3b9 | sdong | 26 October 2015, 23:02:32 UTC | crash_test to trigger some less frequent crash point more frequently Summary: crash_test still has a very low chance to hit some crash point. Have another mode for covering them more likely. Test Plan: Run crash_test and see db_stress is called with expected prameters. Reviewers: kradhakrishnan, igor, anthony, rven, IslamAbdelRahman, yhchiang Reviewed By: yhchiang Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D49473 | 27 October 2015, 19:06:06 UTC |
| 7beb743 | Igor Canadi | 27 October 2015, 16:58:07 UTC | Merge pull request #778 from Vaisman/master Error while cmake by building from zip-archive | 27 October 2015, 16:58:07 UTC |
| 4ce117c | Praveen Rao | 27 October 2015, 02:03:34 UTC | Merge branch 'master' into wal_filter | 27 October 2015, 02:03:34 UTC |
| 32cdec6 | Praveen Rao | 27 October 2015, 01:11:18 UTC | Fail recovery if filter provides more records than original and corresponding unit-test, fix naming conventions | 27 October 2015, 01:11:18 UTC |
| 44d4057 | sdong | 23 October 2015, 16:16:46 UTC | Avoid some includes in io_posix.h Summary: IO Posix depends on too many .h files. Move most of them to .cc files. Test Plan: make all Reviewers: anthony, rven, IslamAbdelRahman, yhchiang, kradhakrishnan, igor Reviewed By: igor Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D49479 | 27 October 2015, 00:00:25 UTC |
| 2adad23 | Alex Loukissas | 26 October 2015, 23:38:14 UTC | Fix unused parameter warnings. | 26 October 2015, 23:38:37 UTC |
| b0980ff | Alex Loukissas | 26 October 2015, 23:00:51 UTC | Fix unused parameter warnings. | 26 October 2015, 23:32:14 UTC |
| bc898c5 | Alex Loukissas | 26 October 2015, 23:00:51 UTC | Fix unused parameter warnings. | 26 October 2015, 23:00:51 UTC |
| 138876a | Siying Dong | 26 October 2015, 22:01:28 UTC | Merge pull request #746 from ceph/wip-recycle Add Options.recycle_log_file_num for Recycling WAL Files | 26 October 2015, 22:01:28 UTC |
| 581f20f | Islam AbdelRahman | 26 October 2015, 18:50:29 UTC | Add LITE tests to Legocastle Summary: Update rocksdb-lego-determinator to include running make check under ROCKSDB_LITE Test Plan: will be tested after landing in fbcode Reviewers: sdong, yhchiang, igor, kradhakrishnan Reviewed By: kradhakrishnan Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D49065 | 26 October 2015, 18:50:29 UTC |
| 3d56d86 | Vasili Svirski | 24 October 2015, 17:05:59 UTC | Merge remote-tracking branch 'upstream/master' | 24 October 2015, 17:05:59 UTC |
| d691111 | sdong | 23 October 2015, 14:36:22 UTC | include/posix/io_posix.h should have a once declartion Summary: include/posix/io_posix.h doesn't not prevent multiple includes. Need to fix it. It is also breaking unity build. Test Plan: Run unity build and see error go away. Reviewers: rven, igor, IslamAbdelRahman, kradhakrishnan, anthony Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D49281 | 23 October 2015, 14:45:00 UTC |
| a6962ed | Igor Canadi | 23 October 2015, 00:14:52 UTC | Merge pull request #783 from yuslepukhin/remove_test_conditional_compilation No need to #ifdef test only code on windows | 23 October 2015, 00:14:52 UTC |
| 3c750b5 | Dmitri Smirnov | 22 October 2015, 22:15:37 UTC | No need to #ifdef test only code on windows | 22 October 2015, 22:15:37 UTC |
| 8c11c5d | Siying Dong | 22 October 2015, 20:33:51 UTC | Merge pull request #768 from OpenChannelSSD/to_fb_master2 Split posix storage backend into Env and library | 22 October 2015, 20:33:51 UTC |
| 6e6dd5f | Javier González | 14 October 2015, 08:14:53 UTC | Split posix storage backend into Env and library Summary: This patch splits the posix storage backend into Env and the actual *File implementations. The motivation is to allow other Envs to use posix as a library. This enables a storage backend different from posix to split its secondary storage between a normal file system partition managed by posix, and it own media. Test Plan: No new functionality is added to posix Env or the library, thus the current tests should suffice. | 22 October 2015, 15:31:31 UTC |
| 980a82e | Alexey Maykov | 22 October 2015, 01:34:25 UTC | Fix a bug in GetApproximateSizes Summary: Need to pass through the memtable parameter. Test Plan: built, tested through myrocks Reviewers: igor, sdong, rven Reviewed By: rven Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D49167 | 22 October 2015, 01:34:39 UTC |
| d0d13eb | Shusen Liu | 20 October 2015, 18:31:27 UTC | fix bug in db_crashtest.py Summary: in tools/db_crashtest.py, cmd_params['db'] by default is a lambda expression, not the actual db_name. fix by get the db_name before passing it to gen_cmd. Test Plan: run `make crashtest` Reviewers: sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D49119 | 21 October 2015, 05:01:11 UTC |
| 01a41af | Vasili Svirski | 21 October 2015, 03:52:10 UTC | Merge remote-tracking branch 'upstream/master' | 21 October 2015, 03:52:10 UTC |
| 5678c05 | Yueh-Hsuan Chiang | 21 October 2015, 00:09:09 UTC | Use DEBUG_LEVEL=0 in make release and make clean Summary: Use DEBUG_LEVEL=0 in make release and make clean Test Plan: make clean make release -j32 Reviewers: MarkCallaghan, sdong, anthony, IslamAbdelRahman, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D49125 | 21 October 2015, 00:09:09 UTC |
| ac25fe6 | Siying Dong | 20 October 2015, 23:32:21 UTC | Merge pull request #779 from yuslepukhin/optimize_windows_build Do not build test only code and unit tests in Release builds | 20 October 2015, 23:32:21 UTC |
| e154ee0 | Dmitri Smirnov | 20 October 2015, 20:35:08 UTC | Do not build test only code and unit tests in Release builds Test code errors are currently blocking Windows Release builew We do not want spend time building in Release what we can not run We want to eliminate a source of most frequent errors when people check-in test only code which can not be built in Release. This feature will work only if you invoke msbuild against rocksdb.sln Invoking it against ALL_BUILD target will attempt to build everything. | 20 October 2015, 20:35:08 UTC |
| cd3286f | Vasili Svirski | 20 October 2015, 18:27:48 UTC | Error while cmake by building from zip-archive * add validation is git found * add validation is .git folder exists in project (project zip archive download without .git folder) * get head commit SHA if git found and .git folder exists Tested: * configure project by CMake 3.0.0 successfully (with and without git), with project zip archive (without .git folder) and with project cloned from github * configure project by command: cmake -G "Visual Studio 12 Win64" * build solution by Visual Studio * manually validate that file utils/build_version.cc contains valid head revision value | 20 October 2015, 18:51:19 UTC |
| e3d4e14 | sdong | 20 October 2015, 06:40:15 UTC | DBCompactionTestWithParam.ManualCompaction to verify block cache is not filled in manual compaction Summary: Manual compaction should not fill block cache. Add the verification in unit test Test Plan: Run the test Reviewers: yhchiang, kradhakrishnan, rven, IslamAbdelRahman, anthony, igor Reviewed By: igor Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D49089 | 20 October 2015, 17:36:49 UTC |
| 033c6f1 | Shusen Liu | 20 October 2015, 03:58:38 UTC | T7916298, bug fix Summary: dbname => cmd_params['db'] Test Plan: Run `make crash_test` Reviewers: sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D49077 | 20 October 2015, 04:09:35 UTC |
| 7717ad1 | krad | 20 October 2015, 03:13:14 UTC | Adding artifacts to stress_crash CI job Summary: Adding the ability to upload logs and db content to storage after the completion of the job Test Plan: Manual run Reviewers: CC: leveldb@ Task ID: #8754201 Blame Rev: | 20 October 2015, 03:13:14 UTC |
| 0bf656b | Igor Canadi | 20 October 2015, 01:47:59 UTC | Don't spew warnings when flint doesn't exist Summary: Before this diff `arc lint` on non-fb machine issued warnings. Now it doesn't. Test Plan: `arc lint` is quiet. Reviewers: yhchiang, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D49071 | 20 October 2015, 01:47:59 UTC |
| 6d6776f | sdong | 19 October 2015, 20:07:05 UTC | Log more information for the add file with overlapping range failure Summary: crash_test sometimes fails, hitting the add file overlapping assert. Add information in info logs help us to find the bug. Test Plan: Run all test suites. Do some manual tests to make sure printing is correct. Reviewers: kradhakrishnan, yhchiang, anthony, IslamAbdelRahman, rven, igor Reviewed By: igor Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D49017 | 20 October 2015, 00:31:13 UTC |
| 7951b9b | Praveen Rao | 20 October 2015, 00:03:01 UTC | make field order match initialization order | 20 October 2015, 00:03:01 UTC |
| 90228bb | Siying Dong | 19 October 2015, 22:26:22 UTC | Merge pull request #771 from maximecaron/patch-1 Fix build error using Visual Studio 12 | 19 October 2015, 22:26:22 UTC |
| 2938c5c | Praveen Rao | 19 October 2015, 22:21:33 UTC | merge upstream changes | 19 October 2015, 22:21:33 UTC |
| e3b1d23 | Islam AbdelRahman | 19 October 2015, 22:05:59 UTC | Bump version to 4.2 Summary: Bump the version to 4.2 ( the unreleased version ), so that when fbcode_unittests run it can differentiate between old and new APIs Test Plan: make check Reviewers: sdong, yhchiang, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D49041 | 19 October 2015, 22:05:59 UTC |
| a7b2bed | Sage Weil | 19 October 2015, 21:24:05 UTC | log_{reader,write}: recyclable record format Introduce new tags for records that have a log_number. This changes the header size from 7 to 11 for these records, making this a backward-incompatible change. If we read a record that belongs to a different log_number (i.e., a previous instantiation of this log file, before it was most recently recycled), we return kOldRecord from ReadPhysicalRecord. ReadRecord will translate this into a kEof or kBadRecord depending on what the WAL recovery mode is. We make several adjustments to the log_test.cc tests to compensate for the fact that the header size varies between the two modes. Signed-off-by: Sage Weil <sage@redhat.com> | 19 October 2015, 21:24:05 UTC |
| 4e07c99 | Igor Canadi | 19 October 2015, 20:40:44 UTC | Fix iOS build Summary: We don't yet have a CI build for iOS, so our iOS compile gets broken sometimes. Most of the errors are from assumption that size_t is 64-bit, while it's actually 32-bit on some (all?) iOS platforms. This diff fixes the compile. Test Plan: TARGET_OS=IOS make static_lib Observe there are no warnings Reviewers: sdong, anthony, IslamAbdelRahman, kradhakrishnan, yhchiang Reviewed By: yhchiang Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D49029 | 19 October 2015, 20:40:44 UTC |
| 0c59691 | Praveen Rao | 19 October 2015, 20:27:40 UTC | Handle multiple batches in single log record - allow app to return a new batch + allow app to return corrupted record status | 19 October 2015, 20:27:40 UTC |
| 32c291e | Shusen Liu | 19 October 2015, 20:25:39 UTC | Merge branch 'master' of github.com:facebook/rocksdb into T7916298 | 19 October 2015, 20:25:39 UTC |
| 4575de5 | Shusen Liu | 19 October 2015, 18:43:14 UTC | #7916298: merge tools/db_crashtest2.py into tools/db_crashtest.py Summary: merge tools/db_crashtest2.py into tools/db_crashtest.py python tools/db_crashtest.py -h # show help message, ALL parameters can be overwrite by arguments Example usages: python tools/db_crashtest.py blackbox # run blackbox with default parameters python tools/db_crashtest.py blackbox --simple python tools/db_crashtest.py whitebox # run whitebox with default parameters python tools/db_crashtest.py whitebox --simple all default parameters are identical to previous version. Test Plan: `make crash_test` and make sure it can run with expected parameters pased to db_stress. Reviewers: igor, rven, anthony, IslamAbdelRahman, yhchiang, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D48567 | 19 October 2015, 20:24:55 UTC |
| 5c727de | Igor Canadi | 19 October 2015, 20:23:41 UTC | Merge pull request #777 from yuslepukhin/fix_win_build_uint uint is a not a datatype on windows. | 19 October 2015, 20:23:41 UTC |
| cfaa33f | agiardullo | 19 October 2015, 19:40:15 UTC | Update transaction iterator documentation Summary: Remove warning about an issue that was resolved. Turns out the issue was a false-alarm. Test Plan: n/a Reviewers: igor, yhchiang, rven, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D49011 | 19 October 2015, 20:08:18 UTC |
| 2f680ed | Dmitri Smirnov | 19 October 2015, 19:29:11 UTC | Make index same type as auto deduced uint32_t | 19 October 2015, 19:29:11 UTC |
| 09f8535 | Dmitri Smirnov | 19 October 2015, 18:28:22 UTC | uint is a not a datatype on windows. | 19 October 2015, 18:28:22 UTC |
| ec1f835 | Yueh-Hsuan Chiang | 19 October 2015, 18:26:12 UTC | Fix the default assignment of DEBUG_LEVEL in Makefile Summary: In the current make file, DEBUG_LEVEL is forced to set to 1 instead of default to 1. This patch fix this issue. Test Plan: DEBUG_LEVEL=2 make db_bench -j32 DEBUG_LEVEL=0 make db_bench -j32 DEBUG_LEVEL=0 make release -j32 And see whether there's a warning pops up. Reviewers: MarkCallaghan, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D48933 | 19 October 2015, 18:26:12 UTC |
| f18acd8 | Alexey Maykov | 19 October 2015, 17:34:55 UTC | Fixed the clang compilation failure Summary: As above. Test Plan: USE_CLANG=1 make check -j Reviewers: igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D48981 | 19 October 2015, 17:38:50 UTC |
| 4104e9b | Sage Weil | 12 October 2015, 21:04:33 UTC | log_reader: introduce kBadHeader; drop wal mode from ReadPhysicalRecord Move the WAL recovery mode logic out of ReadPhysicalRecord. To do this we introduce a new type indicating when we fail to read a valid header. Signed-off-by: Sage Weil <sage@redhat.com> | 19 October 2015, 01:24:32 UTC |
| 9c33f64 | Sage Weil | 02 October 2015, 19:53:59 UTC | log_reader: pass in WALRecoveryMode instead of bool report_eof_inconsistency Soon our behavior will depend on more than just whther we are in kAbsoluteConsistency or not. Signed-off-by: Sage Weil <sage@redhat.com> | 19 October 2015, 01:24:32 UTC |
| 7188052 | Sage Weil | 01 October 2015, 21:14:20 UTC | db_test_util: add recycle_log_files to set of tested options Signed-off-by: Sage Weil <sage@redhat.com> | 19 October 2015, 01:24:32 UTC |
| 3ac13c9 | Sage Weil | 08 October 2015, 17:06:16 UTC | log_reader: pass log_number and optional info_log to ctor We will need the log number to validate the recycle-style CRCs. The log is helpful for debugging, but optional, as not all callers have it. Signed-off-by: Sage Weil <sage@redhat.com> | 19 October 2015, 01:24:32 UTC |
| 5830c69 | Sage Weil | 08 October 2015, 17:07:15 UTC | log_writer: pass log number and whether recycling is enabled to ctor When we recycle log files, we need to mix the log number into the CRC for each record. Note that for logs that don't get recycled (like the manifest), we always pass a log_number of 0 and false. Signed-off-by: Sage Weil <sage@redhat.com> | 19 October 2015, 01:24:32 UTC |
| 6663761 | Sage Weil | 01 October 2015, 13:12:04 UTC | db_impl: recycle log files If log recycling is enabled, put old WAL files on a recycle queue instead of deleting them. When we need a new log file, take a recycled file off the list if one is available. Signed-off-by: Sage Weil <sage@redhat.com> | 19 October 2015, 01:24:32 UTC |
| d666225 | Sage Weil | 28 September 2015, 19:29:31 UTC | db_impl: disable recycle_log_files if WAL archive is enabled We can't recycle the files if they are being archived. Signed-off-by: Sage Weil <sage@redhat.com> | 19 October 2015, 01:21:24 UTC |
| 543c12a | Sage Weil | 08 October 2015, 01:06:28 UTC | options: add recycle_log_file_num option Signed-off-by: Sage Weil <sage@redhat.com> | 19 October 2015, 01:21:24 UTC |
| 1bcafb6 | Sage Weil | 08 October 2015, 02:11:09 UTC | env: add ReuseWritableFile Add an environment method to reuse an existing file. Provide a generic implementation that does a simple rename + open (writeable), and also a posix variant that is more careful about error handling (if we fail to open, do not rename, etc.). Signed-off-by: Sage Weil <sage@redhat.com> | 19 October 2015, 01:21:24 UTC |
| e1a09a7 | Alexey Maykov | 13 October 2015, 21:24:45 UTC | Implementation for GetPropertiesOfTablesInRange Summary: In MyRocks, it is sometimes important to get propeties only for the subset of the database. This diff implements the API in RocksDB. Test Plan: ran the GetPropertiesOfTablesInRange Reviewers: rven, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D48651 | 17 October 2015, 20:34:43 UTC |
| ad47145 | Yueh-Hsuan Chiang | 17 October 2015, 07:16:36 UTC | Allow GetProperty to report the number of currently running flushes / compactions. Summary: Add rocksdb.num-running-compactions and rocksdb.num-running-flushes to GetIntProperty() that reports the number of currently running compactions / flushes. Test Plan: augmented existing tests in db_test Reviewers: igor, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D48693 | 17 October 2015, 07:16:36 UTC |
| 277dea7 | sdong | 16 October 2015, 21:33:47 UTC | Add more kill points Summary: Add kill points in: 1. after creating a file 2. before writing a manifest record 3. before syncing manifest 4. before creating a new current file 5. after creating a new current file Test Plan: Run all current tests. Reviewers: yhchiang, igor, anthony, IslamAbdelRahman, rven, kradhakrishnan Reviewed By: kradhakrishnan Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D48855 | 16 October 2015, 21:35:12 UTC |
| a98fbac | Venkatesh Radhakrishnan | 16 October 2015, 21:10:33 UTC | Moving memtable related files from util to a new directory memtable Summary: We are cleaning up dependencies. This diff takes a first step at moving memtable files to their own directory called memtable. In future diffs, we will move other memtable files from db to memtable. Test Plan: make check Reviewers: sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D48915 | 16 October 2015, 21:10:33 UTC |
| 8f143e0 | Jay Edgar | 15 October 2015, 22:37:23 UTC | Add ClearSnapshot() Summary: MyRocks needs the ability to clear a snapshot for Read Committed support Test Plan: transaction_test Reviewers: anthony Reviewed By: anthony Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D48861 | 16 October 2015, 18:53:30 UTC |
| f9ba79e | sdong | 15 October 2015, 22:39:58 UTC | crash_test to trigger fail points other than file appending more frequently Summary: For half of the crash_test run, disable fail point for file appending, in order to trigger other fail point more frequently. Also, tune crash test parameter a little bit for it to initialize faster. Test Plan: Run crash_test and make sure it issues db_stress commands as expected. Reviewers: yhchiang, igor, rven, anthony, IslamAbdelRahman, kradhakrishnan Reviewed By: kradhakrishnan Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D48843 | 16 October 2015, 18:35:27 UTC |
| 2f2de33 | Islam AbdelRahman | 16 October 2015, 17:47:37 UTC | Run ROCKSDB_LITE tests in travis Summary: Update travis to run ROCKSDB_LITE tests Right now we run ROCKSDB_LITE tests in Jenkins https://ci-builds.fb.com/view/rocksdb/job/rocksdb_lite_build/ Test Plan: I am not sure if there is anyway to test this except by landing it and checking the travis output Reviewers: igor, yhchiang, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D48813 | 16 October 2015, 17:47:37 UTC |
| 680156c | sdong | 14 October 2015, 20:55:10 UTC | crash_test to run with data sync on Summary: Mode of data sync off is a much less used than the case of data sync on. Crash test should cover the more common case than a corner case. So turn data sync on in crash tests. Test Plan: Run crash test and make sure it can run with expected parameters pased to db_stres. Reviewers: IslamAbdelRahman, anthony, igor, rven, kradhakrishnan, yhchiang Reviewed By: yhchiang Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D48729 | 15 October 2015, 21:37:08 UTC |
| e1a5ff8 | sdong | 14 October 2015, 21:08:50 UTC | Allow users to disable some kill points in db_stress Summary: Give a name for every kill point, and allow users to disable some kill points based on prefixes. The kill points can be passed by db_stress through a command line paramter. This provides a way for users to boost the chance of triggering low frequency kill points This allow follow up changes in crash test scripts to improve crash test coverage. Test Plan: Manually run db_stress with variable values of --kill_random_test and --kill_prefix_blacklist. Like this: --kill_random_test=2 --kill_prefix_blacklist=Posix,WritableFileWriter::Append,WritableFileWriter::WriteBuffered,WritableFileWriter::Sync Reviewers: igor, kradhakrishnan, rven, IslamAbdelRahman, yhchiang Reviewed By: yhchiang Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D48735 | 15 October 2015, 21:33:13 UTC |
| d306a7e | Islam AbdelRahman | 15 October 2015, 20:52:31 UTC | Merge pull request #773 from yuslepukhin/update_requirements Update 4 is required for building with MS Visual Studio 13 | 15 October 2015, 20:52:31 UTC |
| bb64d6d | krad | 15 October 2015, 20:40:55 UTC | Disabling TSAN crash test Summary: The test is failing with OOM. Temporarily disabled. Test Plan: Manual check Reviewers: CC: leveldb@ Task ID: #8732029 Blame Rev: | 15 October 2015, 20:42:34 UTC |
| 8c2fe68 | Dmitri Smirnov | 15 October 2015, 18:06:02 UTC | Update 4 is required for building with MS Visual Studio 13 | 15 October 2015, 18:06:02 UTC |
| 952ad99 | Islam AbdelRahman | 15 October 2015, 17:59:31 UTC | Fix db_test under ROCKSDB_LITE Summary: This diff exclude alot of tests in db_test that are not compiling / failing under ROCKSD_LITE Test Plan: OPT=-DROCKSDB_LITE make check -j64 make check -j64 Reviewers: yhchiang, igor, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D48771 | 15 October 2015, 17:59:31 UTC |
| 6d730b4 | Islam AbdelRahman | 15 October 2015, 17:51:00 UTC | Block tests under ROCKSDB_LITE Summary: This patch will block all tests (not including db_test) that don't compile / fail under ROCKSDB_LITE Test Plan: OPT=-DROCKSDB_LITE make db_compaction_filter_test -j64 && OPT=-DROCKSDB_LITE make db_compaction_test -j64 && OPT=-DROCKSDB_LITE make db_dynamic_level_test -j64 && OPT=-DROCKSDB_LITE make db_log_iter_test -j64 && OPT=-DROCKSDB_LITE make db_tailing_iter_test -j64 && OPT=-DROCKSDB_LITE make db_universal_compaction_test -j64 && OPT=-DROCKSDB_LITE make ldb_cmd_test -j64 make clean make db_compaction_filter_test -j64 && make db_compaction_test -j64 && make db_dynamic_level_test -j64 && make db_log_iter_test -j64 && make db_tailing_iter_test -j64 && make db_universal_compaction_test -j64 && make ldb_cmd_test -j64 Reviewers: yhchiang, igor, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D48723 | 15 October 2015, 17:51:00 UTC |
| 5eee1ef | Siying Dong | 15 October 2015, 17:11:08 UTC | Merge pull request #770 from Vaisman/master Error while cmake by building from zip-archive | 15 October 2015, 17:11:08 UTC |
| 63e507c | Venkatesh Radhakrishnan | 15 October 2015, 00:08:28 UTC | Move ldb and sst_dump from utils to tools. Summary: As part of cleaning up dependencies for tech debt week, we are moving ldb and sst_dump tools from util to tools, since they are tools. Test Plan: make check Reviewers: sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D48747 | 15 October 2015, 00:08:28 UTC |
| dae49e8 | sdong | 14 October 2015, 22:13:46 UTC | Make DBTest.ReadLatencyHistogramByLevel more robust Summary: Two fixes: 1. Wait compaction after generating each L0 file so that we are sure there are one L0 file left. 2. https://reviews.facebook.net/D48423 increased from 500 keys to 700 keys but in verification phase we are still querying the first 500 keys. It is a bug to fix. Test Plan: Run the test in the same environment that fails by chance of one in tens of times. It doesn't fail after 1000 times. Reviewers: yhchiang, IslamAbdelRahman, igor, rven, kradhakrishnan Reviewed By: rven, kradhakrishnan Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D48759 | 14 October 2015, 23:08:55 UTC |
| 92060b2 | Maxime Caron | 14 October 2015, 23:05:55 UTC | Fix build error using Visual Studio 12 | 14 October 2015, 23:05:55 UTC |
| 9f74135 | Vasili Svirski | 14 October 2015, 20:47:09 UTC | Error while cmake by building from zip-archive * add -C ${CMAKE_CURRENT_SOURCE_DIR} to git can found revision from current source directory Tested: * configure project by CMake 3.0.0 successfully * configure project by command: cmake -G "Visual Studio 12 Win64" * build solution by Visual Studio * manually validate that file utils/build_version.cc contains valid head revision value | 14 October 2015, 21:02:45 UTC |
| b81b2ec | Islam AbdelRahman | 14 October 2015, 19:43:00 UTC | Fix benchmarks under ROCKSDB_LITE Summary: Fix db_bench and memtablerep_bench under ROCKSDB_LITE Test Plan: OPT=-DROCKSDB_LITE make db_bench -j64 OPT=-DROCKSDB_LITE make memtablerep_bench -j64 make db_bench -j64 make memtablerep_bench -j64 Reviewers: yhchiang, anthony, rven, igor, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D48717 | 14 October 2015, 19:43:00 UTC |
| e587dbe | Venkatesh Radhakrishnan | 14 October 2015, 18:06:27 UTC | Move manual_compaction_test.cc from util to db Summary: manual_compaction_test.cc incorrectly in util. Moved to db. Test Plan: make check Reviewers: sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D48687 | 14 October 2015, 18:06:27 UTC |
| 666fb5d | sdong | 14 October 2015, 00:26:14 UTC | Remove DefaultCompactionFilterFactory. Summary: DefaultCompactionFilterFactory is not used anymore after recent changes. Remove it. Test Plan: Just run existing tests. Reviewers: rven, igor, yhchiang, kradhakrishnan, anthony, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D48699 | 14 October 2015, 18:02:30 UTC |
| d662b8d | Siying Dong | 14 October 2015, 17:07:17 UTC | Merge pull request #766 from PraveenSinghRao/lockfix move debug variable under ifndef NDEBUG | 14 October 2015, 17:07:17 UTC |
| f55d300 | Islam AbdelRahman | 14 October 2015, 00:33:23 UTC | Make db_test_util compile under ROCKSDB_LITE Summary: db_test_util is used in multiple test files but it dont compile under ROCKSDB_LITE Test Plan: make check make static_lib OPT=-DROCKSDB_LITE make db_wal_test Reviewers: igor, yhchiang, rven, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D48579 | 14 October 2015, 00:33:23 UTC |
| 29a47cd | Yueh-Hsuan Chiang | 14 October 2015, 00:24:13 UTC | Include the time unit in the comment of perf_context timers Summary: Include the time unit in the comment of perf_context timers Test Plan: make perf_context_test Reviewers: igor, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D48663 | 14 October 2015, 00:24:13 UTC |
| 2b925cc | Yueh-Hsuan Chiang | 14 October 2015, 00:23:40 UTC | Correct the comments in db/internal_stats.h Summary: Correct the comments in db/internal_stats.h Test Plan: no code change Reviewers: igor, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D48675 | 14 October 2015, 00:23:40 UTC |
| 35ad531 | sdong | 12 October 2015, 22:06:38 UTC | Seperate InternalIterator from Iterator Summary: Separate a new class InternalIterator from class Iterator, when the look-up is done internally, which also means they operate on key with sequence ID and type. This change will enable potential future optimizations but for now InternalIterator's functions are still the same as Iterator's. At the same time, separate the cleanup function to a separate class and let both of InternalIterator and Iterator inherit from it. Test Plan: Run all existing tests. Reviewers: igor, yhchiang, anthony, kradhakrishnan, IslamAbdelRahman, rven Reviewed By: rven Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D48549 | 13 October 2015, 22:32:13 UTC |
