| e3b1d23 | Islam AbdelRahman | 19 October 2015, 22:05:59 UTC | Bump version to 4.2 Summary: Bump the version to 4.2 ( the unreleased version ), so that when fbcode_unittests run it can differentiate between old and new APIs Test Plan: make check Reviewers: sdong, yhchiang, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D49041 | 19 October 2015, 22:05:59 UTC |
| a7b2bed | Sage Weil | 19 October 2015, 21:24:05 UTC | log_{reader,write}: recyclable record format Introduce new tags for records that have a log_number. This changes the header size from 7 to 11 for these records, making this a backward-incompatible change. If we read a record that belongs to a different log_number (i.e., a previous instantiation of this log file, before it was most recently recycled), we return kOldRecord from ReadPhysicalRecord. ReadRecord will translate this into a kEof or kBadRecord depending on what the WAL recovery mode is. We make several adjustments to the log_test.cc tests to compensate for the fact that the header size varies between the two modes. Signed-off-by: Sage Weil <sage@redhat.com> | 19 October 2015, 21:24:05 UTC |
| 4e07c99 | Igor Canadi | 19 October 2015, 20:40:44 UTC | Fix iOS build Summary: We don't yet have a CI build for iOS, so our iOS compile gets broken sometimes. Most of the errors are from assumption that size_t is 64-bit, while it's actually 32-bit on some (all?) iOS platforms. This diff fixes the compile. Test Plan: TARGET_OS=IOS make static_lib Observe there are no warnings Reviewers: sdong, anthony, IslamAbdelRahman, kradhakrishnan, yhchiang Reviewed By: yhchiang Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D49029 | 19 October 2015, 20:40:44 UTC |
| 0c59691 | Praveen Rao | 19 October 2015, 20:27:40 UTC | Handle multiple batches in single log record - allow app to return a new batch + allow app to return corrupted record status | 19 October 2015, 20:27:40 UTC |
| 32c291e | Shusen Liu | 19 October 2015, 20:25:39 UTC | Merge branch 'master' of github.com:facebook/rocksdb into T7916298 | 19 October 2015, 20:25:39 UTC |
| 4575de5 | Shusen Liu | 19 October 2015, 18:43:14 UTC | #7916298: merge tools/db_crashtest2.py into tools/db_crashtest.py Summary: merge tools/db_crashtest2.py into tools/db_crashtest.py python tools/db_crashtest.py -h # show help message, ALL parameters can be overwrite by arguments Example usages: python tools/db_crashtest.py blackbox # run blackbox with default parameters python tools/db_crashtest.py blackbox --simple python tools/db_crashtest.py whitebox # run whitebox with default parameters python tools/db_crashtest.py whitebox --simple all default parameters are identical to previous version. Test Plan: `make crash_test` and make sure it can run with expected parameters pased to db_stress. Reviewers: igor, rven, anthony, IslamAbdelRahman, yhchiang, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D48567 | 19 October 2015, 20:24:55 UTC |
| 5c727de | Igor Canadi | 19 October 2015, 20:23:41 UTC | Merge pull request #777 from yuslepukhin/fix_win_build_uint uint is a not a datatype on windows. | 19 October 2015, 20:23:41 UTC |
| cfaa33f | agiardullo | 19 October 2015, 19:40:15 UTC | Update transaction iterator documentation Summary: Remove warning about an issue that was resolved. Turns out the issue was a false-alarm. Test Plan: n/a Reviewers: igor, yhchiang, rven, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D49011 | 19 October 2015, 20:08:18 UTC |
| 2f680ed | Dmitri Smirnov | 19 October 2015, 19:29:11 UTC | Make index same type as auto deduced uint32_t | 19 October 2015, 19:29:11 UTC |
| 09f8535 | Dmitri Smirnov | 19 October 2015, 18:28:22 UTC | uint is a not a datatype on windows. | 19 October 2015, 18:28:22 UTC |
| ec1f835 | Yueh-Hsuan Chiang | 19 October 2015, 18:26:12 UTC | Fix the default assignment of DEBUG_LEVEL in Makefile Summary: In the current make file, DEBUG_LEVEL is forced to set to 1 instead of default to 1. This patch fix this issue. Test Plan: DEBUG_LEVEL=2 make db_bench -j32 DEBUG_LEVEL=0 make db_bench -j32 DEBUG_LEVEL=0 make release -j32 And see whether there's a warning pops up. Reviewers: MarkCallaghan, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D48933 | 19 October 2015, 18:26:12 UTC |
| f18acd8 | Alexey Maykov | 19 October 2015, 17:34:55 UTC | Fixed the clang compilation failure Summary: As above. Test Plan: USE_CLANG=1 make check -j Reviewers: igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D48981 | 19 October 2015, 17:38:50 UTC |
| 4104e9b | Sage Weil | 12 October 2015, 21:04:33 UTC | log_reader: introduce kBadHeader; drop wal mode from ReadPhysicalRecord Move the WAL recovery mode logic out of ReadPhysicalRecord. To do this we introduce a new type indicating when we fail to read a valid header. Signed-off-by: Sage Weil <sage@redhat.com> | 19 October 2015, 01:24:32 UTC |
| 9c33f64 | Sage Weil | 02 October 2015, 19:53:59 UTC | log_reader: pass in WALRecoveryMode instead of bool report_eof_inconsistency Soon our behavior will depend on more than just whther we are in kAbsoluteConsistency or not. Signed-off-by: Sage Weil <sage@redhat.com> | 19 October 2015, 01:24:32 UTC |
| 7188052 | Sage Weil | 01 October 2015, 21:14:20 UTC | db_test_util: add recycle_log_files to set of tested options Signed-off-by: Sage Weil <sage@redhat.com> | 19 October 2015, 01:24:32 UTC |
| 3ac13c9 | Sage Weil | 08 October 2015, 17:06:16 UTC | log_reader: pass log_number and optional info_log to ctor We will need the log number to validate the recycle-style CRCs. The log is helpful for debugging, but optional, as not all callers have it. Signed-off-by: Sage Weil <sage@redhat.com> | 19 October 2015, 01:24:32 UTC |
| 5830c69 | Sage Weil | 08 October 2015, 17:07:15 UTC | log_writer: pass log number and whether recycling is enabled to ctor When we recycle log files, we need to mix the log number into the CRC for each record. Note that for logs that don't get recycled (like the manifest), we always pass a log_number of 0 and false. Signed-off-by: Sage Weil <sage@redhat.com> | 19 October 2015, 01:24:32 UTC |
| 6663761 | Sage Weil | 01 October 2015, 13:12:04 UTC | db_impl: recycle log files If log recycling is enabled, put old WAL files on a recycle queue instead of deleting them. When we need a new log file, take a recycled file off the list if one is available. Signed-off-by: Sage Weil <sage@redhat.com> | 19 October 2015, 01:24:32 UTC |
| d666225 | Sage Weil | 28 September 2015, 19:29:31 UTC | db_impl: disable recycle_log_files if WAL archive is enabled We can't recycle the files if they are being archived. Signed-off-by: Sage Weil <sage@redhat.com> | 19 October 2015, 01:21:24 UTC |
| 543c12a | Sage Weil | 08 October 2015, 01:06:28 UTC | options: add recycle_log_file_num option Signed-off-by: Sage Weil <sage@redhat.com> | 19 October 2015, 01:21:24 UTC |
| 1bcafb6 | Sage Weil | 08 October 2015, 02:11:09 UTC | env: add ReuseWritableFile Add an environment method to reuse an existing file. Provide a generic implementation that does a simple rename + open (writeable), and also a posix variant that is more careful about error handling (if we fail to open, do not rename, etc.). Signed-off-by: Sage Weil <sage@redhat.com> | 19 October 2015, 01:21:24 UTC |
| e1a09a7 | Alexey Maykov | 13 October 2015, 21:24:45 UTC | Implementation for GetPropertiesOfTablesInRange Summary: In MyRocks, it is sometimes important to get propeties only for the subset of the database. This diff implements the API in RocksDB. Test Plan: ran the GetPropertiesOfTablesInRange Reviewers: rven, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D48651 | 17 October 2015, 20:34:43 UTC |
| ad47145 | Yueh-Hsuan Chiang | 17 October 2015, 07:16:36 UTC | Allow GetProperty to report the number of currently running flushes / compactions. Summary: Add rocksdb.num-running-compactions and rocksdb.num-running-flushes to GetIntProperty() that reports the number of currently running compactions / flushes. Test Plan: augmented existing tests in db_test Reviewers: igor, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D48693 | 17 October 2015, 07:16:36 UTC |
| 277dea7 | sdong | 16 October 2015, 21:33:47 UTC | Add more kill points Summary: Add kill points in: 1. after creating a file 2. before writing a manifest record 3. before syncing manifest 4. before creating a new current file 5. after creating a new current file Test Plan: Run all current tests. Reviewers: yhchiang, igor, anthony, IslamAbdelRahman, rven, kradhakrishnan Reviewed By: kradhakrishnan Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D48855 | 16 October 2015, 21:35:12 UTC |
| a98fbac | Venkatesh Radhakrishnan | 16 October 2015, 21:10:33 UTC | Moving memtable related files from util to a new directory memtable Summary: We are cleaning up dependencies. This diff takes a first step at moving memtable files to their own directory called memtable. In future diffs, we will move other memtable files from db to memtable. Test Plan: make check Reviewers: sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D48915 | 16 October 2015, 21:10:33 UTC |
| 8f143e0 | Jay Edgar | 15 October 2015, 22:37:23 UTC | Add ClearSnapshot() Summary: MyRocks needs the ability to clear a snapshot for Read Committed support Test Plan: transaction_test Reviewers: anthony Reviewed By: anthony Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D48861 | 16 October 2015, 18:53:30 UTC |
| f9ba79e | sdong | 15 October 2015, 22:39:58 UTC | crash_test to trigger fail points other than file appending more frequently Summary: For half of the crash_test run, disable fail point for file appending, in order to trigger other fail point more frequently. Also, tune crash test parameter a little bit for it to initialize faster. Test Plan: Run crash_test and make sure it issues db_stress commands as expected. Reviewers: yhchiang, igor, rven, anthony, IslamAbdelRahman, kradhakrishnan Reviewed By: kradhakrishnan Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D48843 | 16 October 2015, 18:35:27 UTC |
| 2f2de33 | Islam AbdelRahman | 16 October 2015, 17:47:37 UTC | Run ROCKSDB_LITE tests in travis Summary: Update travis to run ROCKSDB_LITE tests Right now we run ROCKSDB_LITE tests in Jenkins https://ci-builds.fb.com/view/rocksdb/job/rocksdb_lite_build/ Test Plan: I am not sure if there is anyway to test this except by landing it and checking the travis output Reviewers: igor, yhchiang, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D48813 | 16 October 2015, 17:47:37 UTC |
| 680156c | sdong | 14 October 2015, 20:55:10 UTC | crash_test to run with data sync on Summary: Mode of data sync off is a much less used than the case of data sync on. Crash test should cover the more common case than a corner case. So turn data sync on in crash tests. Test Plan: Run crash test and make sure it can run with expected parameters pased to db_stres. Reviewers: IslamAbdelRahman, anthony, igor, rven, kradhakrishnan, yhchiang Reviewed By: yhchiang Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D48729 | 15 October 2015, 21:37:08 UTC |
| e1a5ff8 | sdong | 14 October 2015, 21:08:50 UTC | Allow users to disable some kill points in db_stress Summary: Give a name for every kill point, and allow users to disable some kill points based on prefixes. The kill points can be passed by db_stress through a command line paramter. This provides a way for users to boost the chance of triggering low frequency kill points This allow follow up changes in crash test scripts to improve crash test coverage. Test Plan: Manually run db_stress with variable values of --kill_random_test and --kill_prefix_blacklist. Like this: --kill_random_test=2 --kill_prefix_blacklist=Posix,WritableFileWriter::Append,WritableFileWriter::WriteBuffered,WritableFileWriter::Sync Reviewers: igor, kradhakrishnan, rven, IslamAbdelRahman, yhchiang Reviewed By: yhchiang Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D48735 | 15 October 2015, 21:33:13 UTC |
| d306a7e | Islam AbdelRahman | 15 October 2015, 20:52:31 UTC | Merge pull request #773 from yuslepukhin/update_requirements Update 4 is required for building with MS Visual Studio 13 | 15 October 2015, 20:52:31 UTC |
| bb64d6d | krad | 15 October 2015, 20:40:55 UTC | Disabling TSAN crash test Summary: The test is failing with OOM. Temporarily disabled. Test Plan: Manual check Reviewers: CC: leveldb@ Task ID: #8732029 Blame Rev: | 15 October 2015, 20:42:34 UTC |
| 8c2fe68 | Dmitri Smirnov | 15 October 2015, 18:06:02 UTC | Update 4 is required for building with MS Visual Studio 13 | 15 October 2015, 18:06:02 UTC |
| 952ad99 | Islam AbdelRahman | 15 October 2015, 17:59:31 UTC | Fix db_test under ROCKSDB_LITE Summary: This diff exclude alot of tests in db_test that are not compiling / failing under ROCKSD_LITE Test Plan: OPT=-DROCKSDB_LITE make check -j64 make check -j64 Reviewers: yhchiang, igor, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D48771 | 15 October 2015, 17:59:31 UTC |
| 6d730b4 | Islam AbdelRahman | 15 October 2015, 17:51:00 UTC | Block tests under ROCKSDB_LITE Summary: This patch will block all tests (not including db_test) that don't compile / fail under ROCKSDB_LITE Test Plan: OPT=-DROCKSDB_LITE make db_compaction_filter_test -j64 && OPT=-DROCKSDB_LITE make db_compaction_test -j64 && OPT=-DROCKSDB_LITE make db_dynamic_level_test -j64 && OPT=-DROCKSDB_LITE make db_log_iter_test -j64 && OPT=-DROCKSDB_LITE make db_tailing_iter_test -j64 && OPT=-DROCKSDB_LITE make db_universal_compaction_test -j64 && OPT=-DROCKSDB_LITE make ldb_cmd_test -j64 make clean make db_compaction_filter_test -j64 && make db_compaction_test -j64 && make db_dynamic_level_test -j64 && make db_log_iter_test -j64 && make db_tailing_iter_test -j64 && make db_universal_compaction_test -j64 && make ldb_cmd_test -j64 Reviewers: yhchiang, igor, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D48723 | 15 October 2015, 17:51:00 UTC |
| 5eee1ef | Siying Dong | 15 October 2015, 17:11:08 UTC | Merge pull request #770 from Vaisman/master Error while cmake by building from zip-archive | 15 October 2015, 17:11:08 UTC |
| 63e507c | Venkatesh Radhakrishnan | 15 October 2015, 00:08:28 UTC | Move ldb and sst_dump from utils to tools. Summary: As part of cleaning up dependencies for tech debt week, we are moving ldb and sst_dump tools from util to tools, since they are tools. Test Plan: make check Reviewers: sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D48747 | 15 October 2015, 00:08:28 UTC |
| dae49e8 | sdong | 14 October 2015, 22:13:46 UTC | Make DBTest.ReadLatencyHistogramByLevel more robust Summary: Two fixes: 1. Wait compaction after generating each L0 file so that we are sure there are one L0 file left. 2. https://reviews.facebook.net/D48423 increased from 500 keys to 700 keys but in verification phase we are still querying the first 500 keys. It is a bug to fix. Test Plan: Run the test in the same environment that fails by chance of one in tens of times. It doesn't fail after 1000 times. Reviewers: yhchiang, IslamAbdelRahman, igor, rven, kradhakrishnan Reviewed By: rven, kradhakrishnan Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D48759 | 14 October 2015, 23:08:55 UTC |
| 92060b2 | Maxime Caron | 14 October 2015, 23:05:55 UTC | Fix build error using Visual Studio 12 | 14 October 2015, 23:05:55 UTC |
| 9f74135 | Vasili Svirski | 14 October 2015, 20:47:09 UTC | Error while cmake by building from zip-archive * add -C ${CMAKE_CURRENT_SOURCE_DIR} to git can found revision from current source directory Tested: * configure project by CMake 3.0.0 successfully * configure project by command: cmake -G "Visual Studio 12 Win64" * build solution by Visual Studio * manually validate that file utils/build_version.cc contains valid head revision value | 14 October 2015, 21:02:45 UTC |
| b81b2ec | Islam AbdelRahman | 14 October 2015, 19:43:00 UTC | Fix benchmarks under ROCKSDB_LITE Summary: Fix db_bench and memtablerep_bench under ROCKSDB_LITE Test Plan: OPT=-DROCKSDB_LITE make db_bench -j64 OPT=-DROCKSDB_LITE make memtablerep_bench -j64 make db_bench -j64 make memtablerep_bench -j64 Reviewers: yhchiang, anthony, rven, igor, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D48717 | 14 October 2015, 19:43:00 UTC |
| e587dbe | Venkatesh Radhakrishnan | 14 October 2015, 18:06:27 UTC | Move manual_compaction_test.cc from util to db Summary: manual_compaction_test.cc incorrectly in util. Moved to db. Test Plan: make check Reviewers: sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D48687 | 14 October 2015, 18:06:27 UTC |
| 666fb5d | sdong | 14 October 2015, 00:26:14 UTC | Remove DefaultCompactionFilterFactory. Summary: DefaultCompactionFilterFactory is not used anymore after recent changes. Remove it. Test Plan: Just run existing tests. Reviewers: rven, igor, yhchiang, kradhakrishnan, anthony, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D48699 | 14 October 2015, 18:02:30 UTC |
| d662b8d | Siying Dong | 14 October 2015, 17:07:17 UTC | Merge pull request #766 from PraveenSinghRao/lockfix move debug variable under ifndef NDEBUG | 14 October 2015, 17:07:17 UTC |
| f55d300 | Islam AbdelRahman | 14 October 2015, 00:33:23 UTC | Make db_test_util compile under ROCKSDB_LITE Summary: db_test_util is used in multiple test files but it dont compile under ROCKSDB_LITE Test Plan: make check make static_lib OPT=-DROCKSDB_LITE make db_wal_test Reviewers: igor, yhchiang, rven, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D48579 | 14 October 2015, 00:33:23 UTC |
| 29a47cd | Yueh-Hsuan Chiang | 14 October 2015, 00:24:13 UTC | Include the time unit in the comment of perf_context timers Summary: Include the time unit in the comment of perf_context timers Test Plan: make perf_context_test Reviewers: igor, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D48663 | 14 October 2015, 00:24:13 UTC |
| 2b925cc | Yueh-Hsuan Chiang | 14 October 2015, 00:23:40 UTC | Correct the comments in db/internal_stats.h Summary: Correct the comments in db/internal_stats.h Test Plan: no code change Reviewers: igor, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D48675 | 14 October 2015, 00:23:40 UTC |
| 35ad531 | sdong | 12 October 2015, 22:06:38 UTC | Seperate InternalIterator from Iterator Summary: Separate a new class InternalIterator from class Iterator, when the look-up is done internally, which also means they operate on key with sequence ID and type. This change will enable potential future optimizations but for now InternalIterator's functions are still the same as Iterator's. At the same time, separate the cleanup function to a separate class and let both of InternalIterator and Iterator inherit from it. Test Plan: Run all existing tests. Reviewers: igor, yhchiang, anthony, kradhakrishnan, IslamAbdelRahman, rven Reviewed By: rven Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D48549 | 13 October 2015, 22:32:13 UTC |
| 91c041e | Praveen Rao | 13 October 2015, 21:28:11 UTC | move debug variable under ifndef | 13 October 2015, 21:28:11 UTC |
| 198ed58 | Yueh-Hsuan Chiang | 13 October 2015, 18:13:14 UTC | Merge pull request #760 from jwlent55/use-static-library-header-files Modify the way java static builds are done so that: | 13 October 2015, 18:13:14 UTC |
| cc4d13e | Praveen Rao | 13 October 2015, 18:10:14 UTC | Put wal_filter under #ifndef ROCKSDB_LITE | 13 October 2015, 18:10:14 UTC |
| 385b416 | Yueh-Hsuan Chiang | 13 October 2015, 18:06:48 UTC | Merge pull request #764 from dmittendorf/fix-java-static-packaging Fix crossbuild jar packaging | 13 October 2015, 18:06:48 UTC |
| 7062d0e | Yueh-Hsuan Chiang | 13 October 2015, 17:41:48 UTC | Make perf_context.db_mutex_lock_nanos and db_condition_wait_nanos only measures DB Mutex Summary: In the current implementation, perf_context.db_mutex_lock_nanos and perf_context.db_condition_wait_nanos also include the mutex-wait time other than DB Mutex. This patch fix this issue by incrementing the counters only when it detects a DB mutex. Test Plan: perf_context_test Reviewers: anthony, IslamAbdelRahman, sdong, igor Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D48555 | 13 October 2015, 17:41:48 UTC |
| 1fe78a4 | Islam AbdelRahman | 13 October 2015, 17:32:05 UTC | Fix tests failing in ROCKSDB_LITE Summary: Fix tests that compile under ROCKSDB_LITE but currently failing. table_test: RandomizedLongDB test is using internal stats which is not supported in ROCKSDB_LITE compaction_job_test: Using CompactionJobStats which is not supported perf_context_test: KeyComparisonCount test try to open DB in ReadOnly mode which is not supported Test Plan: run the tests under ROCKSDB_LITE Reviewers: yhchiang, sdong, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D48585 | 13 October 2015, 17:32:05 UTC |
| a6efefe | Praveen Rao | 13 October 2015, 02:20:09 UTC | Fix format specifiers | 13 October 2015, 02:20:09 UTC |
| f7b2a7b | Praveen Rao | 13 October 2015, 01:55:46 UTC | Fix format specifiers | 13 October 2015, 01:55:46 UTC |
| 7f58ff7 | Islam AbdelRahman | 13 October 2015, 00:35:43 UTC | Remove db_impl_debug from release build Summary: Remove db_impl_debug from NDEBUG, but allow it in ROCKSDB_LITE These functions by definition should not be included in NDEBUG and they are only used for testing This is based on offline discussion with @yhchiang and @igor Test Plan: make static_lib make check Reviewers: igor, sdong, yhchiang Reviewed By: yhchiang Subscribers: igor, yhchiang, dhruba Differential Revision: https://reviews.facebook.net/D48573 | 13 October 2015, 00:35:43 UTC |
| 1ddd91c | dmittendorf | 13 October 2015, 00:29:58 UTC | Fixed packaging of java crossbuild jar by forcing all compiled binaries to be output to the java/target directory. The uber crossbuild jar is then assembled within the java/target directory. | 13 October 2015, 00:29:58 UTC |
| eb24178 | Praveen Rao | 13 October 2015, 00:24:21 UTC | merge from master | 13 October 2015, 00:24:21 UTC |
| c64ae05 | Islam AbdelRahman | 13 October 2015, 00:22:37 UTC | Move TEST_NewInternalIterator to NewInternalIterator Summary: Long time ago we add InternalDumpCommand to ldb_tool https://reviews.facebook.net/D11517 This command is using TEST_NewInternalIterator although it's not a test. This patch move TEST_NewInternalIterator outside of db_impl_debug.cc Test Plan: make check make static_lib Reviewers: yhchiang, igor, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D48561 | 13 October 2015, 00:22:37 UTC |
| 59a0c21 | Praveen Rao | 13 October 2015, 00:03:03 UTC | Adding log filter to inspect and filter log records on recovery | 13 October 2015, 00:03:03 UTC |
| 0be50ed | Igor Canadi | 12 October 2015, 23:03:04 UTC | Merge pull request #763 from PraveenSinghRao/lockfix Fixing mutex to not use unique_lock | 12 October 2015, 23:03:04 UTC |
| a1d3760 | Praveen Rao | 12 October 2015, 22:41:20 UTC | Fixing mutex to not use unique_lock | 12 October 2015, 22:41:20 UTC |
| 9e819d0 | James Lent | 11 October 2015, 04:12:25 UTC | Modify the way java static builds are done so that: 1) There is no need to download and install the compression libraries twice just to get access to their header files during the compile phase. 2) Ensure that the compression library headers files used during the compile phase are the same ones used to build the static library that is linked into the library. | 12 October 2015, 22:24:51 UTC |
| f1fdf52 | Venkatesh Radhakrishnan | 12 October 2015, 20:05:42 UTC | Clean up dependency: Move db_test_util.* to db directory Summary: As part of tech debt week, we are cleaning up dependencies. This diff moves db_test_util.[h,cc] from util to db directory. Test Plan: make check Reviewers: igor, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D48543 | 12 October 2015, 20:05:42 UTC |
| 2379944 | Yueh-Hsuan Chiang | 12 October 2015, 00:40:50 UTC | Fixed an incorrect replace of const value in util/options_helper.cc Summary: Fixed an incorrect replace of const value in util/options_helper.cc Test Plan: options_test Reviewers: igor, sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D48513 | 12 October 2015, 00:42:22 UTC |
| 0bb8ea5 | Yueh-Hsuan Chiang | 11 October 2015, 19:17:42 UTC | [RocksDB Options File] Add TableOptions section and support BlockBasedTable Summary: Introduce TableOptions section and support BlockBasedTable in RocksDB options file. A TableOptions section has the following format: [TableOptions/<FactoryClassName> "<ColumnFamily Name>"] which includes information about its TableFactory class and belonging column family. Below is an example TableOptions section of a BlockBasedTableOptions that belongs to the default column family: [TableOptions/BlockBasedTable "default"] format_version=0 whole_key_filtering=true block_size_deviation=10 block_size=4096 block_restart_interval=16 filter_policy=nullptr no_block_cache=false checksum=kCRC32c cache_index_and_filter_blocks=false index_type=kBinarySearch hash_index_allow_collision=true flush_block_policy_factory=FlushBlockBySizePolicyFactory Currently, Cache-type options (i.e., block_cache and block_cache_compressed) are not supported. Test Plan: options_test Reviewers: igor, anthony, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D48435 | 11 October 2015, 19:17:42 UTC |
| 4a7970d | James Lent | 11 October 2015, 04:12:25 UTC | Modify the way java static builds are done so that: 1) There is no need to download and install the compression libraries twice just to get access to their header files during the compile phase. 2) Ensure that the compression library headers files used during the compile phase are the same ones used to build the static library that is linked into the library. | 11 October 2015, 13:51:38 UTC |
| c436616 | Yueh-Hsuan Chiang | 10 October 2015, 03:03:39 UTC | Merge pull request #759 from jwlent55/statically-load-compression-libraries Ensure that the compression libraries are statically linked into dyna… | 10 October 2015, 03:03:39 UTC |
| fa4b5b3 | Alexey Maykov | 10 October 2015, 02:30:58 UTC | Fix for the travis build caused by my previous commit Summary: My previous commit ('Passing table properties to compaction callback') broke the clang build. Here is the fix. Test Plan: USE_CLANG=1 make all -j Reviewers: igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D48489 | 10 October 2015, 02:37:51 UTC |
| 3d07b81 | Alexey Maykov | 15 September 2015, 16:03:08 UTC | Passing table properties to compaction callback Summary: It would be nice to have and access to table properties in compaction callbacks. In MyRocks project, it will make possible to update optimizer statistics online. Test Plan: ran the unit test. Ran myrocks with the new way of collecting stats. Reviewers: igor, rven, yhchiang Reviewed By: yhchiang Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D48267 | 10 October 2015, 01:10:55 UTC |
| 64546af | krad | 10 October 2015, 00:54:20 UTC | Adding parser to CI jobs Summary: The parser will help parse the output in order to send meaningful notifications. Test Plan: Manual testing Reviewers: CC: leveldb@ Task ID: #6968635 Blame Rev: | 10 October 2015, 00:58:31 UTC |
| def74f8 | agiardullo | 28 September 2015, 19:12:17 UTC | Deferred snapshot creation in transactions Summary: Support for Transaction::CreateSnapshotOnNextOperation(). This is to fix a write-conflict race-condition that Yoshinori was running into when testing MyRocks with LinkBench. Test Plan: New tests Reviewers: yhchiang, spetrunia, rven, igor, yoshinorim, sdong Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D48099 | 09 October 2015, 22:46:16 UTC |
| c5f3707 | agiardullo | 09 October 2015, 20:31:10 UTC | DisableIndexing() for Transactions Summary: MyRocks reported some perfomance issues when inserting many keys into a transaction due to the cost of inserting new keys into WriteBatchWithIndex. Frequently, they don't even need the keys to be indexed as they don't need to read them back. DisableIndexing() can be used to avoid the cost of indexing. I also plan on eventually investigating if we can improve WriteBatchWithIndex performance. But even if we improved the perf here, it is still beneficial to be able to disable the indexing all together for large transactions. Test Plan: unit test Reviewers: igor, rven, yoshinorim, spetrunia, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D48471 | 09 October 2015, 22:36:09 UTC |
| 776bd8d | sdong | 08 October 2015, 23:57:35 UTC | Pass column family ID to table property collector Summary: Pass column family ID through TablePropertiesCollectorFactory::CreateTablePropertiesCollector() so that users can identify which column family this file is for and handle it differently. Test Plan: Add unit test scenarios in tests related to table properties collectors to verify the information passed in is correct. Reviewers: rven, yhchiang, anthony, kradhakrishnan, igor, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: yoshinorim, leveldb, dhruba Differential Revision: https://reviews.facebook.net/D48411 | 09 October 2015, 21:36:51 UTC |
| 5a72227 | James Lent | 09 October 2015, 18:41:40 UTC | Ensure that the compression libraries are statically linked into dynamic libraries included in the Java jar. Also build the linux libraries using the portable flag to fix a problem with the linux32 build and improve the general portability of the RocksDB dynamic libraries. ==> linux32: util/crc32c.cc:318:39: error: ‘_mm_crc32_u64’ was not declared in this scope | 09 October 2015, 18:41:40 UTC |
| e61d9c1 | sdong | 09 October 2015, 01:00:47 UTC | Make DBTest.AggregatedTableProperties more deterministic Summary: Now based on environment, DBTest.AggregatedTableProperties has a possibility of issuing a L0->L1 compaction after reopening and the results are not what we expected. We tune the L0 compaction trigger to make it less likely to happen. Test Plan: I can't repro the failure but I think the change is better. Just run the test and make sure it passes. Reviewers: kradhakrishnan, yhchiang, igor Reviewed By: igor Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D48423 | 09 October 2015, 16:47:56 UTC |
| b77eb16 | sdong | 03 October 2015, 00:32:46 UTC | New Manifest format to allow customized fields in NewFile. Summary: With this commit, we add a new format in manifest when adding a new file. Now path ID and need-compaction hint are first two customized fields. Test Plan: Add a test case in version_edit_test to verify the encoding and decoding logic. Add a unit test in db_test to verify need compaction is persistent after DB restarting. Reviewers: kradhakrishnan, anthony, IslamAbdelRahman, yhchiang, rven, igor Reviewed By: igor Subscribers: javigon, leveldb, dhruba Differential Revision: https://reviews.facebook.net/D48123 | 08 October 2015, 22:51:45 UTC |
| 6732a57 | Igor Canadi | 08 October 2015, 22:00:05 UTC | Merge pull request #756 from viveknelamangala/master Install snappy headers to standard locations using yum, so that build… | 08 October 2015, 22:00:05 UTC |
| a52888e | Vivek Nelamangala | 08 October 2015, 21:52:22 UTC | Install snappy headers to standard locations using yum, so that build_tools/build_detect_platform sets -DSNAPPY flag to g++ . Current jars of rocksdb do no have snappy compression avaliable . | 08 October 2015, 21:52:22 UTC |
| 831101b | Igor Canadi | 08 October 2015, 21:11:32 UTC | Make it harder for users to run debug builds in production Summary: I see a lot of users compiling RocksDB with `make` or `make all` and then using those binaries in production. They end up running debug builds :( This diff makes it harder for them: 1. I added an explicit warning to INSTALL.md 2. When you compile with `make all`, your resulting library will be librocksdb_debug.a 3. I also print out a warning when you compile in debug mode. Hopefully should be enough :) Test Plan: none Reviewers: rven, yhchiang, kradhakrishnan, anthony, sdong, dhruba, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D48093 | 08 October 2015, 21:11:32 UTC |
| 000836a | sdong | 08 October 2015, 01:10:39 UTC | CompactionFilter::Context to contain column family ID Summary: Add the column family ID to compaction filter context, so it is easier for compaction filter to apply different logic for different column families. Test Plan: Add a unit test to verify the column family ID passed is correct. Reviewers: rven, yhchiang, igor Reviewed By: igor Subscribers: yoshinorim, leveldb, dhruba Differential Revision: https://reviews.facebook.net/D48357 | 08 October 2015, 18:27:38 UTC |
| 3a0bf87 | Yueh-Hsuan Chiang | 08 October 2015, 18:15:18 UTC | Change RocksDB version to 4.1 Summary: Change RocksDB version to 4.1 Test Plan: no code change. Reviewers: sdong, anthony, IslamAbdelRahman, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D48387 | 08 October 2015, 18:15:18 UTC |
| 77318ee | krad | 08 October 2015, 17:13:01 UTC | Enable crash CI jobs Summary: Enabling CI crash test jobs Test Plan: Manual testing Reviewers: CC: leveldb@ Task ID: #8635140 Blame Rev: | 08 October 2015, 17:14:43 UTC |
| 9803e0d | Igor Canadi | 08 October 2015, 16:32:50 UTC | compaction_filter.h cleanup Summary: Two changes: 1. remove *V2 filter stuff. we deprecated that a while ago 2. clarify what happens when user sets max_subcompactions to bigger than 1 Test Plan: none Reviewers: yhchiang, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D47871 | 08 October 2015, 16:32:50 UTC |
| 51fa7ec | Islam AbdelRahman | 07 October 2015, 22:17:20 UTC | Bytes read/written from cache statistics Summary: Add 2 new counters BLOCK_CACHE_BYTES_WRITE, BLOCK_CACHE_BYTES_READ to keep track of how many bytes were written to the cache and how many bytes that we read from cache Test Plan: make check Reviewers: sdong, yhchiang, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D48195 | 07 October 2015, 22:17:20 UTC |
| f925208 | Igor Canadi | 07 October 2015, 21:46:18 UTC | Create Makefile target unity_test Summary: unity_test will compile db_test with unity.a instead of librocksdb.a. This will test both the compilation and some small amount of runtime. Test Plan: This is a test :) Reviewers: kradhakrishnan Reviewed By: kradhakrishnan Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D48297 | 07 October 2015, 21:46:18 UTC |
| a065cdb | dyniusz | 07 October 2015, 18:23:20 UTC | bloom hit/miss stats for SST and memtable Summary: hit and miss bloom filter stats for memtable and SST stats added to perf_context struct key matches and prefix matches combined into one stat Test Plan: unit test veryfing the functionality added, see BloomStatsTest in db_test.cc for details Reviewers: yhchiang, igor, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D47859 | 07 October 2015, 18:23:20 UTC |
| 40cdf79 | Igor Canadi | 07 October 2015, 18:02:23 UTC | Fix compile error on platforms without fallocate() Summary: If a platform doesn't have ROCKSDB_FALLOCATE_PRESENT, then compiler complains: util/env_posix.cc:354:8: error: private field 'allow_fallocate_' is not used [-Werror,-Wunused-private-field] This was caught by travis. Test Plan: compiles with ROCKSDB_FALLOCATE_PRESENT. Reviewers: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D48327 | 07 October 2015, 18:02:23 UTC |
| 77e4ad7 | Igor Canadi | 07 October 2015, 17:17:47 UTC | Fix compile failure on Travis Summary: Travis is complaining against using {} to initialize KVMap: https://travis-ci.org/facebook/rocksdb/jobs/84132600 db/compaction_job_test.cc:526:26: error: chosen constructor is explicit in copy-initialization RunCompaction({files}, {}); This diff should fix it Test Plan: travis Reviewers: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D48309 | 07 October 2015, 17:17:47 UTC |
| 4049bcd | Lakshmi Narayanan | 07 October 2015, 17:04:05 UTC | Added boolean variable to guard fallocate() calls Summary: Added boolean variable to guard fallocate() calls. Set to false to prevent space leaks when tests fail. Test Plan: Compliles Set to false and ran log device tests Reviewers: sdong, lovro, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D48027 | 07 October 2015, 17:04:05 UTC |
| aadf49f | Igor Canadi | 07 October 2015, 16:41:21 UTC | Travis shouldn't fail when clang-format suggests improvements Summary: This diff avoids failing travis test when clang-format fails. We should keep the test green even if clang-format fails. It would be good to look at travis output for pull requests to make sure they're following the good code style. However, we broke travis test 3 times today because of formatting issues. We should keep travis test green if the only thing wrong is the formatting. Test Plan: none Reviewers: sdong, yhchiang, anthony Reviewed By: anthony Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D48291 | 07 October 2015, 16:41:21 UTC |
| d80ce7f | Igor Canadi | 07 October 2015, 16:30:03 UTC | Compaction filter on merge operands Summary: Since Andres' internship is over, I took over https://reviews.facebook.net/D42555 and rebased and simplified it a bit. The behavior in this diff is a bit simpler than in D42555: * only merge operators are passed through FilterMergeValue(). If fitler function returns true, the merge operator is ignored * compaction filter is *not* called on: 1) results of merge operations and 2) base values that are getting merged with merge operands (the second case was also true in previous diff) Do we also need a compaction filter to get called on merge results? Test Plan: make && make check Reviewers: lovro, tnovak, rven, yhchiang, sdong Reviewed By: sdong Subscribers: noetzli, kolmike, leveldb, dhruba, sdong Differential Revision: https://reviews.facebook.net/D47847 | 07 October 2015, 16:30:03 UTC |
| 726d9ce | krad | 07 October 2015, 00:50:04 UTC | Disabling unity Summary: Unity CI job is broken. Disabling. Test Plan: Manual test Reviewers: CC: leveldb@ Task ID: #8632483 Blame Rev: | 07 October 2015, 00:51:31 UTC |
| 0267502 | dyniusz | 07 October 2015, 00:46:22 UTC | Support for LevelDB SST with .ldb suffix Summary: Handle SST files with both ".sst" and ".ldb" suffix. This enables user to migrate from leveldb to rocksdb. Test Plan: Added unit test with DB operating on SSTs with names schema. See db/dc_test.cc:SSTsWithLdbSuffixHandling for details Reviewers: yhchiang, sdong, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D48003 | 07 October 2015, 00:46:22 UTC |
| 5855cdb | Siying Dong | 06 October 2015, 23:41:29 UTC | Merge pull request #750 from yuslepukhin/fixup_build_options FIx up build options | 06 October 2015, 23:41:29 UTC |
| 7bbe10c | Igor Canadi | 06 October 2015, 23:34:43 UTC | Merge pull request #751 from yuslepukhin/return_noerror Mmap reads should not return error if reading past file | 06 October 2015, 23:34:43 UTC |
| e95b703 | Dmitri Smirnov | 06 October 2015, 23:19:58 UTC | Mmap reads should not return error if reading past file Summary: This mirrors https://reviews.facebook.net/D45645 Currently, mmap returns IOError when user tries to read data past the end of the file. This diff changes the behavior. Now, we return just the bytes that we can, and report the size we returned via a Slice result. This is consistent with non-mmap behavior and also pread() system call. | 06 October 2015, 23:19:58 UTC |
| 25c58a2 | Dmitri Smirnov | 06 October 2015, 22:52:09 UTC | Add shared_linked DEBUG flag, remove port from among the include directories. | 06 October 2015, 22:52:09 UTC |
| 60b1c0a | krad | 06 October 2015, 21:21:10 UTC | Fix to CI job definition Summary: Sandcastle does not raise a task if assignee is specified. Removing the assignee will force the system to use the oncall. Test Plan: Manual Reviewers: CC: leveldb@ Task ID: #6968635 Blame Rev: | 06 October 2015, 21:21:10 UTC |
