aea3ce4 | Bassam Tabbara | 11 October 2016, 00:32:04 UTC | Avoid string CONCAT which is not supported in cmake 2.6 (#1383) Signed-off-by: Bassam Tabbara <bassam.tabbara@quantum.com> | 11 October 2016, 00:32:04 UTC |
2ad68b9 | Islam AbdelRahman | 08 October 2016, 00:21:45 UTC | Support running consistency checks in release mode Summary: We always run consistency checks when compiling in debug mode allow users to set Options::force_consistency_checks to true to be able to run such checks even when compiling in release mode Test Plan: make check -j64 make release Reviewers: lightmark, sdong, yiwu Reviewed By: yiwu Subscribers: hermanlee4, andrewkr, yoshinorim, jkedgar, dhruba Differential Revision: https://reviews.facebook.net/D64701 | 08 October 2016, 00:21:45 UTC |
67501cf | Islam AbdelRahman | 08 October 2016, 00:16:13 UTC | Fix -ve std::string::resize Summary: I saw this exception thrown because sometimes we may resize with -ve value if we have empty max_bytes_for_level_multiplier_additional vector Test Plan: run the tests Reviewers: yiwu Reviewed By: yiwu Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D64791 | 08 October 2016, 00:16:13 UTC |
04b02dd | Joel Marcey | 07 October 2016, 23:28:44 UTC | Testing asset links after config change | 07 October 2016, 23:28:44 UTC |
8c55bb8 | Reid Horuff | 06 October 2016, 18:52:57 UTC | Make Lock Info test multiple column families Summary: Modifies the lock info export test to test multiple column families after I was experiencing a bug while developing the MyRocks front-end for this. Test Plan: is test. Reviewers: mung Reviewed By: mung Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D64725 | 07 October 2016, 22:04:05 UTC |
d062328 | Islam AbdelRahman | 07 October 2016, 20:59:16 UTC | Revert "Support SST files with Global sequence numbers" This reverts commit ab01da5437385e3142689077c647a3b13ba3402f. | 07 October 2016, 21:05:12 UTC |
5cd2883 | Adam Retter | 07 October 2016, 19:32:21 UTC | [RocksJava] Adjusted RateLimiter to 3.10.0 (#1368) Summary: - Deprecated RateLimiterConfig and GenericRateLimiterConfig - Introduced RateLimiter It is now possible to use all C++ related methods also in RocksJava. A noteable method is setBytesPerSecond which can change the allowed number of bytes per second at runtime. Test Plan: make rocksdbjava make jtest Reviewers: adamretter, yhchiang, ankgup87 Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D35715 | 07 October 2016, 19:32:21 UTC |
37737c3 | Reid Horuff | 05 October 2016, 20:39:00 UTC | Expose Transaction State Publicly Summary: This exposes a transactions state through a public api rather than through a public member variable. I also do some name refactoring. ExecutionStatus => TransactionState exec_status_ => trx_state_ Test Plan: It compiles and transaction_test passes. Reviewers: IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: andrewkr, mung, dhruba, sdong Differential Revision: https://reviews.facebook.net/D64689 | 07 October 2016, 18:58:53 UTC |
2c1f952 | Reid Horuff | 07 October 2016, 18:31:26 UTC | Add facility to write only a portion of WriteBatch to WAL Summary: When constructing a write batch a client may now call MarkWalTerminationPoint() on that batch. No batch operations after this call will be added written to the WAL but will still be inserted into the Memtable. This facility is used to remove one of the three WriteImpl calls in 2PC transactions. This produces a ~1% perf improvement. ``` RocksDB - unoptimized 2pc, sync_binlog=1, disable_2pc=off INFO 2016-08-31 14:30:38,814 [main]: REQUEST PHASE COMPLETED. 75000000 requests done in 2619 seconds. Requests/second = 28628 RocksDB - optimized 2pc , sync_binlog=1, disable_2pc=off INFO 2016-08-31 16:26:59,442 [main]: REQUEST PHASE COMPLETED. 75000000 requests done in 2581 seconds. Requests/second = 29054 ``` Test Plan: Two unit tests added. Reviewers: sdong, yiwu, IslamAbdelRahman Reviewed By: yiwu Subscribers: hermanlee4, dhruba, andrewkr Differential Revision: https://reviews.facebook.net/D64599 | 07 October 2016, 18:32:10 UTC |
043cb62 | Peter (Stig) Edwards | 06 October 2016, 17:45:31 UTC | Fix record_size in log_write_bench, swap args to std::string::assign. (#1373) Hello and thank you for RocksDB, I noticed when using log_write_bench that writes were always 88 bytes: > strace -e trace=write ./log_write_bench -num_records 2 2>&1 | head -n 2 write(3, "\371\371\371\371\371\371\371\371\371\371\371\371\371\371\371\371\371\371\371\371\371\371\371\371\371\371\371\371\371\371\371\371"..., 88) = 88 write(3, "\371\371\371\371\371\371\371\371\371\371\371\371\371\371\371\371\371\371\371\371\371\371\371\371\371\371\371\371\371\371\371\371"..., 88) = 88 > strace -e trace=write ./log_write_bench -record_size 4096 -num_records 2 2>&1 | head -n 2 write(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 88) = 88 write(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 88) = 88 I think this should be: << record.assign('X', FLAGS_record_size); >> record.assign(FLAGS_record_size, 'X'); So fill and not buffer. Otherwise I always see writes of size 88 (the decimal value for chr "X"). string& assign (const char* s, size_t n); buffer - Copies the first n characters from the array of characters pointed by s. string& assign (size_t n, char c); fill - Replaces the current value by n consecutive copies of character c. perl -le 'print ord "X"' 88 With the change: > strace -e trace=write ./log_write_bench -record_size 4096 -num_records 2 2>&1 | head -n 2 write(3, "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"..., 4096) = 4096 write(3, "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"..., 4096) = 4096 > strace -e trace=write ./log_write_bench -num_records 2 2>&1 | head -n 2 write(3, "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"..., 249) = 249 write(3, "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"..., 249) = 249 Thanks. https://github.com/facebook/rocksdb/commit/01c27be5fb42524c5052b4b4a23e05501e1d1421 https://reviews.facebook.net/D16239 | 06 October 2016, 17:45:31 UTC |
4985f60 | Sage Weil | 06 October 2016, 17:43:05 UTC | env_mirror: fix a few leaks (#1363) * env_mirror: fix leak from LockFile Signed-off-by: Sage Weil <sage@redhat.com> * env_mirror: instruct EnvMirror whether mirrored Envs should be destroyed The lifecycle rules for Env are frustrating and undocumented. Notably, Env::Default() should *not* be freed, but any Env instances we created should be. Explicitly instruct EnvMirror whether to clean up child Env instances. Default to false so that we do not affect existing callers. Signed-off-by: Sage Weil <sage@redhat.com> | 06 October 2016, 17:43:05 UTC |
5aded67 | Igor Mihalik | 06 October 2016, 17:37:19 UTC | update of c.h (#1371) Added rocksdb_options_set_memtable_prefix_bloom_size_ratio function implemented in c.cc but not exported via c.h | 06 October 2016, 17:37:19 UTC |
912aec1 | Joel Marcey | 05 October 2016, 17:29:11 UTC | "Recent Posts" -> "All Posts" Blog sidebar shows all the posts, not just the most recent ones. | 05 October 2016, 17:29:11 UTC |
7cbb298 | Joel Marcey | 04 October 2016, 22:38:41 UTC | Make sure that when contribtuing we call out creating appropriate directories .... if they do not exist | 04 October 2016, 22:38:41 UTC |
a06ad47 | Joel Marcey | 04 October 2016, 22:27:28 UTC | Add top level doc information to CONTRIBUTING.md | 04 October 2016, 22:27:28 UTC |
3fdd5b9 | Joel Marcey | 04 October 2016, 22:22:28 UTC | A little more generic CONTRIBUTING.md | 04 October 2016, 22:22:28 UTC |
ed4fc31 | Joel Marcey | 04 October 2016, 22:21:43 UTC | Add link to CONTRIBUTING.md to main docs README.md | 04 October 2016, 22:21:43 UTC |
e4922e1 | Joel Marcey | 04 October 2016, 22:20:15 UTC | Forgot to truncate one blog post | 04 October 2016, 22:20:15 UTC |
6d8cd7e | Joel Marcey | 04 October 2016, 22:19:00 UTC | Add CONTRIBUTING.md for rocksdb.org contribution guidance | 04 October 2016, 22:19:00 UTC |
bd55e5a | Joel Marcey | 04 October 2016, 21:33:07 UTC | Fix some formatting of compaction blog post | 04 October 2016, 21:33:07 UTC |
0f60358 | Joel Marcey | 04 October 2016, 21:31:36 UTC | CRLF -> LF mod (including removing trailing whitespace for those files) | 04 October 2016, 21:31:36 UTC |
b90e29c | Joel Marcey | 04 October 2016, 21:20:26 UTC | Truncate posts on the main /blog/ page | 04 October 2016, 21:20:26 UTC |
0d7acad | Joel Marcey | 04 October 2016, 21:11:04 UTC | Add author fields to blog posts Now the author associated with fbid will be shown at top of blog post | 04 October 2016, 21:11:04 UTC |
01be441 | Joel Marcey | 04 October 2016, 20:49:13 UTC | Add GitHub link to the landing page header | 04 October 2016, 20:49:13 UTC |
9d6c961 | Islam AbdelRahman | 04 October 2016, 01:25:10 UTC | Fix Mac build | 04 October 2016, 01:25:10 UTC |
ab01da5 | Islam AbdelRahman | 03 October 2016, 23:12:39 UTC | Support SST files with Global sequence numbers Summary: - Update SstFileWriter to include a property for a global sequence number in the SST file `rocksdb.external_sst_file.global_seqno` - Update TableProperties to be aware of the offset of each property in the file - Update BlockBasedTableReader and Block to be able to honor the sequence number in `rocksdb.external_sst_file.global_seqno` property and use it to overwrite all sequence number in the file Something worth mentioning is that we don't update the seqno in the index block since and when doing a binary search, the reason for that is that it's guaranteed that SST files with global seqno will have only one user_key and each key will have seqno=0 encoded in it, This mean that this key is greater than any other key with seqno> 0. That mean that we can actually keep the current logic for these blocks Test Plan: unit tests Reviewers: andrewkr, yhchiang, yiwu, sdong Reviewed By: sdong Subscribers: hcz, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D62523 | 03 October 2016, 23:12:39 UTC |
d346ba2 | Adam Retter | 03 October 2016, 18:58:08 UTC | Minor fixes around Windows 64 Java Artifacts (#1366) | 03 October 2016, 18:58:08 UTC |
e91b4d0 | krad | 30 September 2016, 01:02:13 UTC | Add factory method for creating persistent cache that is accessible from public Summary: Currently there is no mechanism to create persistent cache from headers. Adding a simple factory method to create a simple persistent cache with default or NVM optimized settings. note: Any idea to test this factory is appreciated. Test Plan: None Reviewers: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D64527 | 03 October 2016, 17:55:46 UTC |
be1f109 | Manuel Ung | 28 September 2016, 00:43:06 UTC | Expose transaction id, lock state information and transaction wait information Summary: This diff does 3 things: Expose TransactionID so that we can identify transactions when we retrieve locking and lock wait information. This is exposed as `Transaction::GetID`. Expose lock state information by locking all stripes in all column families and copying their contents to a data structure. This is exposed as `TransactionDB::GetLockStatusData`. Adds support for tracking the transaction and the key being waited on, and exposes this as `Transaction::GetWaitingTxn`. Test Plan: unit tests Reviewers: horuff, sdong Reviewed By: sdong Subscribers: vasilep, hermanlee4, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D64413 | 30 September 2016, 18:41:21 UTC |
6009c47 | Andrew Kryczka | 12 September 2016, 21:14:40 UTC | Store range tombstones in memtable Summary: - Store range tombstones in a separate MemTableRep instantiated with ColumnFamilyOptions::memtable_factory - MemTable::NewRangeTombstoneIterator() returns a MemTableIterator over the separate MemTableRep - Part of the read path is not implemented yet (i.e., MemTable::Get()) Test Plan: see unit tests Reviewers: wanning Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D62217 | 30 September 2016, 16:06:43 UTC |
3c21c64 | Adam Retter | 29 September 2016, 22:55:53 UTC | Use size hint for HashMap in multiGet. Similar to https://github.com/facebook/rocksdb/pull/1344 (#1367) | 29 September 2016, 22:55:53 UTC |
13f7a01 | Tomas Kolda | 29 September 2016, 21:11:32 UTC | Fixing JNI release build for gcc (#975) | 29 September 2016, 21:11:32 UTC |
7260662 | Injun Song | 29 September 2016, 21:04:41 UTC | Add Java API for SstFileWriter Add Java API for SstFileWriter. Closes https://github.com/facebook/rocksdb/issues/1248 | 29 September 2016, 21:04:41 UTC |
2638824 | Aaron Gao | 29 September 2016, 20:19:58 UTC | delete unused variable for PrevInterval() Summary: delete unused variable Test Plan: make check Reviewers: sdong, andrewkr, IslamAbdelRahman, tianx Reviewed By: tianx Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D64509 | 29 September 2016, 20:19:58 UTC |
87dfc1d | Islam AbdelRahman | 28 September 2016, 22:42:06 UTC | Fix conflict between AddFile() and CompactRange() Summary: Fix the conflict bug between AddFile() and CompactRange() by - Make sure that no AddFile calls are running when asking CompactionPicker to pick compaction for manual compaction - If AddFile() run after we pick the compaction for the manual compaction it will be aware of it since we will add the manual compaction to running_compactions_ after picking it This will solve these 2 scenarios - If AddFile() is running, we will wait for it to finish before we pick a compaction for the manual compaction - If we already picked a manual compaction and then AddFile() started ... we ensure that it never ingest a file in a level that will overlap with the manual compaction Test Plan: unit tests Reviewers: sdong Reviewed By: sdong Subscribers: andrewkr, yoshinorim, jkedgar, dhruba Differential Revision: https://reviews.facebook.net/D64449 | 28 September 2016, 22:42:06 UTC |
eb44ed6 | Siying Dong | 28 September 2016, 20:29:36 UTC | Update 2016-09-28-rocksdb-4-11-2-released.markdown | 28 September 2016, 20:29:36 UTC |
e443761 | Siying Dong | 28 September 2016, 20:28:59 UTC | Update 2016-09-28-rocksdb-4-11-2-released.markdown | 28 September 2016, 20:28:59 UTC |
501f051 | Siying Dong | 28 September 2016, 20:28:18 UTC | Update 2016-09-28-rocksdb-4-11-2-released.markdown | 28 September 2016, 20:28:18 UTC |
dec9009 | Siying Dong | 28 September 2016, 20:27:19 UTC | Update 2016-09-28-rocksdb-4-11-2-released.markdown | 28 September 2016, 20:27:19 UTC |
4ed69dd | Siying Dong | 28 September 2016, 20:24:27 UTC | Create 2016-09-28-rocksdb-4-11-2-released.markdown | 28 September 2016, 20:24:27 UTC |
21f4bb5 | Kefu Chai | 28 September 2016, 18:53:15 UTC | cmake support for linux and osx (#1358) * enable cmake to work on linux and osx also * port part of build_detect_platform not covered by thirdparty.inc to cmake. - detect fallocate() - detect malloc_usable_size() - detect JeMalloc - detect snappy * check for asan,tsan,ubsan * create 'build_version.cc' in build directory. * add `check` target to support 'make check'. * add `tools` target to match its counterpart in Makefile. * use `date` on non-win32 platforms. * pass different cflags on non-win32 platforms * detect pthead library using FindThread cmake module. * enable CMP0042 to silence the cmake warning on osx * reorder the linked libraries. because testutillib references gtest, to enable the linker to find the referenced symbols, we need to put gtest after testutillib. Signed-off-by: Marcus Watts <mwatts@redhat.com> Signed-off-by: Kefu Chai <kchai@redhat.com> * hash_table_bench.cc: fix build without gflags Signed-off-by: Kefu Chai <kchai@redhat.com> * remove gtest from librocksdb linkage testharness.cc is included in librocksdb sources, and it uses gtest. but gtest is not supposed to be part of the public API of librocksdb. so, in this change, the testharness.cc is moved out out librocksdb, and is built as an object target, then linked with the tools and tests instead. Signed-off-by: Marcus Watts <mwatts@redhat.com> Signed-off-by: Kefu Chai <kchai@redhat.com> | 28 September 2016, 18:53:15 UTC |
4defe30 | wenduo | 28 September 2016, 03:39:15 UTC | fix typo in comments (#1360) * fix typo in option.h's comment * fix typo in checkpoint's comment | 28 September 2016, 03:39:15 UTC |
f517d9d | Aaron Gao | 28 September 2016, 01:20:57 UTC | Add SeekForPrev() to Iterator Summary: Add new Iterator API, `SeekForPrev`: find the last key that <= target key support prefix_extractor support prefix_same_as_start support upper_bound not supported in iterators without Prev() Also add tests in db_iter_test and db_iterator_test Pass all tests Cheers! Test Plan: make all check -j64 Reviewers: andrewkr, yiwu, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D64149 | 28 September 2016, 01:20:57 UTC |
eb3894c | yiwu-arbug | 27 September 2016, 18:17:15 UTC | Recompute compaction score on SetOptions (#1346) Summary: We didn't recompute compaction score on SetOptions, and end up not having compaction if no flush happens afterward. The PR fixing it. Test Plan: See unit test. Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D64167 | 27 September 2016, 18:17:15 UTC |
5c64fb6 | Islam AbdelRahman | 26 September 2016, 21:44:48 UTC | Fix AddFile() conflict with compaction output [WaitForAddFile()] Summary: Since AddFile unlock/lock the mutex inside LogAndApply() we need to ensure that during this period other compactions cannot run since such compactions are not aware of the file we are ingesting and could create a compaction that overlap wit this file this diff add - WaitForAddFile() call that will ensure that no AddFile() calls are being processed right now - Call `WaitForAddFile()` in 3 locations -- When doing manual Compaction -- When starting automatic Compaction -- When doing CompactFiles() Test Plan: unit test Reviewers: lightmark, yiwu, andrewkr, sdong Reviewed By: sdong Subscribers: andrewkr, yoshinorim, jkedgar, dhruba Differential Revision: https://reviews.facebook.net/D64383 | 27 September 2016, 07:14:55 UTC |
9e9f5a0 | Islam AbdelRahman | 26 September 2016, 17:39:07 UTC | Fix CompactFilesTest.ObsoleteFiles timeout (#1353) | 26 September 2016, 17:39:07 UTC |
c2a62a4 | Aaron Gao | 24 September 2016, 00:16:38 UTC | not cut compaction output when compact to level 0 Summary: we should not call ShouldStopBefore() in compaction when the compaction targets level 0. Otherwise, CheckConsistency will fail the assertion of seq number check on level 0. Test Plan: make all check -j64 I also manully test that using db_bench to compact files to level 0. Without this line change, the assertion files and multiple files are generated on level 0 after compaction. Reviewers: yhchiang, andrewkr, yiwu, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D64269 | 24 September 2016, 00:16:38 UTC |
9ed928e | Yi Wu | 23 September 2016, 23:34:04 UTC | Split DBOptions into ImmutableDBOptions and MutableDBOptions Summary: Use ImmutableDBOptions/MutableDBOptions internally and DBOptions only for user-facing APIs. MutableDBOptions is barely a placeholder for now. I'll start to move options to MutableDBOptions in following diffs. Test Plan: make all check Reviewers: yhchiang, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D64065 | 23 September 2016, 23:34:04 UTC |
4bc8c88 | yiwu-arbug | 23 September 2016, 23:15:14 UTC | Recover same sequence id from WAL (#1350) Summary: Revert the behavior where we don't read sequence id from WAL, but increase it as we replay the log. We still keep the behave for 2PC for now but will fix later. This change fixes github issue 1339, where some writes come with WAL disabled and we may recover records with wrong sequence id. Test Plan: Added unit test. Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D64275 | 23 September 2016, 23:15:14 UTC |
0a1bd9c | Aaron Gao | 22 September 2016, 18:56:18 UTC | add cfh deletion started listener Summary: add ColumnFamilyHandleDeletionStarted listener which can be called when user deletes handler. Test Plan: ./listener_test Reviewers: yiwu, IslamAbdelRahman, sdong, andrewkr Reviewed By: andrewkr Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D60717 | 22 September 2016, 18:56:18 UTC |
da5a9a6 | Islam AbdelRahman | 22 September 2016, 03:22:09 UTC | Fix mac build | 22 September 2016, 03:22:09 UTC |
d45eb6c | Giuseppe Ottaviano | 22 September 2016, 03:06:56 UTC | Fix typo (#1349) | 22 September 2016, 03:06:56 UTC |
abc0ae4 | Islam AbdelRahman | 21 September 2016, 21:24:08 UTC | Add AddFile() InternalStats for Total files/L0 files/total keys ingested Summary: Report more information about the ingested files in CF InternalStats - Total files - Total L0 files - Total keys There was also noticed that we were reporting files that failed to ingest, fix this bug Test Plan: print stats in tests Reviewers: sdong, andrewkr, lightmark Reviewed By: lightmark Subscribers: jkedgar, andrewkr, dhruba, yoshinorim Differential Revision: https://reviews.facebook.net/D63039 | 21 September 2016, 21:24:08 UTC |
7152563 | Aaron Gao | 21 September 2016, 18:05:07 UTC | forbid merge during recovery Summary: Mitigate regression bug of options.max_successive_merges hit during DB Recovery For https://reviews.facebook.net/D62625 Test Plan: make all check Reviewers: horuff, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D62655 | 21 September 2016, 18:05:07 UTC |
5735b3d | Islam AbdelRahman | 20 September 2016, 20:02:41 UTC | Fix compiling under -Werror=missing-field-initializers Summary: MyRocks build is broken because they are using "-Werror=missing-field-initializers" We should fix that by explicitly passing these arguments Test Plan: Build MyRocks Reviewers: sdong, yiwu Reviewed By: yiwu Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D64161 | 20 September 2016, 20:02:41 UTC |
654ed9a | Aaron Gao | 20 September 2016, 19:28:59 UTC | loose the assertion condition of rate_limiter_test Summary: 0.9 can make the test flaky since just found one test fail with 0.88 Test Plan: make all check Reviewers: sdong, andrewkr Reviewed By: andrewkr Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D63939 | 20 September 2016, 19:28:59 UTC |
e4d3f5d | Yi Wu | 20 September 2016, 17:17:28 UTC | Fix DBImpl::GetWalPreallocateBlockSize Mac build error Summary: Specify type param with std::min to resolve compile error on Mac. Test Plan: https://travis-ci.org/facebook/rocksdb/builds/161223845 Reviewers: sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D64143 | 20 September 2016, 17:17:28 UTC |
7afbb74 | panfengfeng | 20 September 2016, 17:11:51 UTC | solve the problem of table_factory_to_write_=nullptr (#1342) | 20 September 2016, 17:11:51 UTC |
d78a440 | sdong | 19 September 2016, 23:21:35 UTC | DBImpl::GetWalPreallocateBlockSize() should return size_t Summary: WritableFile::SetPreallocationBlockSize() requires parameter as size_t, and options used in DBImpl::GetWalPreallocateBlockSize() are all size_t. WritableFile::SetPreallocationBlockSize() should return size_t to avoid build break if size_t is not uint64_t. Test Plan: Run existing tests. Reviewers: andrewkr, IslamAbdelRahman, yiwu Reviewed By: yiwu Subscribers: leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D64137 | 19 September 2016, 23:51:38 UTC |
42ac9c5 | Islam AbdelRahman | 19 September 2016, 21:50:52 UTC | Retry getting arcanist token on failure Summary: Many of our diffs dont have sandcastle tests because we failed to load arcanist token file (loaded over the network) this diff try for at least 5 seconds (once every 0.2 second) to load the file instead of failing the first time the file is not found This will make it less probable that diffs are submitted without sandcastle tests Test Plan: arc diff --preview Reviewers: kradhakrishnan, gunnarku, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D63951 | 19 September 2016, 21:50:52 UTC |
b666f85 | sdong | 13 September 2016, 23:38:28 UTC | Consider more factors when determining preallocation size of WAL files Summary: Currently the WAL file preallocation size is 1.1 * write_buffer_size. This, however, will be over-estimated if options.db_write_buffer_size or options.max_total_wal_size is set and is much smaller. Test Plan: Add a unit test. Reviewers: andrewkr, yiwu Reviewed By: yiwu Subscribers: leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D63957 | 19 September 2016, 19:04:35 UTC |
4c3f449 | rockeet | 18 September 2016, 05:30:43 UTC | Add TableBuilderOptions::level and relevant changes (#1335) | 18 September 2016, 05:30:43 UTC |
3edb946 | sdong | 06 September 2016, 19:28:55 UTC | Avoid hard-coded sleep in EnvPosixTestWithParam.TwoPools Summary: EnvPosixTestWithParam.TwoPools relies on explicit sleeping, so it sometimes fail. Fix it. Test Plan: Run tests with high parallelism many times and make sure the test passes. Reviewers: yiwu, andrewkr Reviewed By: andrewkr Subscribers: leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D63417 | 17 September 2016, 00:45:12 UTC |
0a88f38 | Yi Wu | 16 September 2016, 22:09:14 UTC | Remove ColumnFamilyData::options() Summary: One more small refactor before I split DBOptions into mutable and immutable parts. Test Plan: existing unit tests. Reviewers: yhchiang, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D64047 | 16 September 2016, 22:09:14 UTC |
41a9070 | Yi Wu | 16 September 2016, 17:54:31 UTC | Fix java makefile dependencies Summary: Fix dependencies in java makefile, to avoid java build failure. Test Plan: run "make rocksdbjava" multiple times. Reviewers: IslamAbdelRahman, sdong, yhchiang Reviewed By: yhchiang Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D64023 | 16 September 2016, 17:54:31 UTC |
8d9bf5c | Yi Wu | 15 September 2016, 21:57:32 UTC | Fix DBOptionsTest.GetLatestOptions Summary: RandomInitCFOptions will allocate a new compaction filter, which we have to delete afterward. Test Plan: valgrind against the test Reviewers: IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D64011 | 15 September 2016, 21:57:32 UTC |
40cfa3e | Yi Wu | 15 September 2016, 20:44:33 UTC | Fix DBWALTest.RecoveryWithLogDataForSomeCFs with mac Summary: Seems there's no std::array on mac+clang. Use raw array instead. Test Plan: run ./db_wal_test on mac. Reviewers: andrewkr Reviewed By: andrewkr Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D64005 | 15 September 2016, 20:44:33 UTC |
06b4785 | Andrew Kryczka | 15 September 2016, 18:40:48 UTC | Fix recovery for WALs without data for all CFs Summary: if one or more CFs had no data in the WAL, the log number that's used by FindObsoleteFiles() wasn't updated. We need to treat this case the same as if the data for that WAL had been flushed. Test Plan: new unit test Reviewers: IslamAbdelRahman, yiwu, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D63963 | 15 September 2016, 18:40:48 UTC |
d7242ff | Andrew Kryczka | 15 September 2016, 16:55:02 UTC | Fix GetSortedWalFiles when log recycling enabled Summary: Previously the sequence number was mistakenly passed in an argument where the log number should go. This caused the reader to assume the old WAL format was used, which is incompatible with the WAL recycling format. Test Plan: new unit test, verified it fails before this change and passes afterwards. Reviewers: yiwu, lightmark, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D63987 | 15 September 2016, 16:55:02 UTC |
17f76fc | Yi Wu | 15 September 2016, 05:10:28 UTC | DB::GetOptions() reflect dynamic changed options Summary: DB::GetOptions() reflect dynamic changed options. Test Plan: See the new unit test. Reviewers: yhchiang, sdong, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D63903 | 15 September 2016, 05:10:28 UTC |
215d128 | Jo Liss | 14 September 2016, 21:12:31 UTC | Fix typo (#903) Presumably a leftover from optimistic_transaction_example.cc. | 14 September 2016, 21:12:31 UTC |
a958c26 | Yi Wu | 14 September 2016, 20:12:55 UTC | Rename jvalue to jval in rocksjni Summary: jvalue shadows a global name in <jni.h>. Rename it to jval to fix java build. Test Plan: JAVA_HOME=/usr/local/jdk-7u10-64 make rocksdbjava -j64 Reviewers: adamretter, yhchiang, IslamAbdelRahman Reviewed By: yhchiang, IslamAbdelRahman Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D63981 | 14 September 2016, 20:12:55 UTC |
0a165bd | Joel Marcey | 14 September 2016, 17:42:58 UTC | Have Facebook link point to RocksDB on FB Summary: ...not just the main FB landing page Test Plan: visual Reviewers: lgalanis, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D63969 | 14 September 2016, 17:42:58 UTC |
3639f32 | xh931076284 | 14 September 2016, 17:17:34 UTC | Fix bug in UnScSigned-off-by: xh931076284 <931076284@qq.com> (#1336) Fix HdfsEnv::UnSchedule() API error | 14 September 2016, 17:17:34 UTC |
8e061f9 | Yi Wu | 14 September 2016, 04:12:43 UTC | Refactor GetMutableOptionsFromStrings Summary: Add mutable options info into `OptionsTypeInfo` and use it to parse mutable options map. Also support `max_bytes_for_level_multiplier_additional` in option file. Test Plan: unit test Reviewers: yhchiang, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D63843 | 14 September 2016, 04:12:43 UTC |
81747f1 | Yi Wu | 14 September 2016, 04:11:59 UTC | Refactor MutableCFOptions Summary: * Change constructor of MutableCFOptions to depends only on ColumnFamilyOptions. * Move `max_subcompactions`, `compaction_options_fifo` and `compaction_pri` to ImmutableCFOptions to make it clear that they are immutable. Test Plan: existing unit tests. Reviewers: yhchiang, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D63945 | 14 September 2016, 04:11:59 UTC |
ba65c81 | Islam AbdelRahman | 13 September 2016, 19:08:22 UTC | Support POSIX RandomRWFile Summary: Add Env::RandomRWFile in env.h and implement it for POSIX RandomRWFile is a file that allow us to read from / write to random offsets in the file I will implement it for other Envs later after finishing the whole task for AddFile() Test Plan: unit tests Reviewers: andrewkr, kradhakrishnan, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D62433 | 13 September 2016, 19:08:22 UTC |
1d980a8 | Joel Marcey | 13 September 2016, 16:50:04 UTC | Create CNAME | 13 September 2016, 16:50:04 UTC |
2adab1d | Joel Marcey | 13 September 2016, 16:13:59 UTC | Add API links to the header bar Summary: Until we get new API infra for gh-pages, this will point to the C++ headers and Java files, respectively for API information. Test Plan: Visual https://www.facebook.com/pxlcld/pwzQ Reviewers: lgalanis, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D63771 | 13 September 2016, 16:13:59 UTC |
a182b29 | Joel Marcey | 13 September 2016, 16:13:29 UTC | Preserve blog comments in markdown Summary: While we won't have new comments when we move to gh-pages (for now), this preserves the current comments in markdown format. Test Plan: Visual https://www.facebook.com/pxlcld/pwCR Reviewers: lgalanis, sdong Reviewed By: sdong Subscribers: jamesgpearce, andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D63783 | 13 September 2016, 16:13:29 UTC |
f54de92 | Pawan Rawal | 13 September 2016, 00:33:44 UTC | Adding Dgraph to list of Users (#1291) * Adding Dgraph * open-source | 13 September 2016, 00:33:44 UTC |
9e4aa79 | somnathr | 12 September 2016, 23:53:42 UTC | Summary: (#1313) If log recycling is enabled with the rocksdb (recycle_log_file_num=16) db->Writebatch is erroring out with keynotfound after ~5-6 hours of run (1M seq but can happen to any workload I guess).See my detailed bug report here (https://github.com/facebook/rocksdb/issues/1303). This commit is the fix for this, a check is been added not to delete the log file if it is already there in the recycle list. Test Plan: Unit tested it and ran the similar profile. Not reproducing anymore. | 12 September 2016, 23:53:42 UTC |
a10e8a0 | Adam Faulkner | 12 September 2016, 22:31:42 UTC | Fix C api memtable rep bugs. (#1328) | 12 September 2016, 22:31:42 UTC |
eb1d4d5 | Islam AbdelRahman | 12 September 2016, 19:34:08 UTC | Release RocksDB 4.12 Summary: Release 4.12 Test Plan: none Reviewers: andrewkr, yiwu, lightmark, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D63885 | 12 September 2016, 19:34:08 UTC |
22d88e2 | Adam Retter | 12 September 2016, 18:51:08 UTC | Allow an offset as well as a length to be specified for byte[] operations in RocksJava JNI (#1264) Test Plan: Execute the Java test suite Reviewers: yhchiang Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D61971 | 12 September 2016, 18:51:08 UTC |
b06b191 | zhangjinpeng1987 | 09 September 2016, 17:11:30 UTC | add C api for set wal_recovery_mode (#1327) * add C api for set wal recovery mode * add test | 09 September 2016, 17:11:30 UTC |
1cca091 | Islam AbdelRahman | 08 September 2016, 21:45:32 UTC | Temporarily revert Prev() prefix support Summary: Temporarily revert commits for supporting prefix Prev() to unblock MyRocks and RocksDB release These are the commits reverted - 6a14d55bd913490dbd61d682567e6e0625756c0d - b18f9c9eace89d63f37432ce1a3dba48bddbcef0 - db74b1a21905336e2c178ff1f2ffd12c7852b7b8 - 2482d5fb45d8f01b0b065d649d01f43dacad799c Test Plan: make check -j64 Reviewers: sdong, lightmark Reviewed By: lightmark Subscribers: andrewkr, dhruba, yoshinorim Differential Revision: https://reviews.facebook.net/D63789 | 08 September 2016, 21:45:32 UTC |
de28a25 | Andrew Kryczka | 08 September 2016, 21:02:19 UTC | Update HISTORY.md for thread-local stats Summary: as titled Test Plan: doitlive Reviewers: sdong, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D63777 | 08 September 2016, 21:02:19 UTC |
0fcb6db | ammongit | 08 September 2016, 18:31:06 UTC | Remove extraneous function prototypes from c.h (#1326) * Fix function prototypes from upstream commit 32149059. * Fix removed function. * Readd removed function. | 08 September 2016, 18:31:06 UTC |
52ee07b | Islam AbdelRahman | 07 September 2016, 22:41:54 UTC | Move AddFile() tests to external_sst_file_test.cc Summary: Simply move the tests Test Plan: make check -j64 Reviewers: andrewkr, lightmark, yiwu, yhchiang, kradhakrishnan, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D62529 | 07 September 2016, 22:41:54 UTC |
66a91e2 | Edouard A | 07 September 2016, 19:37:45 UTC | Add NoSpace subcode to IOError (#1320) Add a sub code to distinguish "out of space" errors from regular I/O errors | 07 September 2016, 19:37:45 UTC |
67036c0 | sdong | 06 September 2016, 19:57:46 UTC | Fix Flaky ColumnFamilyTest.FlushCloseWALFiles Summary: In ColumnFamilyTest.FlushCloseWALFiles, there is a small window in which the flush has finished but the log writer is not yet closed, causing the assert failure. Fix it by explicitly waiting the flush job to finish. Test Plan: Run the test many times in high parallelism. Subscribers: leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D63423 | 07 September 2016, 18:19:15 UTC |
0e2da49 | wenduo | 07 September 2016, 08:36:16 UTC | fix typo in option.h's comment (#1321) | 07 September 2016, 08:36:16 UTC |
6d61358 | Joel Marcey | 07 September 2016, 04:10:14 UTC | Add real Google Analytics ID Summary: This adds the actual RocksDB Google Analytics ID that will be used when we switch over from WP to GH-pages Test Plan: visual Reviewers: IslamAbdelRahman, lgalanis, sdong Reviewed By: sdong Subscribers: sdong, andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D63429 | 07 September 2016, 04:10:14 UTC |
2d9d36e | Joel Marcey | 07 September 2016, 04:08:53 UTC | Have "Edit on GitHub" point to master instead of gh-pages Summary: If someone clicks on `Edit on GitHub` to edit a doc for a pull request, for example, then we should point to `master` instead of `gh-pages`. > Also fixed some Windows-based line endings. From CRLF to LF. Test Plan: https://www.facebook.com/pxlcld/pvV2 Reviewers: IslamAbdelRahman, lgalanis, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D63435 | 07 September 2016, 04:08:53 UTC |
9377518 | Joel Marcey | 07 September 2016, 04:07:47 UTC | Update landing page content Summary: Changes based on comments in D62985 Test Plan: Visual https://www.facebook.com/pxlcld/pvVs Reviewers: lgalanis, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D63477 | 07 September 2016, 04:07:47 UTC |
1ec75ee | Joel Marcey | 07 September 2016, 04:07:13 UTC | Add redirects from old blog posts link to new format Summary: The new blog post links will be formatted differently coming over to gh-pages. But we can redirect from the old style over to the new style for existing blog posts. Test Plan: Visual https://www.facebook.com/pxlcld/pvWQ Reviewers: lgalanis, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D63513 | 07 September 2016, 04:07:13 UTC |
607628d | sdong | 01 September 2016, 22:28:40 UTC | Support ZSTD with finalized format Summary: ZSTD 1.0.0 is coming. We can finally add a support of ZSTD without worrying about compatibility. Still keep ZSTDNotFinal for compatibility reason. Test Plan: Run all tests. Run db_bench with ZSTD version with RocksDB built with ZSTD 1.0 and older. Reviewers: andrewkr, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: cyan, igor, IslamAbdelRahman, leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D63141 | 06 September 2016, 19:22:16 UTC |
ce1be2c | Injun Song | 06 September 2016, 15:41:43 UTC | Fix build error on Windows (AppVeyor) (#1315) Add 'cf_options' to source list and db_imple.cc fix casting | 06 September 2016, 15:41:43 UTC |
f7669b4 | sdong | 02 September 2016, 21:06:04 UTC | Fix Windows Build Summary: Fix two Windows build problems. Test Plan: Build on Windows and run all Linux tests. Reviewers: IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D63189 | 03 September 2016, 00:10:28 UTC |