| 9d24fda | sdong | 14 September 2020, 18:11:46 UTC | Swap pdf file | 14 September 2020, 18:11:46 UTC |
| 4e8fd75 | Siying Dong | 14 September 2020, 18:10:50 UTC | Minor fix with one pdf | 14 September 2020, 18:10:50 UTC |
| 8366619 | sdong | 14 September 2020, 18:01:03 UTC | Rename talk file names | 14 September 2020, 18:01:03 UTC |
| 431bfac | Siying Dong | 14 September 2020, 17:37:55 UTC | RocksDB brownbag slides | 14 September 2020, 17:37:55 UTC |
| 9269cf5 | Siying Dong | 12 May 2017, 23:55:27 UTC | Add files via upload | 12 May 2017, 23:55:27 UTC |
| 29b75aa | Siying Dong | 12 May 2017, 23:25:40 UTC | Add files via upload | 12 May 2017, 23:25:40 UTC |
| d6aafa3 | Siying Dong | 12 May 2017, 23:21:24 UTC | Add files via upload | 12 May 2017, 23:21:24 UTC |
| d98c2f8 | Siying Dong | 12 May 2017, 23:17:34 UTC | Add files via upload | 12 May 2017, 23:17:34 UTC |
| 758677f | Siying Dong | 12 May 2017, 23:14:50 UTC | Add files via upload | 12 May 2017, 23:14:50 UTC |
| 5a36d5f | Siying Dong | 12 May 2017, 23:14:06 UTC | Add files via upload | 12 May 2017, 23:14:06 UTC |
| d45fd09 | Siying Dong | 12 May 2017, 23:12:42 UTC | Add files via upload | 12 May 2017, 23:12:42 UTC |
| c6be140 | Siying Dong | 12 May 2017, 23:09:52 UTC | Add files via upload | 12 May 2017, 23:09:52 UTC |
| d611a76 | Siying Dong | 12 May 2017, 23:08:19 UTC | Add files via upload | 12 May 2017, 23:08:19 UTC |
| 6377c22 | Siying Dong | 12 May 2017, 22:56:25 UTC | Add files via upload | 12 May 2017, 22:56:25 UTC |
| 7f7d647 | Siying Dong | 12 May 2017, 22:46:20 UTC | Add files via upload | 12 May 2017, 22:46:20 UTC |
| 7ba81b1 | Siying Dong | 12 May 2017, 22:25:52 UTC | Add files via upload | 12 May 2017, 22:25:52 UTC |
| f980c7b | Siying Dong | 21 July 2016, 23:12:41 UTC | Improving RocksDB’s Write Scalability (Nathan Bronson) | 21 July 2016, 23:12:41 UTC |
| 1d15985 | sdong | 26 January 2016, 01:19:08 UTC | Update slides for talk for HPTS | 26 January 2016, 01:19:08 UTC |
| 473fb19 | agiardullo | 25 August 2015, 21:56:40 UTC | Add Transactions Meetup talk slides Summary: ...everyone else seems to send out a diff, so I'm sending out a diff. Test Plan: n/a Reviewers: sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D45573 | 25 August 2015, 23:12:00 UTC |
| 8392bf1 | sdong | 06 August 2014, 18:52:00 UTC | Upload Slides for talk in Flash Memory Summit Summary: Upload talk slides. Test Plan: N/A Reviewers: xjin, ljin, yhchiang Subscribers: igor, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D21195 | 06 August 2014, 18:53:45 UTC |
| b52f067 | Dhruba Borthakur | 03 June 2014, 06:35:23 UTC | Substitute sub-heading "Facebook Group" with "Developer Discussion Group". Summary: Many people have complained to me saying that they do not know where the engineering development discussions are occuring. Test Plan: Reviewers: CC: Task ID: # Blame Rev: | 03 June 2014, 06:35:23 UTC |
| da7070d | Xing Jin | 02 April 2014, 22:05:57 UTC | Upload meetup slides Summary: As discussed with Igor, will upload these slides here and share the links with the community. Test Plan: n/a Reviewers: igor CC: leveldb Differential Revision: https://reviews.facebook.net/D17421 | 02 April 2014, 22:16:47 UTC |
| d420d38 | Xing Jin | 26 March 2014, 17:05:53 UTC | Revise meetup page Summary: Revise according to recent change. Also revise .arcconfig as linter codes are deleted from this branch. Otherwise "arc" always fails. This branch is supposed to contain rocksdb.org webpages, no RocksDB source codes. So no linter is fine. Test Plan: N/A Reviewers: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D17187 | 26 March 2014, 17:35:01 UTC |
| 6d4608d | James Pearce | 25 March 2014, 03:52:02 UTC | Delete non-docs files from gh-page branch | 25 March 2014, 03:52:25 UTC |
| b814efc | Xing Jin | 24 March 2014, 17:17:37 UTC | Revise meetup page Summary: Revise contents as discussed offline. Test Plan: n/a Reviewers: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D17127 | 24 March 2014, 17:18:34 UTC |
| c4ff611 | Xing Jin | 20 March 2014, 19:30:30 UTC | Revise meetup page Summary: Add registration deadline. We need to send guest lists to security before that. Test Plan: N/A Reviewers: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D17085 | 20 March 2014, 19:31:31 UTC |
| 6ac19a3 | Xing Jin | 19 March 2014, 00:12:24 UTC | Revise meetup page Summary: Revise talk topics Test Plan: N/A Reviewers: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D16989 | 19 March 2014, 00:26:14 UTC |
| c179cf9 | Igor Canadi | 18 March 2014, 23:35:43 UTC | Add twitter | 18 March 2014, 23:35:43 UTC |
| 79e1273 | Dhruba Borthakur | 07 March 2014, 05:28:28 UTC | Fine-tune performance page. Summary: Test Plan: Reviewers: CC: Task ID: # Blame Rev: | 07 March 2014, 05:28:28 UTC |
| 1065ce4 | Dhruba Borthakur | 07 March 2014, 05:23:07 UTC | Cleanup performance benchmark webpage. Summary: Test Plan: Reviewers: CC: Task ID: # Blame Rev: | 07 March 2014, 05:23:07 UTC |
| 916f6d1 | Dhruba Borthakur | 07 March 2014, 05:22:10 UTC | Cleanup page with place-holders. Summary: Test Plan: Reviewers: CC: Task ID: # Blame Rev: | 07 March 2014, 05:22:10 UTC |
| 95bb12a | Dhruba Borthakur | 07 March 2014, 05:20:58 UTC | Refine the performance benchmark page. Summary: Test Plan: Reviewers: CC: Task ID: # Blame Rev: | 07 March 2014, 05:20:58 UTC |
| 484c85b | Dhruba Borthakur | 07 March 2014, 04:07:33 UTC | Vendors Compete. Summary: Encourage various vendors to publish their RocksDB benchmark. This will be a central repositiry that will have links to vendor-published benchmark results. Test Plan: none. Reviewers: xjin, haobo, igor, sdong, jamesgpearce Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D16677 | 07 March 2014, 05:15:41 UTC |
| aef768b | Xing Jin | 27 February 2014, 23:32:58 UTC | Change registration contact Summary: Vanessa has kindly offered to help. Test Plan: N/A Reviewers: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D16413 | 27 February 2014, 23:33:40 UTC |
| 19176a3 | Xing Jin | 26 February 2014, 15:51:48 UTC | Add NDA link to meetup page Summary: As the title Test Plan: locally loaded Reviewers: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D16335 | 26 February 2014, 15:52:35 UTC |
| 882810c | Xing Jin | 26 February 2014, 06:28:00 UTC | Add a web page for meetup Summary: By following https://www.facebook.com/groups/629866953728590/permalink/629872443728041/, add a web page for meetup information at rocksdb.or/meetup Test Plan: Locally loaded: https://www.facebook.com/pxlcld/l15s The header is a bit weird. That's my browser's issue. I use the same header as existing pages. Reviewers: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D16329 | 26 February 2014, 06:31:28 UTC |
| e85f103 | James Pearce | 16 December 2013, 17:12:00 UTC | Update index.html Add group link | 16 December 2013, 17:12:00 UTC |
| 785a220 | James Pearce | 15 November 2013, 19:44:32 UTC | Deck pdf | 15 November 2013, 19:44:32 UTC |
| 3a0a8a0 | James Pearce | 15 November 2013, 19:34:59 UTC | Merge branch 'master' of github.com:facebook/rocksdb into gh-pages | 15 November 2013, 19:34:59 UTC |
| 4f873b9 | James Pearce | 15 November 2013, 19:34:04 UTC | Intro slides from Data @ Scale event | 15 November 2013, 19:34:04 UTC |
| 29c931f | Igor Canadi | 15 November 2013, 06:42:02 UTC | Avoid populating live set if we don't need to Summary: Also changed some comments Test Plan: ./deletefile_test Reviewers: haobo Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D14091 | 15 November 2013, 06:42:02 UTC |
| 36b40fe | James Pearce | 15 November 2013, 02:45:46 UTC | updated hero image | 15 November 2013, 02:46:26 UTC |
| a0ce3fd | Igor Canadi | 15 November 2013, 02:03:57 UTC | PurgeObsoleteFiles() unittest Summary: Created a unittest that verifies that automatic deletion performed by PurgeObsoleteFiles() works correctly. Also, few small fixes on the logic part -- call version_set_->GetObsoleteFiles() in FindObsoleteFiles() instead of on some arbitrary positions. Test Plan: Created a unit test Reviewers: dhruba, haobo, nkg- Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D14079 | 15 November 2013, 02:03:57 UTC |
| 3ee1380 | James Pearce | 15 November 2013, 01:32:57 UTC | Updated OG NS | 15 November 2013, 01:32:57 UTC |
| ff6dd53 | James Pearce | 15 November 2013, 01:32:29 UTC | Missing OG NS | 15 November 2013, 01:32:29 UTC |
| 94dde68 | Vamsi Ponnekanti | 13 November 2013, 22:50:29 UTC | [Merge operand meant for key K is being applied on wrong key] Summary: We iterate until we find a different key than original key. ikey is pointing to next key when we break out of loop. After the loop we apply all merge operands meant for original key on the next key! Test Plan: Need to give a build to Marcin to test out. Revert Plan: OK Task ID: #3181932 Reviewers: haobo, emayanke, dhruba Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D14073 | 15 November 2013, 01:13:24 UTC |
| e0ad0f2 | Igor Canadi | 14 November 2013, 22:05:15 UTC | Fix bloom filters Summary: https://reviews.facebook.net/D13167 broke bloom filters. If filter is not in cache, we want to return true (safe thing). Am I right? Test Plan: when benchmarking https://reviews.facebook.net/D14031 I got different results when using bloom filters vs. when not using them. This fixed the issue. I will also be putting this change to the other diff, but that one will probably be in review for longer time. Reviewers: kailiu, dhruba, haobo Reviewed By: kailiu CC: leveldb Differential Revision: https://reviews.facebook.net/D14085 | 14 November 2013, 22:05:15 UTC |
| f48c540 | James Pearce | 14 November 2013, 04:40:05 UTC | Overview page and yellow theme | 14 November 2013, 04:41:26 UTC |
| fda8142 | Igor Canadi | 13 November 2013, 22:54:54 UTC | Delete log files in the correct dir Summary: Log files are stored in wal_dir, not dbname_ Test Plan: deletfile_test Reviewers: nkg- Reviewed By: nkg- CC: leveldb Differential Revision: https://reviews.facebook.net/D14067 | 13 November 2013, 22:54:54 UTC |
| 80bb81c | Kai Liu | 13 November 2013, 07:54:31 UTC | Add the correct table_factory for tables in table_tests | 13 November 2013, 07:54:31 UTC |
| 88ba331 | Kai Liu | 13 November 2013, 06:46:51 UTC | Add the index/filter block cache Summary: This diff leverage the existing block cache and extend it to cache index/filter block. Test Plan: Added new tests in db_test and table_test The correctness is checked by: 1. make check 2. make valgrind_check Performance is test by: 1. 10 times of build_tools/regression_build_test.sh on two versions of rocksdb before/after the code change. Test results suggests no significant difference between them. For the two key operatons `overwrite` and `readrandom`, the average iops are both 20k and ~260k, with very small variance). 2. db_stress. Reviewers: dhruba Reviewed By: dhruba CC: leveldb, haobo, xjin Differential Revision: https://reviews.facebook.net/D13167 | 13 November 2013, 06:46:51 UTC |
| aed9f1f | Kai Liu | 13 November 2013, 05:40:25 UTC | The updated sed still doesn't work in mac, revert. | 13 November 2013, 05:40:25 UTC |
| f3b3316 | Kai Liu | 13 November 2013, 05:26:19 UTC | Fix a sed command issue that cannot generated *.d files Summary: The original sed command is not recognized by mac's sed, which generates ".d-e" extension instead of ".d" Test Plan: make clean && make -j32 | 13 November 2013, 05:26:19 UTC |
| 22e1b04 | Kai Liu | 13 November 2013, 05:15:49 UTC | Quick fix for a string format Summary: Fix one more string format issue that throws warning in mac | 13 November 2013, 05:22:32 UTC |
| 35460cc | Kai Liu | 13 November 2013, 05:02:03 UTC | Fix the string format issue Summary: mac and our dev server has totally differnt definition of uint64_t, therefore fixing the warning in mac has actually made code in linux uncompileable. Test Plan: make clean && make -j32 | 13 November 2013, 05:05:39 UTC |
| d88d8ec | Igor Canadi | 13 November 2013, 04:32:07 UTC | Fix deleting files Summary: One more fix! In some cases, our filenames start with "/". Apparently, env_ can't handle filenames with double // Test Plan: deletefile_test does not include this line in the LOG anymore: 2013/11/12-18:11:43.150149 7fe4a6fff700 RenameFile logfile #3 FAILED -- IO error: /tmp/rocksdbtest-3574/deletefile_test//000003.log: No such file or directory Reviewers: dhruba, haobo Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D14055 | 13 November 2013, 04:32:07 UTC |
| 2158776 | kailiu | 13 November 2013, 04:05:28 UTC | Fixing the warning messages captured under mac os # Consider using `git commit -m 'One line title' && arc diff`. # You will save time by running lint and unit in the background. Summary: The work to make sure mac os compiles rocksdb is not completed yet. But at least we can start cleaning some warnings captured only by g++ from mac os.. Test Plan: ran make in mac os Reviewers: dhruba Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D14049 | 13 November 2013, 04:05:28 UTC |
| c3dda72 | Igor Canadi | 13 November 2013, 00:09:57 UTC | Update documentation Summary: Added more options for compaction settings + thread pools. Please check if thread pool description is correct. Test Plan: - Reviewers: dhruba Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D14043 | 13 November 2013, 00:09:57 UTC |
| 9ae624a | James Pearce | 12 November 2013, 23:13:32 UTC | Better OG image | 12 November 2013, 23:13:32 UTC |
| 907048c | James Pearce | 12 November 2013, 23:13:05 UTC | Better OG image ref | 12 November 2013, 23:13:05 UTC |
| 6f14dde | James Pearce | 12 November 2013, 18:50:46 UTC | RocksDB website | 12 November 2013, 22:38:10 UTC |
| 9df2b21 | Igor Canadi | 12 November 2013, 20:42:42 UTC | Move fast and break things Summary: Broke the compile when I removed purge_log_after_memtable_flush. sorrybus Test Plan: make db_bench works now Reviewers: haobo Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D14037 | 12 November 2013, 20:42:42 UTC |
| 9bc4a26 | Igor Canadi | 12 November 2013, 19:53:26 UTC | Small changes in Deleting obsolete files Summary: @haobo's suggestions from https://reviews.facebook.net/D13827 Renaming some variables, deprecating purge_log_after_flush, changing for loop into auto for loop. I have not implemented deleting objects outside of mutex yet because it would require a big code change - we would delete object in db_impl, which currently does not know anything about object because it's defined in version_edit.h (FileMetaData). We should do it at some point, though. Test Plan: Ran deletefile_test Reviewers: haobo Reviewed By: haobo CC: leveldb, haobo Differential Revision: https://reviews.facebook.net/D14025 | 12 November 2013, 19:53:26 UTC |
| dad4255 | Igor Canadi | 12 November 2013, 18:07:55 UTC | Move the comment Summary: Moving the comment per @haobo suggestion. Test Plan: No Reviewers: haobo Reviewed By: haobo CC: leveldb, haobo Differential Revision: https://reviews.facebook.net/D14019 | 12 November 2013, 18:07:55 UTC |
| 4abd219 | Igor Canadi | 12 November 2013, 05:41:32 UTC | Combine two FindObsoleteFiles() Summary: We don't need to call FindObsoleteFiles() twice Test Plan: deletefile_test Reviewers: dhruba Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D14007 | 12 November 2013, 05:41:32 UTC |
| 0ef6285 | Kai Liu | 12 November 2013, 05:05:16 UTC | Don't not suggest flushing data when data block is still empty Summary: This diff fix the bug when the Options::block_size is too small. | 12 November 2013, 05:05:16 UTC |
| 94e139f | Igor Canadi | 12 November 2013, 05:03:41 UTC | Fixing failed delete file test Summary: FindObsoleteFiles() has to be called before PurgeObsoleteFiles() because FindObsoleteFiles() sets manifest_file_number, log_number and prev_log_number to valid values. Test Plan: deletefile_test now works Reviewers: dhruba, emayanke, kailiu Reviewed By: kailiu CC: leveldb Differential Revision: https://reviews.facebook.net/D13995 | 12 November 2013, 05:03:41 UTC |
| 65e45f0 | Igor Canadi | 12 November 2013, 05:02:38 UTC | Update documentation Summary: Changed leveldb documentation with rocksdb in doc/index.html. Added some of the important options from options.h to doc. Also removed benchmark files and impl.h, since this is all replaced by RocksDB wikis. Test Plan: - Reviewers: dhruba, haobo, kailiu, emayanke, sdong Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D13977 | 12 November 2013, 05:02:38 UTC |
| 318a491 | Dhruba Borthakur | 11 November 2013, 22:42:52 UTC | Fix valgrind check by initialising DeletionState. Summary: The valgrind error was introduced by commit 1510339e5257073af82f8c07e6fa1f2a9144e6aa. Initialize DeletionState in constructor. Test Plan: valgrind --leak-check=yes ./deletefile_test Reviewers: igor, kailiu Reviewed By: kailiu CC: leveldb Differential Revision: https://reviews.facebook.net/D13983 | 12 November 2013, 00:01:13 UTC |
| e7c4d82 | Kai Liu | 10 November 2013, 23:36:30 UTC | Fix two bugs that caused 3rd party release failure Summary: * Fix the link to gflags. * Fix a warning for the uninitialized data member. | 10 November 2013, 23:36:30 UTC |
| 551ecfa | Kai Liu | 10 November 2013, 09:17:32 UTC | Move down the time consuming tests in table_test Summary: it helps us to better check the tests we really care. Test Plan: make | 10 November 2013, 09:17:32 UTC |
| 8a46ecd | lovro | 07 November 2013, 20:37:58 UTC | WriteBatch::Put() overload that gathers key and value from arrays of slices Summary: In our project, when writing to the database, we want to form the value as the concatenation of a small header and a larger payload. It's a shame to have to copy the payload just so we can give RocksDB API a linear view of the value. Since RocksDB makes a copy internally, it's easy to support gather writes. Test Plan: write_batch_test, new test case Reviewers: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D13947 | 09 November 2013, 00:34:32 UTC |
| 1510339 | Igor Canadi | 08 November 2013, 23:23:46 UTC | Speed up FindObsoleteFiles Summary: Here's one solution we discussed on speeding up FindObsoleteFiles. Keep a set of all files in DBImpl and update the set every time we create a file. I probably missed few other spots where we create a file. It might speed things up a bit, but makes code uglier. I don't really like it. Much better approach would be to abstract all file handling to a separate class. Think of it as layer between DBImpl and Env. Having a separate class deal with file namings and deletion would benefit both code cleanliness (especially with huge DBImpl) and speed things up. It will take a huge effort to do this, though. Let's discuss offline today. Test Plan: Ran ./db_stress, verified that files are getting deleted Reviewers: dhruba, haobo, kailiu, emayanke Reviewed By: dhruba Differential Revision: https://reviews.facebook.net/D13827 | 08 November 2013, 23:23:46 UTC |
| dd218bb | Igor Canadi | 08 November 2013, 20:23:12 UTC | Forgot to change interface everywhere Summary: Changed the name and interface for creating HashSkipListRep. Forgot to change it in db_test. Test Plan: make db_test Reviewers: haobo Reviewed By: haobo Differential Revision: https://reviews.facebook.net/D13965 | 08 November 2013, 20:23:12 UTC |
| 8b3379d | Igor Canadi | 08 November 2013, 08:31:09 UTC | Implementing DynamicIterator for TransformRepNoLock Summary: What @haobo done with TransformRep, now in TransformRepNoLock. Similar implementation, except that I made DynamicIterator a subclass of Iterator which makes me have less iterator initializations. Test Plan: ./prefix_test. Seeing huge savings vs. TransformRep again! Reviewers: dhruba, haobo, sdong, kailiu Reviewed By: haobo CC: leveldb, haobo Differential Revision: https://reviews.facebook.net/D13953 | 08 November 2013, 08:31:09 UTC |
| fd075d6 | Kai Liu | 08 November 2013, 05:27:21 UTC | Provide mechanism to configure when to flush the block Summary: Allow block based table to configure the way flushing the blocks. This feature will allow us to add support for prefix-aligned block. Test Plan: make check Reviewers: dhruba, haobo, sdong, igor Reviewed By: sdong CC: leveldb Differential Revision: https://reviews.facebook.net/D13875 | 08 November 2013, 05:27:21 UTC |
| bba6595 | Kai Liu | 07 November 2013, 23:46:48 UTC | Fix the valgrind error Summary: I this bug from valgrind report and found a place that may potentially leak memory. Test Plan: re-ran the valgrind and no error any more Reviewers: emayanke Reviewed By: emayanke CC: leveldb Differential Revision: https://reviews.facebook.net/D13959 | 07 November 2013, 23:46:48 UTC |
| 444cf88 | Igor Canadi | 07 November 2013, 19:31:56 UTC | Flush the log outside of lock Summary: Added a new call LogFlush() that flushes the log contents to the OS buffers. We never call it with lock held. We call it once for every Read/Write and often in compaction/flush process so the frequency should not be a problem. Test Plan: db_test Reviewers: dhruba, haobo, kailiu, emayanke Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D13935 | 07 November 2013, 19:31:56 UTC |
| fd20448 | Haobo Xu | 04 November 2013, 00:32:46 UTC | [RocksDB] Generalize prefix-aware iterator to be used for more than one Seek Summary: Added a prefix_seek flag in ReadOptions to indicate that Seek is prefix aware(might not return data with different prefix), and also not bound to a specific prefix. Multiple Seeks and range scans can be invoked on the same iterator. If a specific prefix is specified, this flag will be ignored. Just a quick prototype that works for PrefixHashRep, the new lockless memtable could be easily extended with this support too. Test Plan: test it on Leaf Reviewers: dhruba, kailiu, sdong, igor Reviewed By: igor CC: leveldb Differential Revision: https://reviews.facebook.net/D13929 | 07 November 2013, 04:45:49 UTC |
| c2be2cb | shamdor | 07 November 2013, 02:46:28 UTC | WAL log retention policy based on archive size. Summary: Archive cleaning will still happen every WAL_ttl seconds but archived logs will be deleted only if archive size is greater then a WAL_size_limit value. Empty archived logs will be deleted evety WAL_ttl. Test Plan: 1. Unit tests pass. 2. Benchmark. Reviewers: emayanke, dhruba, haobo, sdong, kailiu, igor Reviewed By: emayanke CC: leveldb Differential Revision: https://reviews.facebook.net/D13869 | 07 November 2013, 02:46:28 UTC |
| 292c2b3 | Dhruba Borthakur | 06 November 2013, 22:16:22 UTC | Fix stress test failure when using mmap-reads. Summary: The mmap-read file->Read() does not use the scratch buffer to read in file-contents. Test Plan: ./db_stress --test_batches_snapshots=1 --ops_per_thread=100000000 --threads=32 --write_buffer_size=4194304 --destroy_db_initially=0 --reopen=0 --readpercent=45 --prefixpercent=5 --writepercent=35 --delpercent=5 --iterpercent=10 --db=/tmp/dhruba --max_key=100000000 --disable_seek_compaction=0 --mmap_read=1 --block_size=16384 --cache_size=1048576 --open_files=500000 --verify_checksum=1 --sync=1 --disable_wal=0 --disable_data_sync=0 --target_file_size_base=2097152 --target_file_size_multiplier=2 --max_write_buffer_number=3 --max_background_compactions=20 --max_bytes_for_level_base=10485760 --filter_deletes=0 Reviewers: haobo, kailiu Reviewed By: kailiu CC: leveldb, kailiu, emayanke Differential Revision: https://reviews.facebook.net/D13923 | 06 November 2013, 23:40:26 UTC |
| 95a8213 | Igor Canadi | 06 November 2013, 22:19:46 UTC | Log flush every 0 seconds Summary: We have to be able to catch last few log outputs before a crash Test Plan: no Reviewers: dhruba Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D13917 | 06 November 2013, 22:19:46 UTC |
| 36409e0 | Igor Canadi | 06 November 2013, 22:11:52 UTC | Fix slow no-io iterator Summary: This fixes #3130525. Dhruba's suggestion and Tnovak's implementation :) The issue was with SkipEmptyDataBlocksForward(), but I also changed SkipEmptyDataBlocksBackward(). Is that OK? Test Plan: Run the logdevice test Reviewers: dhruba Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D13911 | 06 November 2013, 22:11:52 UTC |
| be96f24 | Igor Canadi | 06 November 2013, 19:55:43 UTC | TransformRep - use array instead of unordered_map Summary: I'm sending this diff together with https://reviews.facebook.net/D13881 because it didn't allow me to send only the array one. Here I also replaced unordered_map with just an array of shared_ptrs. This elminated all the locks. I will run the new benchmark and post the results here. Test Plan: db_test Reviewers: dhruba, haobo Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D13893 | 06 November 2013, 19:55:43 UTC |
| fe4a449 | Haobo Xu | 23 October 2013, 04:59:44 UTC | [RocksDB] prefixhash memtable test Summary: as title, half baked test for prefixhash memtable. Also contains deadlock test option Test Plan: run it Reviewers: igor, dhruba Reviewed By: igor CC: leveldb Differential Revision: https://reviews.facebook.net/D13887 | 06 November 2013, 07:20:10 UTC |
| f0b0b28 | Kai Liu | 06 November 2013, 05:04:22 UTC | Remove invalid items in .gitignore | 06 November 2013, 05:04:22 UTC |
| 3919058 | Dhruba Borthakur | 05 November 2013, 22:05:30 UTC | Fix failure in rocksdb unit test CompressedCache Summary: The problem was that there was only a single key-value in a block and its compressibility was less than 88%. Rocksdb refuses to compress a block unless its compresses to lesser than 88% of its original size. If a block is not compressed, it does nto get inserted into the compressed block cache. Create the test data so that multiple records fit into the same data block. This increases the compressibility of these data block. Test Plan: ./db_test Reviewers: kailiu, haobo Reviewed By: kailiu CC: leveldb Differential Revision: https://reviews.facebook.net/D13905 | 06 November 2013, 00:11:34 UTC |
| 7845fd9 | Dhruba Borthakur | 05 November 2013, 06:02:59 UTC | Fixed valgrind error in DBTest.CompressedCache Summary: Fixed valgrind error in DBTest.CompressedCache. This fixes the valgrind error (thanks to Haobo). I am still trying to reproduce the test-failure case deterministically. Test Plan: db_test Reviewers: haobo Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D13899 | 05 November 2013, 19:12:39 UTC |
| f837f5b | Mayank Agarwal | 25 October 2013, 02:09:02 UTC | Making the transaction log iterator more robust Summary: strict essentially means that we MUST find the startsequence. Thus we should return if starteSequence is not found in the first file in case strict is set. This will take care of ending the iterator in case of permanent gaps due to corruptions in the log files Also created NextImpl function that will have internal variable to distinguish whether Next is being called from StartSequence or by application. Set NotFoudn::gaps status to give an indication of gaps happeneing. Polished the inline documentation at various places Test Plan: * db_repl_stress test * db_test relating to transaction log iterator * fbcode/wormhole/rocksdb/rocks_log_iterator * sigma production machine sigmafio032.prn1 Reviewers: dhruba Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D13689 | 05 November 2013, 04:49:03 UTC |
| b4ad5e8 | Dhruba Borthakur | 02 September 2013, 06:23:40 UTC | Implement a compressed block cache. Summary: Rocksdb can now support a uncompressed block cache, or a compressed block cache or both. Lookups first look for a block in the uncompressed cache, if it is not found only then it is looked up in the compressed cache. If it is found in the compressed cache, then it is uncompressed and inserted into the uncompressed cache. It is possible that the same block resides in the compressed cache as well as the uncompressed cache at the same time. Both caches have their own individual LRU policy. Test Plan: Unit test case attached. Reviewers: kailiu, sdong, haobo, leveldb Reviewed By: haobo CC: xjin, haobo Differential Revision: https://reviews.facebook.net/D12675 | 01 November 2013, 21:31:35 UTC |
| 1e4375d | Piyush Garg | 01 November 2013, 20:59:14 UTC | Task #3071144 Enhance ldb (db dump tool for leveldb) to report row counters for each row type Summary: Added an option --count_delim=<char> which takes the given character as delimiter ('.' by default) and reports count of each row type found in the db Test Plan: 1. Created test in file (for DBDumperCommand) rocksdb/tools/ldb_test.py which puts various key value pair in db and checks the output using dump --count_delim ,--count_delim="." and --count_delim=",". 2. Created test in file (for InternalDumperCommand) rocksdb/tools/ldb_test.py which puts various key value pair in db and checks the output using dump --count_delim ,--count_delim="." and --count_delim=",". 3. Manually created a database with several keys of several type and verified by running the command ./ldb db=<path> dump --count_delim="<char>" ./ldb db=<path> idump --count_delim="<char>" Reviewers: vamsi, dhruba, emayanke, kailiu Reviewed By: vamsi CC: leveldb Differential Revision: https://reviews.facebook.net/D13815 | 01 November 2013, 20:59:14 UTC |
| beeb74b | Igor Canadi | 01 November 2013, 19:32:27 UTC | Move I/O outside of lock Summary: I'm figuring out how Version[Set, Edit, ] classes work and I stumbled on this. It doesn't seem that the comment is accurate anymore. What I read is when the manifest grows too big, create a new file (and not only when we call LogAndApply for the first time). Test Plan: make check (currently running) Reviewers: dhruba, haobo, kailiu, emayanke Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D13839 | 01 November 2013, 19:32:27 UTC |
| b572e81 | Igor Canadi | 31 October 2013, 22:36:40 UTC | Flush Log every 5 seconds Summary: This might help with p99 performance, but does not solve the real problem. More discussion on #2947135 Test Plan: make check Reviewers: dhruba, haobo Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D13809 | 31 October 2013, 22:36:40 UTC |
| 82b7e37 | Siying Dong | 31 October 2013, 22:26:06 UTC | Fix a bug of table_reader_bench Summary: Iterator benchmark case is timed incorrectly. Fix it Test Plan: Run the benchmark Reviewers: haobo, dhruba Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D13845 | 31 October 2013, 22:26:06 UTC |
| 7caadf2 | Siying Dong | 31 October 2013, 20:38:54 UTC | A very simple benchmark to measure Table implemenation's Get() And Iterator performance Summary: It is a very simple benchmark to measure a Table implementation's Get() and iterator performance if all the data is in memory. Test Plan: N/A Reviewers: dhruba, haobo, kailiu Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D13743 | 31 October 2013, 20:38:54 UTC |
| 8cbe5bb | Haobo Xu | 27 October 2013, 06:01:26 UTC | [RocksDB] Add OnCompactionStart to CompactionFilter class Summary: This is to give application compaction filter a chance to access context information of a specific compaction run. For example, depending on whether a compaction goes through all data files, the application could do things differently. Test Plan: make check Reviewers: dhruba, kailiu, sdong Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D13683 | 31 October 2013, 20:36:43 UTC |
| b4fab3b | Naman Gupta | 31 October 2013, 18:51:03 UTC | Merge branch 'master' of github.com:facebook/rocksdb into inplace | 31 October 2013, 18:51:03 UTC |
| 138a8ee | Igor Canadi | 31 October 2013, 18:47:22 UTC | Fix make release Summary: Don't define if already defined. Test Plan: Running make release in parallel, will not commit if it fails. Reviewers: emayanke Reviewed By: emayanke CC: leveldb Differential Revision: https://reviews.facebook.net/D13833 | 31 October 2013, 18:47:22 UTC |
| fe25070 | Naman Gupta | 19 August 2013, 21:12:47 UTC | In-place updates for equal keys and similar sized values Summary: Currently for each put, a fresh memory is allocated, and a new entry is added to the memtable with a new sequence number irrespective of whether the key already exists in the memtable. This diff is an attempt to update the value inplace for existing keys. It currently handles a very simple case: 1. Key already exists in the current memtable. Does not inplace update values in immutable memtable or snapshot 2. Latest value type is a 'put' ie kTypeValue 3. New value size is less than existing value, to avoid reallocating memory TODO: For a put of an existing key, deallocate memory take by values, for other value types till a kTypeValue is found, ie. remove kTypeMerge. TODO: Update the transaction log, to allow consistent reload of the memtable. Test Plan: Added a unit test verifying the inplace update. But some other unit tests broken due to invalid sequence number checks. WIll fix them next. Reviewers: xinyaohu, sumeet, haobo, dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D12423 Automatic commit by arc | 31 October 2013, 18:27:12 UTC |
| f03b2df | Siying Dong | 30 October 2013, 17:52:33 UTC | Follow-up Cleaning-up After D13521 Summary: This patch is to address @haobo's comments on D13521: 1. rename Table to be TableReader and make its factory function to be GetTableReader 2. move the compression type selection logic out of TableBuilder but to compaction logic 3. more accurate comments 4. Move stat name constants into BlockBasedTable implementation. 5. remove some uncleaned codes in simple_table_db_test Test Plan: pass test suites. Reviewers: haobo, dhruba, kailiu Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D13785 | 30 October 2013, 17:52:33 UTC |
