| eda3984 | Ankit Gupta | 19 April 2014, 10:35:01 UTC | Add more iterator functions | 19 April 2014, 10:35:01 UTC |
| 5bbeefa | Ankit Gupta | 19 April 2014, 10:26:22 UTC | Adding iterator JNI binding | 19 April 2014, 10:26:22 UTC |
| 27d3bc1 | sdong | 18 April 2014, 01:00:58 UTC | Use a different approach to make sure BlockBasedTableReader can use hash index on older files Summary: A recent commit https://github.com/facebook/rocksdb/commit/e37dd216f9384bfdabc6760fa296e8ee28c79d30 makes sure hash index can be used when reading existing files. This patch uses another way to achieve the approach: (1) Currently, always writing kBinarySearch to files, despite of BlockBasedTableOptions.IndexType setting. (2) When reading a file, read out the field, and make sure it is kBinarySearch, while always use index type by users. The reason for doing it is, to reserve kHashSearch property on disk to future. If now we write out binary index for both of kHashSearch and kBinarySearch. We have to use a new flag in the future for hash index on disk, otherwise compatibility would break. Also, we want the real index type and type shown in properties block to be consistent. Test Plan: make all check Reviewers: haobo, kailiu Reviewed By: kailiu CC: igor, ljin, yhchiang, xjin, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D18009 | 18 April 2014, 21:09:21 UTC |
| 35c968f | Igor Canadi | 18 April 2014, 20:53:52 UTC | Merge pull request #122 from ankgup87/master [Java] Add statistics JNI bindings | 18 April 2014, 20:53:52 UTC |
| 686fdea | Ankit Gupta | 18 April 2014, 17:48:48 UTC | Fix formatting issues | 18 April 2014, 17:48:48 UTC |
| ebd85e8 | Ankit Gupta | 18 April 2014, 17:47:03 UTC | Fix build | 18 April 2014, 17:47:03 UTC |
| dc291f5 | Ankit Gupta | 18 April 2014, 17:32:14 UTC | Merge branch 'master' of https://github.com/facebook/rocksdb Conflicts: Makefile java/Makefile java/org/rocksdb/Options.java java/rocksjni/portal.h | 18 April 2014, 17:32:14 UTC |
| 1a8abe7 | Igor Canadi | 18 April 2014, 16:45:48 UTC | Merge pull request #120 from jamesgpearce/master Added period | 18 April 2014, 16:45:48 UTC |
| a745089 | James Pearce | 18 April 2014, 16:33:27 UTC | Added period This is a PR to test some tooling; please do not merge without talking to @jamesgpearce :) | 18 April 2014, 16:33:27 UTC |
| 9b2a093 | Yueh-Hsuan Chiang | 18 April 2014, 08:14:29 UTC | [Java] Add Java bindings for 30 options for rocksdb::DBOptions. Summary: 1. Add Java bindings for 30 options for rocksdb::DBOptions. 2. Add org.rocksdb.test.OptionsTest 3. Codes are semi-auto generated, JavaDocs are manually polished. Test Plan: make rocksdbjava make jtest Reviewers: haobo, ankgup87, sdong, dhruba Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D18015 | 18 April 2014, 08:14:29 UTC |
| bb6fd15 | Yueh-Hsuan Chiang | 18 April 2014, 00:28:51 UTC | [Java] Add a basic binding and test for BackupableDB and StackableDB. Summary: Add a skeleton binding and test for BackupableDB which shows that BackupableDB and RocksDB can share the same JNI calls. Test Plan: make rocksdbjava make jtest Reviewers: haobo, ankgup87, sdong, dhruba Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D17793 | 18 April 2014, 00:28:51 UTC |
| 6517922 | sdong | 17 April 2014, 22:14:04 UTC | Fix bugs introduced by D17961 Summary: D17961 has two bugs: (1) two level iterator fails to populate FileMetaData.table_reader, causing performance regression. (2) table cache handle the !status.ok() case in the wrong place, causing seg fault which shouldn't happen. Test Plan: make all check Reviewers: ljin, igor, haobo Reviewed By: ljin CC: yhchiang, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D17991 | 18 April 2014, 00:25:28 UTC |
| ce353c2 | Igor Canadi | 17 April 2014, 21:43:42 UTC | Nuke tools/shell Summary: We don't use or build this code Test Plan: builds Reviewers: dhruba Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D17979 | 17 April 2014, 21:43:42 UTC |
| 86ae820 | Igor Canadi | 17 April 2014, 21:29:06 UTC | Fix ifdef NDEBUG | 17 April 2014, 21:29:28 UTC |
| fa430bf | sdong | 17 April 2014, 21:07:05 UTC | Minimize accessing multiple objects in Version::Get() Summary: One of our profilings shows that Version::Get() sometimes is slow when getting pointer of user comparators or other global objects. In this patch: (1) we keep pointers of immutable objects in Version to avoid accesses them though option objects or cfd objects (2) table_reader is directly cached in FileMetaData so that table cache don't have to go through handle first to fetch it (3) If level 0 has less than 3 files, skip the filtering logic based on SST tables' key range. Smallest and largest key are stored in separated memory locations, which has potential cache misses Test Plan: make all check Reviewers: haobo, ljin Reviewed By: haobo CC: igor, yhchiang, nkg-, leveldb Differential Revision: https://reviews.facebook.net/D17739 | 17 April 2014, 21:14:00 UTC |
| e37dd21 | Kai Liu | 11 April 2014, 20:55:26 UTC | Index type doesn't have to be persisted Summary: With the recent changes, there is no need to check the property block about the index block type. If user want to use it, they don't really need any disk format change; everything happens in the fly. Also another team encountered an error while reading the index type from properties. Test Plan: ran all the tests Reviewers: sdong CC: Task ID: # Blame Rev: | 17 April 2014, 18:08:12 UTC |
| 62551b1 | Igor Canadi | 17 April 2014, 17:49:58 UTC | Don't compile sync_point if NDEBUG Summary: We don't really need sync_point.o if we're compiling with NDEBUG. This diff depends on D17823 Test Plan: compiles Reviewers: haobo, ljin, sdong Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D17829 | 17 April 2014, 17:49:58 UTC |
| 5cef458 | sdong | 17 April 2014, 02:30:33 UTC | RocksDB 2.8 to be able to read files generated by 2.6 Summary: From 2.6 to 2.7, property block name is renamed from rocksdb.stats to rocksdb.properties. Older properties were not able to be loaded. In 2.8, we seem to have added some logic that uses property block without checking null pointers, which create segment faults. In this patch, we fix it by: (1) try rocksdb.stats if rocksdb.properties is not found (2) add some null checking before consuming rep->table_properties Test Plan: make sure a file generated in 2.7 couldn't be opened now can be opened. Reviewers: haobo, igor, yhchiang Reviewed By: igor CC: ljin, xjin, dhruba, kailiu, leveldb Differential Revision: https://reviews.facebook.net/D17961 | 17 April 2014, 16:51:43 UTC |
| 136e4ef | Ankit Gupta | 17 April 2014, 05:02:46 UTC | Add doc | 17 April 2014, 05:02:46 UTC |
| d160595 | Ankit Gupta | 17 April 2014, 05:00:44 UTC | Add doc | 17 April 2014, 05:00:44 UTC |
| d3b44f0 | Ankit Gupta | 17 April 2014, 04:55:15 UTC | Fix white spaces | 17 April 2014, 04:55:15 UTC |
| 6b0cc41 | Ankit Gupta | 17 April 2014, 04:39:44 UTC | Merge branch 'master' of https://github.com/facebook/rocksdb | 17 April 2014, 04:39:44 UTC |
| 5108ed7 | Ankit Gupta | 17 April 2014, 04:39:29 UTC | Add fixed classes | 17 April 2014, 04:39:29 UTC |
| 320ae72 | Ankit Gupta | 17 April 2014, 04:38:33 UTC | Add histogramType for statistics | 17 April 2014, 04:38:33 UTC |
| 161d9e5 | Igor Canadi | 16 April 2014, 22:15:22 UTC | Don't overflow size_t in mac | 16 April 2014, 22:15:22 UTC |
| 5c12f27 | Igor Canadi | 16 April 2014, 16:09:28 UTC | Remove tautological assert | 16 April 2014, 16:09:28 UTC |
| 144066a | Ankit Gupta | 16 April 2014, 01:08:37 UTC | Deleted statisticsType | 16 April 2014, 01:08:37 UTC |
| 495fc80 | Ankit Gupta | 16 April 2014, 00:38:20 UTC | Merge branch 'master' of https://github.com/facebook/rocksdb | 16 April 2014, 00:38:20 UTC |
| b18d914 | Ankit Gupta | 16 April 2014, 00:37:50 UTC | Change statisticsType to tickerType | 16 April 2014, 00:37:50 UTC |
| faf7691 | Igor Canadi | 16 April 2014, 00:12:18 UTC | Close DB at the end of DontRollEmptyLogs test | 16 April 2014, 00:20:56 UTC |
| 1803ed2 | Igor Canadi | 15 April 2014, 23:31:49 UTC | Fix Mac OS compile | 15 April 2014, 23:31:49 UTC |
| 7d83885 | Igor Canadi | 15 April 2014, 22:59:34 UTC | Fix compile issues when doing make release | 15 April 2014, 23:00:10 UTC |
| 0f40fe4 | sdong | 15 April 2014, 16:06:13 UTC | When creating a new DB, fail it when wal_dir contains existing log files Summary: Current behavior of creating new DB is, if there is existing log files, we will go ahead and replay them on top of empty DB. This is a behavior that no user would expect. With this patch, we will fail the creation if a user creates a DB with existing log files. Test Plan: make all check Reviewers: haobo, igor, ljin Reviewed By: haobo CC: nkg-, yhchiang, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D17817 | 15 April 2014, 21:01:57 UTC |
| c166615 | Igor Canadi | 15 April 2014, 20:51:07 UTC | Fix compile issues introduced by RocksDBLite | 15 April 2014, 20:51:07 UTC |
| 588bca2 | Igor Canadi | 15 April 2014, 20:39:26 UTC | RocksDBLite Summary: Introducing RocksDBLite! Removes all the non-essential features and reduces the binary size. This effort should help our adoption on mobile. Binary size when compiling for IOS (`TARGET_OS=IOS m static_lib`) is down to 9MB from 15MB (without stripping) Test Plan: compiles :) Reviewers: dhruba, haobo, ljin, sdong, yhchiang Reviewed By: yhchiang CC: leveldb Differential Revision: https://reviews.facebook.net/D17835 | 15 April 2014, 20:39:26 UTC |
| 23c8f89 | Igor Canadi | 15 April 2014, 18:29:02 UTC | Revert "Don't compile ldb tool into static library" This reverts commit e296577ef64deac036a275a1a4c0d6172cfa42df. | 15 April 2014, 18:29:02 UTC |
| a347ffe | Igor Canadi | 15 April 2014, 18:28:52 UTC | Revert "Fix sst_dump and reduce_levels_test compile errors" This reverts commit d8f00b4109e3df10be56141f3ff3ba9b0d10f585. | 15 April 2014, 18:28:52 UTC |
| d8f00b4 | Igor Canadi | 15 April 2014, 18:08:48 UTC | Fix sst_dump and reduce_levels_test compile errors | 15 April 2014, 18:13:12 UTC |
| e296577 | Igor Canadi | 15 April 2014, 17:52:39 UTC | Don't compile ldb tool into static library Summary: This is first step of my effort to reduce size of librocksdb.a for use in mobile. ldb object files are huge and are ment to be used as a command line tool. I moved them to `tools/` directory and include them only when compiling `ldb` This diff reduced librocksdb.a from 42MB to 39MB on my mac (not stripped). Test Plan: ran ldb Reviewers: dhruba, haobo, sdong, ljin, yhchiang Reviewed By: yhchiang CC: leveldb Differential Revision: https://reviews.facebook.net/D17823 | 15 April 2014, 17:52:39 UTC |
| dbe0f32 | Igor Canadi | 15 April 2014, 17:28:34 UTC | Set log_empty to false even when options.sync is off [fix tests] | 15 April 2014, 17:28:34 UTC |
| e6acb87 | Igor Canadi | 15 April 2014, 16:57:25 UTC | Don't roll empty logs Summary: With multiple column families, especially when manual Flush is executed, we might roll the log file, although the current log file is empty (no data has been written to the log). After the diff, we won't create new log file if current is empty. Next, I will write an algorithm that will flush column families that reference old log files (i.e., that weren't flushed in a while) Test Plan: Added an unit test. Confirmed that unit test failes in master Reviewers: dhruba, haobo, ljin, sdong Reviewed By: ljin CC: leveldb Differential Revision: https://reviews.facebook.net/D17631 | 15 April 2014, 16:57:25 UTC |
| 3723d08 | Ankit Gupta | 15 April 2014, 05:16:10 UTC | Add newline at end of file | 15 April 2014, 05:16:10 UTC |
| 9842177 | Ankit Gupta | 15 April 2014, 04:14:29 UTC | remove unused header | 15 April 2014, 04:14:29 UTC |
| 8756a32 | Ankit Gupta | 15 April 2014, 04:11:03 UTC | Remove unused headers | 15 April 2014, 04:11:03 UTC |
| c199e0e | Ankit Gupta | 15 April 2014, 04:08:04 UTC | Add statistics | 15 April 2014, 04:08:04 UTC |
| a044398 | Ankit Gupta | 15 April 2014, 04:06:13 UTC | Add statistics object | 15 April 2014, 04:06:13 UTC |
| c87ed09 | sdong | 14 April 2014, 22:42:20 UTC | Fix db_bench's multireadrandom Summary: multireadrandom is broken. Fix it Test Plan: run it and see segfault has gone. Reviewers: ljin Reviewed By: ljin CC: leveldb Differential Revision: https://reviews.facebook.net/D17781 | 14 April 2014, 22:43:34 UTC |
| 118f88d | Yueh-Hsuan Chiang | 14 April 2014, 21:05:15 UTC | Fix compile error in tailing_iter.h Summary: Fix the following compile error ./db/tailing_iter.h:17:1: error: class 'SuperVersion' was previously declared as a struct [-Werror,-Wmismatched-tags] class SuperVersion; ^ ./db/column_family.h:77:8: note: previous use is here struct SuperVersion { ^ ./db/tailing_iter.h:17:1: note: did you mean struct here? class SuperVersion; ^~~~~ struct 1 error generated. Test Plan: make Reviewers: ljin, igor, haobo, sdong Reviewed By: ljin CC: leveldb Differential Revision: https://reviews.facebook.net/D17799 | 14 April 2014, 21:05:15 UTC |
| 51f4b50 | Yueh-Hsuan Chiang | 14 April 2014, 21:03:43 UTC | [Java] Add SizeUnit in org.rocksdb.util to store const like KB, GB. Summary: * Add a class SizeUnit to store frequently used consts. Currently it has KB, MB, GB, TB, and PB. * Change the parameter type of Options.writeBufferSize and Options.blockSize from int to long. Test Plan: make rocksdbjava make jtest Reviewers: haobo, ankgup87, sdong, dhruba Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D17703 | 14 April 2014, 21:03:43 UTC |
| 31e7e7f | Yueh-Hsuan Chiang | 14 April 2014, 20:42:36 UTC | [JAVA] Add java binding for Options.block_cache. Summary: Add java bindings for Options.block_cache and allow DbBenchmark to set cache_size. Test Plan: make rocksdbjava make jtest make jdb_Bench Reviewers: haobo, sdong, ankgup87 Reviewed By: ankgup87 CC: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D17481 | 14 April 2014, 20:42:36 UTC |
| 2885ad9 | Yueh-Hsuan Chiang | 14 April 2014, 19:48:50 UTC | [JNI] Each set function of Options / WriteOptions now returns its option instance. Summary: Make each set function of Options / WriteOptions return its option instance. Java developers can now easier specify each option like the following: options.setCreateIfMissing(true) .setWriteBufferSize(8 * 1024) .setMaxWriteBufferNumber(3) .setDisableSeekCompaction(true) .setBlockSize(64 * 1024) .setMaxBackgroundCompactions(10); Test Plan: make rocksdbjava make jtest Reviewers: haobo, ankgup87, sdong, dhruba Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D17661 | 14 April 2014, 19:48:50 UTC |
| be01661 | Igor Canadi | 14 April 2014, 19:28:15 UTC | Expose in memory Env to the world Summary: That will help with some iOS testing I'm doing. Test Plan: compiles Reviewers: dhruba, haobo, ljin, yhchiang, sdong Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D17787 | 14 April 2014, 19:28:15 UTC |
| 327102e | Yueh-Hsuan Chiang | 14 April 2014, 19:06:30 UTC | Fix merge_test failure due to incorrect assert behavior in the release mode. | 14 April 2014, 19:06:49 UTC |
| 82b37a1 | Lei Jin | 14 April 2014, 17:48:01 UTC | thread local for tailing iterator Summary: replace the super version acquisision in tailing itrator with thread local Test Plan: will post results Reviewers: igor, haobo, sdong, yhchiang, dhruba Reviewed By: igor CC: leveldb Differential Revision: https://reviews.facebook.net/D17757 | 14 April 2014, 17:48:01 UTC |
| 539dd20 | Lei Jin | 14 April 2014, 16:34:59 UTC | using thread local SuperVersion for NewIterator Summary: Similar to GetImp(), use SuperVersion from thread local instead of acquriing mutex. I don't expect this change will make a dent on NewIterator() performance because the bottleneck seems to be on the rest part of the API Test Plan: make asan_check will post perf numbers Reviewers: haobo, igor, sdong, dhruba, yhchiang Reviewed By: sdong CC: leveldb Differential Revision: https://reviews.facebook.net/D17643 | 14 April 2014, 16:34:59 UTC |
| d5e087b | sdong | 11 April 2014, 19:15:09 UTC | db_bench: add a mode to operate multiple DBs Summary: This patch introduces a new parameter num_multi_db in db_bench. When this parameter is larger than 1, multiple DBs will be created. In all benchmarks, any operation applies to a random DB among them. This is to benchmark the performance of similar applications. Test Plan: run db_bench on both of num_multi_db=0 and more. Reviewers: haobo, ljin, igor Reviewed By: igor CC: igor, yhchiang, dhruba, nkg-, leveldb Differential Revision: https://reviews.facebook.net/D17769 | 11 April 2014, 23:59:08 UTC |
| 30aff72 | Igor Canadi | 11 April 2014, 21:48:20 UTC | Don't shadow in ColumnFamilyDescriptor | 11 April 2014, 21:48:20 UTC |
| eba3fc6 | Lei Jin | 11 April 2014, 19:48:38 UTC | make corruption_test:CompactionInputErrorParanoid deterministic Summary: it writes ~10M data, default L0 compaction trigger is 4, plus 2 writer buffer, so that can accommodate ~6M data before compaction happens for sure. I guess encoding is doing a good job to shrink the data so that sometime, compaction does not get triggered. I get test failure quite often. Test Plan: ran it multiple times and all got pass Reviewers: igor, sdong Reviewed By: sdong CC: leveldb Differential Revision: https://reviews.facebook.net/D17775 | 11 April 2014, 19:48:38 UTC |
| 9433e35 | Igor Canadi | 11 April 2014, 17:54:47 UTC | Dont build version if compiling for IOS | 11 April 2014, 17:54:47 UTC |
| de41357 | Igor Canadi | 11 April 2014, 17:19:58 UTC | Don't dump rocksdb version on IOS | 11 April 2014, 17:19:58 UTC |
| 0af36d6 | Lei Jin | 11 April 2014, 16:47:20 UTC | SeekRandomWhileWriting Summary: as title Test Plan: ran it Reviewers: igor, haobo, yhchiang Reviewed By: yhchiang CC: leveldb Differential Revision: https://reviews.facebook.net/D17751 | 11 April 2014, 16:47:20 UTC |
| e23e73e | Kai Liu | 11 April 2014, 00:23:49 UTC | Use shorten index key for hash-index Summary: I was wrong about the "index builder", right now since we create index by scanning both whole table and index, there is not need to preserve the whole key as the index key. I switch back to original way index which is both space efficient and able to supprot in-fly construction of hash index. IN this patch, I made minimal change since I'm not sure if we still need the "pluggable index builder", under current circumstance it is of no use and kind of over-engineered. But I'm not sure if we can still exploit its usefulness in the future; otherwise I think I can just burn them with great vengeance. Test Plan: unit tests Reviewers: sdong, haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D17745 | 11 April 2014, 05:45:25 UTC |
| b3d7435 | Igor Canadi | 11 April 2014, 00:18:55 UTC | No shadow in public headers | 11 April 2014, 00:19:03 UTC |
| 1405232 | Kai Liu | 11 April 2014, 00:16:23 UTC | Temporarily disable a test case in db_test Summary: Root cause is still under investigation. Just Disable the troubling use case for now. | 11 April 2014, 00:17:39 UTC |
| 6c0fc5d | Igor Canadi | 10 April 2014, 23:49:49 UTC | Fix some InfoLogLevels | 10 April 2014, 23:49:49 UTC |
| dfe2d2f | Yueh-Hsuan Chiang | 10 April 2014, 23:12:04 UTC | [JNI] Add readwhilewriting to Java db_bench Summary: Add readwhilewriting to Java db_bench Test Plan: make jni make jdb_bench Reviewers: haobo, ankgup87, sdong, dhruba Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D17619 | 10 April 2014, 23:12:04 UTC |
| ddef684 | Igor Canadi | 10 April 2014, 22:27:42 UTC | Renamed InfoLogLevel::DEBUG to InfoLogLevel::DEBUG_LEVEL Summary: XCode for some reason injects `#define DEBUG 1` into our code, which makes compile fail because we use `DEBUG` keyword for other stuff. This diff fixes the issue by renaming `DEBUG` to `DEBUG_LEVEL`. Test Plan: compiles Reviewers: dhruba, haobo, sdong, yhchiang, ljin Reviewed By: yhchiang CC: leveldb Differential Revision: https://reviews.facebook.net/D17709 | 10 April 2014, 22:27:42 UTC |
| 75b59d5 | Kai Liu | 10 April 2014, 21:19:43 UTC | Enable hash index for block-based table Summary: Based on previous patches, this diff eventually provides the end-to-end mechanism for users to specify the hash-index. Test Plan: Wrote several new unit tests. Reviewers: sdong, haobo, dhruba Reviewed By: sdong CC: leveldb Differential Revision: https://reviews.facebook.net/D16539 | 10 April 2014, 21:19:43 UTC |
| 7a92537 | Lei Jin | 10 April 2014, 17:15:59 UTC | db_bench: add IteratorCreationWhileWriting mode and allow prefix_seek Summary: as title Test Plan: ran it Reviewers: igor, haobo, yhchiang Reviewed By: igor CC: leveldb Differential Revision: https://reviews.facebook.net/D17655 | 10 April 2014, 17:15:59 UTC |
| ca4fa20 | Yueh-Hsuan Chiang | 10 April 2014, 17:04:48 UTC | [Java] rename 'make jni' to 'make rocksdbjava' | 10 April 2014, 17:04:48 UTC |
| 4daea66 | Igor Canadi | 10 April 2014, 04:17:14 UTC | Turn on -Wmissing-prototypes Summary: Compiling for iOS has by default turned on -Wmissing-prototypes, which causes rocksdb to fail compiling. This diff turns on -Wmissing-prototypes in our compile options and cleans up all functions with missing prototypes. Test Plan: compiles Reviewers: dhruba, haobo, ljin, sdong Reviewed By: ljin CC: leveldb Differential Revision: https://reviews.facebook.net/D17649 | 10 April 2014, 04:17:14 UTC |
| df2a8b6 | sdong | 09 April 2014, 00:30:45 UTC | Polish IterKey and use it in DBImpl::ProcessKeyValueCompaction() Summary: 1. Polish IterKey a little bit. 2. Turn to use it in local parameter of current_user_key in DBImpl::ProcessKeyValueCompaction(). Our profile showing that DBImpl::ProcessKeyValueCompaction() has about 14% costs in std::string (the base including reading and writing data but excluding compaction filtering), which is higher than it should be. There are two std::string used in DBImpl::ProcessKeyValueCompaction(), compaction_filter_value and current_user_key and it's hard to distinguish the two. Test Plan: make all check Reviewers: haobo, ljin Reviewed By: haobo CC: igor, yhchiang, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D17613 | 10 April 2014, 03:50:58 UTC |
| dc55903 | Igor Canadi | 09 April 2014, 18:43:14 UTC | Improved CompressedCache Summary: This is testing behavior that was reported in https://github.com/facebook/rocksdb/issues/111 No issue was found, but it still good to commit this and make CompressedCache more robust. Test Plan: this is a plan Reviewers: ljin, dhruba Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D17625 | 09 April 2014, 18:43:14 UTC |
| 4824014 | Lei Jin | 09 April 2014, 18:25:21 UTC | speed up db_bench filluniquerandom mode Summary: filluniquerandom is painfully slow due to the naive bitmap check to find out if a key has been seen before. Majority of time is spent on searching the last few keys. Split a giant BitSet to smaller ones so that we can quickly check if a BitSet is full and thus can skip quickly. It used to take over one hour to filluniquerandom for 100M keys, now it takes about 10 mins. Test Plan: unit test also verified correctness in db_bench and make sure all keys are generated Reviewers: igor, haobo, yhchiang Reviewed By: igor CC: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D17607 | 09 April 2014, 18:25:21 UTC |
| 2014915 | Igor Canadi | 09 April 2014, 17:38:05 UTC | Fix ASAN issue | 09 April 2014, 17:38:05 UTC |
| e9ed28f | sdong | 09 April 2014, 16:44:23 UTC | PlainTableBuilder::Add() to use local char array instead of reused std::string as tmp buffer Summary: Our profile shows that in one of the applications, 5% of the CPU costs of PlainTableBuilder::Add() are spent on std::string stacks. By this simple change, we avoid this global reusable string. Also, we avoid another call of file appending, which probably gives another 2%. Test Plan: make all check Reviewers: haobo, ljin Reviewed By: haobo CC: igor, yhchiang, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D17601 | 09 April 2014, 17:17:32 UTC |
| b947fdc | Igor Canadi | 09 April 2014, 16:56:17 UTC | Column family support for DB::OpenForReadOnly() Summary: When opening DB in read-only mode, client can choose to only specify a subset of column families ("default" column family can't be omitted, though) Test Plan: added a unit test in column_family_test Reviewers: haobo, sdong, ljin, dhruba Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D17565 | 09 April 2014, 16:56:17 UTC |
| 0f5cbcd | Yueh-Hsuan Chiang | 09 April 2014, 07:48:20 UTC | [JNI] Add an initial benchmark for java binding for rocksdb. Summary: * Add a benchmark for java binding for rocksdb. The java benchmark is a complete rewrite based on the c++ db/db_bench.cc and the DbBenchmark in dain's java leveldb. * Support multithreading. * 'readseq' is currently not supported as it requires RocksDB Iterator. * usage: --benchmarks Comma-separated list of operations to run in the specified order Actual benchmarks: fillseq -- write N values in sequential key order in async mode fillrandom -- write N values in random key order in async mode fillbatch -- write N/1000 batch where each batch has 1000 values in random key order in sync mode fillsync -- write N/100 values in random key order in sync mode fill100K -- write N/1000 100K values in random order in async mode readseq -- read N times sequentially readrandom -- read N times in random order readhot -- read N times in random order from 1% section of DB Meta Operations: delete -- delete DB DEFAULT: [fillseq, readrandom, fillrandom] --compression_ratio Arrange to generate values that shrink to this fraction of their original size after compression DEFAULT: 0.5 --use_existing_db If true, do not destroy the existing database. If you set this flag and also specify a benchmark that wants a fresh database, that benchmark will fail. DEFAULT: false --num Number of key/values to place in database. DEFAULT: 1000000 --threads Number of concurrent threads to run. DEFAULT: 1 --reads Number of read operations to do. If negative, do --nums reads. --key_size The size of each key in bytes. DEFAULT: 16 --value_size The size of each value in bytes. DEFAULT: 100 --write_buffer_size Number of bytes to buffer in memtable before compacting (initialized to default value by 'main'.) DEFAULT: 4194304 --cache_size Number of bytes to use as a cache of uncompressed data. Negative means use default settings. DEFAULT: -1 --seed Seed base for random number generators. DEFAULT: 0 --db Use the db with the following name. DEFAULT: /tmp/rocksdbjni-bench * Add RocksDB.write(). Test Plan: make jbench Reviewers: haobo, sdong, dhruba, ankgup87 Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D17433 | 09 April 2014, 07:48:20 UTC |
| c65448f | Yueh-Hsuan Chiang | 09 April 2014, 05:21:39 UTC | Merge branch 'master' of github.com:facebook/rocksdb into HEAD | 09 April 2014, 05:21:39 UTC |
| 731e55c | Igor Canadi | 08 April 2014, 21:57:00 UTC | Fix GetProperty() test Summary: GetProperty test is flakey. Before this diff: P8635927 After: P8635945 We need to make sure the thread is done before we destruct sleeping tasks. Otherwise, bad things happen. Test Plan: See summary Reviewers: ljin, sdong, haobo, dhruba Reviewed By: ljin CC: leveldb Differential Revision: https://reviews.facebook.net/D17595 | 08 April 2014, 21:57:00 UTC |
| 2e0d432 | Yueh-Hsuan Chiang | 08 April 2014, 21:15:55 UTC | Merge pull request #110 from ankgup87/jni [JNI] Add JNI bindings for rocksdb Options | 08 April 2014, 21:15:55 UTC |
| 34455de | Igor Canadi | 08 April 2014, 21:01:09 UTC | Fix Mac OS compile issues | 08 April 2014, 21:05:53 UTC |
| 5b345b7 | Igor Canadi | 08 April 2014, 20:40:42 UTC | Remove env_ from MergingIterator Summary: env_ is not used. Compiling for iOS complains. Test Plan: compiles now Reviewers: ljin, haobo, sdong, dhruba Reviewed By: ljin CC: leveldb Differential Revision: https://reviews.facebook.net/D17589 | 08 April 2014, 20:40:42 UTC |
| 0c1126d | Lei Jin | 08 April 2014, 18:21:09 UTC | db_bench cleanup Summary: clean up the db_bench a little bit. also avoid allocating memory for key in the loop Test Plan: I verified a run with filluniquerandom & readrandom. Iterator seek will be used lot to measure performance. Will fix whatever comes up Reviewers: haobo, igor, yhchiang Reviewed By: igor CC: leveldb Differential Revision: https://reviews.facebook.net/D17559 | 08 April 2014, 18:21:09 UTC |
| beeee9d | Igor Canadi | 08 April 2014, 18:06:39 UTC | Small speedup of CompactionFilterV2 Summary: ToString() is expensive. Profiling shows that most compaction threads are stuck in jemalloc, allocating a new string. This will help out a litte. Test Plan: make check Reviewers: haobo, danguo Reviewed By: danguo CC: leveldb Differential Revision: https://reviews.facebook.net/D17583 | 08 April 2014, 18:06:39 UTC |
| 92c1eb0 | Lei Jin | 08 April 2014, 17:58:07 UTC | macros for perf_context Summary: This will allow us to disable them completely for iOS or for better performance Test Plan: will run make all check Reviewers: igor, haobo, dhruba Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D17511 | 08 April 2014, 17:58:07 UTC |
| 5abae2c | Yueh-Hsuan Chiang | 08 April 2014, 16:52:26 UTC | [JNI] Fixed a column family related compile error. Summary: Fixed the following column family related compile error. ./java/rocksjni/write_batch.cc:211:66: error: cannot initialize a parameter of type 'rocksdb::ColumnFamilyMemTables *' with an lvalue of type 'rocksdb::MemTable *' rocksdb::Status s = rocksdb::WriteBatchInternal::InsertInto(b, mem, &options); ^~~ Test Plan: make jni make jtest Reviewers: igor, haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D17577 | 08 April 2014, 16:54:59 UTC |
| 8a50971 | Ankit Gupta | 08 April 2014, 16:30:35 UTC | Fix formatting issues | 08 April 2014, 16:30:35 UTC |
| 1422044 | Ankit Gupta | 08 April 2014, 16:27:24 UTC | Fix formatting | 08 April 2014, 16:27:24 UTC |
| 5e2db3b | sdong | 07 April 2014, 23:56:26 UTC | PlainTableIterator not to store copied key in std::string Summary: Move PlainTableIterator's copied key from std::string local buffer to avoid paying the extra costs in std::string related to sharing. Reuse the same buffer class in DbIter. Move the class to dbformat.h. This patch improves iterator performance significantly. Running this benchmark: ./table_reader_bench --num_keys2=17 --iterator --plain_table --time_unit=nanosecond The average latency is improved to about 750 nanoseconds from 1100 nanoseconds. Test Plan: Add a unit test. make all check Reviewers: haobo, ljin Reviewed By: haobo CC: igor, yhchiang, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D17547 | 08 April 2014, 02:06:09 UTC |
| 22d45de | Ankit Gupta | 08 April 2014, 01:32:09 UTC | Fix formatting errors | 08 April 2014, 01:32:09 UTC |
| a4d73dd | Igor Canadi | 07 April 2014, 23:33:03 UTC | Initialize Options::bloom_locality Summary: I think this issue was caused by bad merge. We have to initialize bloom_locality, otherwise valgrind complains: "Use of uninitialised value of size 8" Test Plan: Run valgrind ./prefix_test Reviewers: ljin Reviewed By: ljin CC: leveldb Differential Revision: https://reviews.facebook.net/D17553 | 07 April 2014, 23:33:03 UTC |
| 664559f | Igor Canadi | 07 April 2014, 22:38:53 UTC | Small final fixes before merge | 07 April 2014, 22:38:53 UTC |
| d1e2bce | Igor Canadi | 07 April 2014, 22:03:15 UTC | CallFlushDuringCompaction | 07 April 2014, 22:03:15 UTC |
| b42ceb9 | Igor Canadi | 07 April 2014, 21:21:25 UTC | Simplify cleanup of dead (refcount == 0) column families | 07 April 2014, 21:31:02 UTC |
| e48348d | Igor Canadi | 07 April 2014, 20:46:51 UTC | Make flush part of compaction process This will enable user to use only 1 background thread. | 07 April 2014, 20:53:08 UTC |
| 2a0917b | Igor Canadi | 07 April 2014, 20:04:25 UTC | Merge branch 'master' into columnfamilies | 07 April 2014, 20:04:25 UTC |
| f6ed420 | Igor Canadi | 07 April 2014, 20:04:16 UTC | Fix default path in geodb_test | 07 April 2014, 20:04:16 UTC |
| 751e4b1 | Igor Canadi | 07 April 2014, 18:36:03 UTC | Fix wal_dir sanitizing | 07 April 2014, 18:36:03 UTC |
| 3d2fe84 | Igor Canadi | 07 April 2014, 18:29:48 UTC | Merge branch 'master' into columnfamilies Conflicts: db/db_impl.cc db/db_impl.h db/memtable_list.cc db/version_set.cc | 07 April 2014, 18:31:11 UTC |
