| 37c6740 | Lei Jin | 11 August 2014, 22:04:41 UTC | make statistics ToString function empty instead of pure virtual Summary: as title Test Plan: make release Reviewers: yhchiang, sdong Reviewed By: sdong Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D21549 | 11 August 2014, 22:04:41 UTC |
| 18efdba | Siying Dong | 11 August 2014, 18:10:23 UTC | Merge pull request #228 from miguelportilla/develop Changes to support unity build: Script for building the unity.cc file via Makefile Unity executable Makefile target for testing builds Source code changes to fix compilation of unity build | 11 August 2014, 18:10:23 UTC |
| d3f2ec6 | Feng Zhu | 11 August 2014, 17:47:52 UTC | check prefix_size when using hash search in db_bench Summary: 1. Check prefix_size when enable use_hash_search in db_bench 2. Remove include/statistics.h in db_bench Test Plan: ./db_bench --use_hash_search=1 Reviewers: ljin, yhchiang, igor, sdong Reviewed By: sdong Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D21375 | 11 August 2014, 17:47:52 UTC |
| 93e6b5e | miguelportilla | 07 August 2014, 17:05:04 UTC | Changes to support unity build: * Script for building the unity.cc file via Makefile * Unity executable Makefile target for testing builds * Source code changes to fix compilation of unity build | 11 August 2014, 17:22:47 UTC |
| 54153ab | sdong | 08 August 2014, 19:03:30 UTC | Include candidate files under options.db_log_dir in FindObsoleteFiles() Summary: In FindObsoleteFiles(), we don't scan db_log_dir. Add it. Test Plan: make all check Reviewers: ljin, igor, yhchiang Reviewed By: yhchiang Subscribers: leveldb, yhchiang Differential Revision: https://reviews.facebook.net/D21429 | 09 August 2014, 00:37:03 UTC |
| 4632239 | sdong | 07 August 2014, 20:19:58 UTC | Need to schedule compactions when manual compaction finishes Summary: If there is an outstanding compaction scheduled but at the time a manual compaction is triggered, the manual compaction will preempt. In the end of the manual compaction, we should try to schedule compactions to make sure those preempted ones are not skipped. Test Plan: make all check Reviewers: yhchiang, ljin Reviewed By: ljin Subscribers: leveldb, dhruba, igor Differential Revision: https://reviews.facebook.net/D21321 | 08 August 2014, 19:28:36 UTC |
| 5e08681 | Igor Canadi | 08 August 2014, 17:24:00 UTC | Fix SIGSEGV in travis Summary: Travis build was failing a lot. For example see https://travis-ci.org/facebook/rocksdb/builds/31425845 This fixes it. Also, please don't put any code after SignalAll :) Test Plan: no more SIGSEGV Reviewers: yhchiang, sdong, ljin Reviewed By: ljin Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D21417 | 08 August 2014, 17:24:00 UTC |
| 7c88249 | Stanislau Hlebik | 08 August 2014, 16:44:14 UTC | Fix db_test and DBIter Summary: Fix old issue with DBTest.Randomized with BlockBasedTableWithWholeKeyHashIndex + added printing in DBTest.Randomized. Test Plan: make all check Reviewers: zagfox, igor, ljin, yhchiang, sdong Reviewed By: sdong Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D21003 | 08 August 2014, 16:44:14 UTC |
| 894a77a | Igor Canadi | 07 August 2014, 22:06:52 UTC | Fix leak in c_test | 07 August 2014, 22:06:52 UTC |
| 323d6e3 | Igor Canadi | 07 August 2014, 18:29:38 UTC | Fix c_test | 07 August 2014, 18:29:38 UTC |
| f8d6a29 | Igor Canadi | 07 August 2014, 18:10:54 UTC | Merge pull request #224 from cockroachdb/spencerkimball/compaction-filter-v2-c-bindings Add support for C bindings to the compaction V2 filter mechanism. | 07 August 2014, 18:10:54 UTC |
| 1a1957b | Igor Canadi | 07 August 2014, 14:40:11 UTC | Merge pull request #226 from bbiao/master fix various 'comparison of integers of different signs' compiling errors... | 07 August 2014, 14:40:11 UTC |
| 63d5cc7 | ZHANG Biao | 07 August 2014, 09:06:07 UTC | fix various 'comparison of integers of different signs' compiling errors under macosx | 07 August 2014, 09:06:07 UTC |
| 76dcf7e | sdong | 07 August 2014, 00:46:26 UTC | Minor: fix a format Summary: A format fixing Test Plan: N/A Reviewers: ljin Reviewed By: ljin Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D21255 | 07 August 2014, 01:11:33 UTC |
| 7abe965 | sdong | 07 August 2014, 00:37:35 UTC | Fix valgrind failure caused by recent checked-in. Summary: Initialize un-initialized parameters Test Plan: run the failed test (c_test) Reviewers: yhchiang, ljin Reviewed By: ljin Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D21249 | 07 August 2014, 00:45:47 UTC |
| 38e8b72 | Spencer Kimball | 06 August 2014, 22:42:15 UTC | Fix typo, add missing inclusion of state void* in invocation of create_compaction_filter_v2_. | 06 August 2014, 22:42:15 UTC |
| c1f588a | Spencer Kimball | 06 August 2014, 18:06:41 UTC | Add support for C bindings to the compaction V2 filter mechanism. Test Plan: make c_test && ./c_test Some fixes after merge. | 06 August 2014, 19:55:48 UTC |
| 1242bfc | sdong | 05 August 2014, 18:27:34 UTC | Add DB property "rocksdb.estimate-table-readers-mem" Summary: Add a DB Property "rocksdb.estimate-table-readers-mem" to return estimated memory usage by all loaded table readers, other than allocated from block cache. Refactor the property codes to allow getting property from a version, with DB mutex not acquired. Test Plan: Add several checks of this new property in existing codes for various cases. Reviewers: yhchiang, ljin Reviewed By: ljin Subscribers: xjin, igor, leveldb Differential Revision: https://reviews.facebook.net/D20733 | 06 August 2014, 18:39:46 UTC |
| 606a126 | Radheshyam Balasundaram | 06 August 2014, 03:55:46 UTC | Changing implementaiton of CuckooTableBuilder to not take file_size, key_length, value_length. Summary: - Maintain a list of key-value pairs as vectors during Add operation. - Start building hash table only when Finish() is called. - This approach takes more time and space but avoids taking file_size, key and value lengths. - Rewrote cuckoo_table_builder_test I did not know about IterKey while writing this diff. I shall change places where IterKey could be used instead of std::string tomorrow. Please review rest of the logic. Test Plan: cuckoo_table_reader_test --enable_perf cuckoo_table_builder_test valgrind_check asan_check Reviewers: sdong, igor, yhchiang, ljin Reviewed By: ljin Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20907 | 06 August 2014, 03:55:46 UTC |
| 2124c85 | Radheshyam Balasundaram | 05 August 2014, 23:35:02 UTC | Implementing CuckooTableReader::NewIterator Summary: - Reads key-value pairs from file and builds an in-memory index of key-to-bucket id map in sorted order of key. - Assumes bytewise comparator for sorting keys. - Test changes Test Plan: cuckoo_table_reader_test --enable_perf valgrind_check asan_check Reviewers: yhchiang, sdong, ljin Reviewed By: ljin Subscribers: leveldb, igor Differential Revision: https://reviews.facebook.net/D20721 | 05 August 2014, 23:35:02 UTC |
| 02c4023 | sdong | 05 August 2014, 18:24:01 UTC | Remove port::MemoryBarrier() from table_reader_bench Summary: port::MemoryBarrier() is not recommended to use outside of port. Remove it. Test Plan: run table_reader_bench Reviewers: ljin, yhchiang, igor Reviewed By: igor Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D21075 | 05 August 2014, 18:33:56 UTC |
| 1129921 | Feng Zhu | 04 August 2014, 18:25:42 UTC | logging_when_create_and_delete_manifest Summary: 1. logging when create and delete manifest file 2. fix formating in table/format.cc Test Plan: make all check run db_bench, track the LOG file. Reviewers: ljin, yhchiang, igor, yufei.zhu, sdong Reviewed By: sdong Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D21009 | 04 August 2014, 18:25:42 UTC |
| 5e3d5c5 | Igor Canadi | 01 August 2014, 20:50:40 UTC | Simplify SpatialIndexCursor Summary: Since we have enough memory to hold all primary keys loaded from spatial index, it is better if we first load all of them (store them in unordered_set for deduplication) and then query on primary key column family one by one. We need to dedup all IDs, so we'll end up storing all of them in memory even with the current approach. Test Plan: ./spatial_db_test is happy Reviewers: yinwang Reviewed By: yinwang Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20949 | 01 August 2014, 20:50:40 UTC |
| 9c5a3f4 | Igor Canadi | 01 August 2014, 15:55:37 UTC | FeatureSet DebugString Summary: This will help debugging Test Plan: ran, observed output Reviewers: yinwang Reviewed By: yinwang Differential Revision: https://reviews.facebook.net/D20937 | 01 August 2014, 15:55:37 UTC |
| e4c3673 | Igor Canadi | 01 August 2014, 13:41:48 UTC | Never CompactRange to level 0 in level compaction Summary: I was bit by this when developing SpatialDB. In case all files are at level 0, CompactRange() will output the compacted files to level 0. This is not ideal, since read amp. is much better at level 1 and higher. Test Plan: Compacted data in SpatialDB, read manifest using ldb, verified that files are now at level 1 instead of 0. Reviewers: sdong, ljin, yhchiang, dhruba Reviewed By: dhruba Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D20901 | 01 August 2014, 13:41:48 UTC |
| 0c9d03b | Radheshyam Balasundaram | 01 August 2014, 03:52:13 UTC | Fixing broken Mac build Summary: Made some small changes to fix the broken mac build Test Plan: make check all in both linux and mac. All tests pass. Reviewers: sdong, igor, ljin, yhchiang Reviewed By: ljin, yhchiang Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20895 | 01 August 2014, 03:52:13 UTC |
| 1903aa5 | Yueh-Hsuan Chiang | 31 July 2014, 21:48:00 UTC | Fixed a warning / error in signed and unsigned comparison Summary: Fixed the following compilation error detected in mac: db/db_test.cc:2524:3: note: in instantiation of function template specialization 'rocksdb::test::Tester::IsEq<unsigned long long, int>' requested here ASSERT_EQ(int_num, 0); ^ Test Plan: make | 31 July 2014, 21:48:00 UTC |
| 67dae25 | Yueh-Hsuan Chiang | 31 July 2014, 19:58:16 UTC | Remove a check for merge operator in builder.cc Summary: Previously, builder.cc has a check for merge operator which prevents RocksDB from crash when reopening a DB w/o properly specifying the merge operator. However, currently we observed a memory leak on failing in RocksDB recovery. This diff removes such check and let it crash instead of causing memory leak for now before we have identified the real cause of the memory leak. Test Plan: make all check Reviewers: sdong Subscribers: ljin, igor Differential Revision: https://reviews.facebook.net/D20913 | 31 July 2014, 21:22:21 UTC |
| 2105eca | Yueh-Hsuan Chiang | 31 July 2014, 18:20:49 UTC | Temporary remove the last test in merge_test | 31 July 2014, 18:20:49 UTC |
| b099901 | Feng Zhu | 31 July 2014, 06:11:59 UTC | use stack instead of heap memory in ReadBlockContents in some case Summary: When compression is enabled, and blocksize is not too big, use the space in stack to hold bytes read from block. Bencmark: base version: commit 8f09d53fd11a7debe1e48b73a192de3a458d37bf malloc: 1.30% -> 0.98% free: 1.49% -> 1.07% Test Plan: make all check Reviewers: ljin, yhchiang, dhruba, igor, sdong Reviewed By: sdong Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20679 | 31 July 2014, 06:11:59 UTC |
| 2ea5e78 | Yueh-Hsuan Chiang | 31 July 2014, 04:04:27 UTC | Merge pull request #217 from zxcvdavid/patch-1 fix project name in the comments | 31 July 2014, 04:04:27 UTC |
| 7ef7df0 | Demon | 31 July 2014, 03:42:36 UTC | fix project name in the comments fix project name in the comments | 31 July 2014, 03:42:36 UTC |
| 3215967 | Stanislau Hlebik | 31 July 2014, 01:21:55 UTC | Fix readonly db Summary: DBImplReadOnly::CompactRange wasn't override DBImpl::CompactRange; this can cause problem when using StackableDB inheritors like DbWithTtl. P. S. Thanks C++11 for override :) Test Plan: make all check Reviewers: igor, sdong Reviewed By: sdong Subscribers: yhchiang, leveldb Differential Revision: https://reviews.facebook.net/D20829 | 31 July 2014, 01:21:55 UTC |
| e9269e6 | Yueh-Hsuan Chiang | 31 July 2014, 00:25:11 UTC | Fixed a typo in the comment for merge operator. Summary: Fixed a typo in the comment for merge operator. Test Plan: n/a | 31 July 2014, 00:25:11 UTC |
| 49ee5a4 | Yueh-Hsuan Chiang | 31 July 2014, 00:24:36 UTC | Fixed the crash when merge_operator is not properly set after reopen. Summary: Fixed the crash when merge_operator is not properly set after reopen and added two test cases for this. Test Plan: make merge_test ./merge_test Reviewers: igor, ljin, sdong Reviewed By: sdong Subscribers: benj, mvikjord, leveldb Differential Revision: https://reviews.facebook.net/D20793 | 31 July 2014, 00:24:36 UTC |
| 8f09d53 | Feng Zhu | 30 July 2014, 23:34:35 UTC | remove malloc when create data and index iterator in Get Summary: Define Block::Iter to be an independent class to be used by block_based_table_reader When creating data and index iterator, update an existing iterator rather than new one Thus malloc and free could be reduced Benchmark, Base: commit 76286ee67ef4b89579a92134b996a681c36a1331 commands: --db=/dev/shm/rocksdb --num_levels=6 --key_size=20 --prefix_size=20 --keys_per_prefix=0 --value_size=100 --write_buffer_size=134217728 --max_write_buffer_number=2 --target_file_size_base=33554432 --max_bytes_for_level_base=1073741824 --verify_checksum=false --max_background_compactions=4 --use_plain_table=0 --memtablerep=prefix_hash --open_files=-1 --mmap_read=1 --mmap_write=0 --bloom_bits=10 --bloom_locality=1 --memtable_bloom_bits=500000 --compression_type=lz4 --num=2621440 --use_hash_search=1 --block_size=1024 --block_restart_interval=1 --use_existing_db=1 --threads=1 --benchmarks=readrandom —disable_auto_compactions=1 malloc: 3.30% -> 1.42% free: 3.59%->1.61% Test Plan: make all check run db_stress valgrind ./db_test ./table_test Reviewers: ljin, yhchiang, dhruba, igor, sdong Reviewed By: sdong Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20655 | 30 July 2014, 23:34:35 UTC |
| 76286ee | Stanislau Hlebik | 30 July 2014, 20:53:08 UTC | Remove unnecessary constructor parameter from ColumnFamilyData Summary: const string& dbname parameter is not used Test Plan: make all Reviewers: sdong, igor Reviewed By: sdong Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20703 | 30 July 2014, 20:53:08 UTC |
| 473e829 | sdong | 30 July 2014, 01:37:00 UTC | Fix ldb dump_manifest Summary: We now reads table properties in VersionSet::LogAndApply(), which requires options.db_paths to be set. But since ldb_cmd directly creates VersionSet without initialization db_paths, causing a seg fault. This patch fix it by initializing db_paths. log_and_apply_bench still shows segfault, because table cache is nullptr in VersionSet created. Test Plan: Run ldb dump_manifest which used to fail. Reviewers: yhchiang, ljin, igor Reviewed By: igor Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20751 | 30 July 2014, 17:17:48 UTC |
| 99e03bc | Igor Canadi | 30 July 2014, 16:32:47 UTC | Better comment for inplace_update_support Summary: See https://github.com/facebook/rocksdb/issues/215 Test Plan: none Reviewers: dhruba, sdong, ljin, yhchiang, nkg- Reviewed By: nkg- Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20769 | 30 July 2014, 16:32:47 UTC |
| 91c0148 | Radheshyam Balasundaram | 29 July 2014, 00:14:25 UTC | Minor changes to CuckooTableBuilder Summary: - Copy the key and value to in-memory hash table during Add operation. Also modified cuckoo_table_reader_test to use this. - Store only the user_key in in-memory hash table if it is last level file. - Handle Carryover while chosing unused key in Finish() method in case unused key was never found before Finish() call. Test Plan: cuckoo_table_reader_test --enable_perf cuckoo_table_builder_test valgrind_check asan_check Reviewers: sdong, yhchiang, igor, ljin Reviewed By: ljin Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20715 | 29 July 2014, 00:14:25 UTC |
| f04356e | sdong | 28 July 2014, 22:28:53 UTC | Add DB::GetIntProperty() to return integer properties to be returned as integers Summary: We have quite some properties that are integers and we are adding more. Add a function to directly return them as an integer, instead of a string Test Plan: Add several unit test checks Reviewers: yhchiang, igor, dhruba, haobo, ljin Reviewed By: ljin Subscribers: yoshinorim, leveldb Differential Revision: https://reviews.facebook.net/D20637 | 28 July 2014, 23:55:57 UTC |
| f678476 | sdong | 28 July 2014, 21:50:16 UTC | Add DB property estimated number of keys Summary: Add a DB property of estimated number of live keys, by adding number of entries of all mem tables and all files, subtracted by all deletions in all files. Test Plan: Add the case in unit tests Reviewers: hobbymanyp, ljin Reviewed By: ljin Subscribers: MarkCallaghan, yoshinorim, leveldb, igor, dhruba Differential Revision: https://reviews.facebook.net/D20631 | 28 July 2014, 22:27:58 UTC |
| 7e8bb71 | Lei Jin | 28 July 2014, 19:27:08 UTC | InternalStats to take cfd on constructor Summary: It has one-to-one relationship with CFD. Take a pointer to CFD on constructor to avoid passing cfd through member functions. Test Plan: make Reviewers: sdong, yhchiang, igor Reviewed By: igor Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20565 | 28 July 2014, 19:27:08 UTC |
| 1bd3431 | Lei Jin | 28 July 2014, 19:22:37 UTC | Change StopWatch interface Summary: So that we can avoid calling NowSecs() in MakeRoomForWrite twice Test Plan: make all check Reviewers: yhchiang, igor, sdong Reviewed By: sdong Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20529 | 28 July 2014, 19:22:37 UTC |
| f6ca226 | Lei Jin | 28 July 2014, 19:10:49 UTC | make statistics forward-able Summary: Make StatisticsImpl being able to forward stats to provided statistics implementation. The main purpose is to allow us to collect internal stats in the future even when user supplies custom statistics implementation. It avoids intrumenting 2 sets of stats collection code. One immediate use case is tuning advisor, which needs to collect some internal stats, users may not be interested. Test Plan: ran db_bench and see stats show up at the end of run Will run make all check since some tests rely on statistics Reviewers: yhchiang, sdong, igor Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D20145 | 28 July 2014, 19:10:49 UTC |
| 40fa8a4 | Lei Jin | 28 July 2014, 19:05:36 UTC | make statistics forward-able Summary: Make StatisticsImpl being able to forward stats to provided statistics implementation. The main purpose is to allow us to collect internal stats in the future even when user supplies custom statistics implementation. It avoids intrumenting 2 sets of stats collection code. One immediate use case is tuning advisor, which needs to collect some internal stats, users may not be interested. Test Plan: ran db_bench and see stats show up at the end of run Will run make all check since some tests rely on statistics Reviewers: yhchiang, sdong, igor Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D20145 | 28 July 2014, 19:05:36 UTC |
| 4a8f0c9 | sdong | 25 July 2014, 21:38:18 UTC | Block::Iter::PrefixSeek() to have an extra check to filter out some false matches Summary: In block based table's hash index checking, when looking for a key that doesn't exist, there is a high chance that a false block is returned because of hash bucket conflicts. In this revision, another check is done to filter out some of those cases: comparing previous key of the block boundary to see whether the target block is what we are looking for. In a favored test setting (bloom filter disabled, 8 L0 files), I saw about 80% improvements. In a non-favored test setting (bloom filter enabled, files are all in L1, files are all cached), I see the performance penalty is less than 3%. Test Plan: make all check Reviewers: haobo, ljin Reviewed By: ljin Subscribers: wuj, leveldb, zagfox, yhchiang Differential Revision: https://reviews.facebook.net/D20595 | 26 July 2014, 00:27:57 UTC |
| 62f9b07 | Radheshyam Balasundaram | 25 July 2014, 23:37:32 UTC | Implementation of CuckooTableReader Summary: Contains: - Implementation of TableReader based on Cuckoo Hashing - Unittests for CuckooTableReader - Performance test for TableReader Test Plan: make cuckoo_table_reader_test ./cuckoo_table_reader_test make valgrind_check make asan_check Reviewers: yhchiang, sdong, igor, ljin Reviewed By: ljin Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20511 | 25 July 2014, 23:37:32 UTC |
| d650612 | Lei Jin | 25 July 2014, 22:17:06 UTC | expose RateLimiter definition Summary: User gets undefinied error since the definition is not exposed. Also re-enable the db test with only upper bound check Test Plan: db_test, rate_limit_test Reviewers: igor, yhchiang, sdong Reviewed By: sdong Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20403 | 25 July 2014, 22:17:06 UTC |
| 28b367d | Igor Canadi | 25 July 2014, 17:29:50 UTC | Initialize next_id | 25 July 2014, 17:29:50 UTC |
| c8e70e6 | Igor Canadi | 25 July 2014, 00:26:48 UTC | Fix valgrind test | 25 July 2014, 00:31:23 UTC |
| 6480717 | Yueh-Hsuan Chiang | 25 July 2014, 00:06:00 UTC | Fixed compaction-related errors where number of input levels are hard-coded. Summary: Fixed compaction-related errors where number of input levels are hard-coded. It's a bug found in compaction branch. This diff will be pushed into master. Test Plan: export ROCKSDB_TESTS=Compact make db_test -j32 ./db_test also passed the tests in compaction branch Reviewers: igor, sdong, ljin Reviewed By: ljin Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20577 | 25 July 2014, 00:06:00 UTC |
| f780f35 | Igor Canadi | 24 July 2014, 20:45:26 UTC | Fix compile warning | 24 July 2014, 20:45:26 UTC |
| 0754d4c | Igor Canadi | 24 July 2014, 20:39:33 UTC | SpatialDB change API Summary: I changed SpatialDB API so that we only specify list of indexes when we create the database. That way, whoever is querying the DB doesn't need to know the full list of indexes and their options. Test Plan: spatial_db_test Reviewers: yinwang Reviewed By: yinwang Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20571 | 24 July 2014, 20:39:33 UTC |
| 07a7d87 | Radheshyam Balasundaram | 24 July 2014, 17:07:41 UTC | Addressing TODOs in CuckooTableBuilder Summary: Contains the following changes in CuckooTableBuilder: - Take an extra parameter in constructor to identify last level file. - Implement a better way to identify if a bucket has been inserted into the tree already during BFS search. - Minor typos Test Plan: make cuckoo_table_builder ./cuckoo_table_builder make valgrind_check Reviewers: sdong, igor, yhchiang, ljin Reviewed By: ljin Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20445 | 24 July 2014, 17:07:41 UTC |
| 4b61a3d | Yueh-Hsuan Chiang | 23 July 2014, 22:49:14 UTC | Merge branch 'ankgup87-master' | 23 July 2014, 22:49:14 UTC |
| 10fc6c7 | Yueh-Hsuan Chiang | 23 July 2014, 22:44:25 UTC | [Java] Add compaction style to options Summary: Add compression type to options make rocksdbjava make sample Reviewers: haobo, yhchiang, sdong, dhruba, rsumbaly, zzbennett, swapnilghike Reviewed By: yhchiang, sdong CC: leveldb Differential Revision: https://reviews.facebook.net/D20463 | 23 July 2014, 22:45:30 UTC |
| 41a6972 | Igor Canadi | 23 July 2014, 20:52:11 UTC | NewIterators in read-only mode Summary: As title. Test Plan: Added test to column_family_test Reviewers: ljin, yhchiang, sdong Reviewed By: sdong Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20523 | 23 July 2014, 20:52:11 UTC |
| e5f6980 | Igor Canadi | 23 July 2014, 20:49:58 UTC | Fix release compile error | 23 July 2014, 20:50:31 UTC |
| da92745 | Feng Zhu | 23 July 2014, 19:31:11 UTC | Use IterKey instead of string in Block::Iter to reduce malloc Summary: Modify a functioin TrimAppend in dbformat.h: IterKey. Write a test for it in dbformat_test Use IterKey in block::Iter to replace std::string to reduce malloc. Evaluate it using perf record. malloc: 4.26% -> 2.91% free: 3.61% -> 3.08% Test Plan: make all check ./valgrind db_test dbformat_test Reviewers: ljin, haobo, yhchiang, dhruba, igor, sdong Reviewed By: sdong Differential Revision: https://reviews.facebook.net/D20433 | 23 July 2014, 19:31:11 UTC |
| 6296330 | Igor Canadi | 23 July 2014, 18:22:58 UTC | SpatialDB Summary: This diff is adding spatial index support to RocksDB. When creating the DB user specifies a list of spatial indexes. Spatial indexes can cover different areas and have different resolution (i.e. number of tiles). This is useful for supporting different zoom levels. Each element inserted into SpatialDB has: * a bounding box, which determines how will the element be indexed * string blob, which will usually be WKB representation of the polygon (http://en.wikipedia.org/wiki/Well-known_text) * feature set, which is a map of key-value pairs, where value can be int, double, bool, null or a string. FeatureSet will be a set of tags associated with geo elements (for example, 'road': 'highway' and similar) * a list of indexes to insert the element in. For example, small river element will be inserted in index for high zoom level, while country border will be inserted in all indexes (including the index for low zoom level). Each query is executed on single spatial index. Query guarantees that it will return all elements intersecting the specified bounding box, but it might also return some extra non-intersecting elements. Test Plan: Added bunch of unit tests in spatial_db_test Reviewers: dhruba, yinwang Reviewed By: yinwang Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20361 | 23 July 2014, 18:22:58 UTC |
| b5c4c0b | Yueh-Hsuan Chiang | 23 July 2014, 18:16:18 UTC | [Java] Add the missing ROCKSDB_JAR variable in Makefile Summary: Add the missing ROCKSDB_JAR variable in Makefile, which is mistakenly removed in https://reviews.facebook.net/D20289. Test Plan: export ROCKSDB_JAR= make rocksdbjava | 23 July 2014, 18:16:18 UTC |
| 6e7e3e4 | Yueh-Hsuan Chiang | 23 July 2014, 18:15:14 UTC | [Java] Update header inclusion of utilities files Summary: Update header inclusion as include/utilities/*.h has been moved to include/rocksdb/utilities/*.h Test Plan: make rocksdbjava Reviewers: ljin, sdong, igor Reviewed By: igor Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20517 | 23 July 2014, 18:15:14 UTC |
| 00f56df | Yueh-Hsuan Chiang | 23 July 2014, 17:31:32 UTC | Fixed a compile error in util/options_builder.cc Summary: Fixed the following compile error by replacing pow by shift, as it computes power of 2. util/options_builder.cc:133:14: error: no member named 'pow' in namespace 'std' std::pow(2, std::max(0, std::min(3, level0_stop_writes_trigger - ~~~~~^ 1 error generated. make: *** [util/options_builder.o] Error 1 Test Plan: make success in mac and linux Reviewers: ljin, igor, sdong Reviewed By: sdong Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20475 | 23 July 2014, 17:31:32 UTC |
| 0e1b478 | Yueh-Hsuan Chiang | 23 July 2014, 17:22:21 UTC | Fixed a bug in Compaction.cc where input_levels_ was not properly resized. Summary: Fixed a bug in Compaction.cc where input_levels_ was not properly resized. Without this fix, there would be invalid access in input_levels_ when more than two levels are involved in one compaction run. This fix will go to master instead of compaction branch. Test Plan: tested in compaction branch. Reviewers: ljin, sdong, igor Reviewed By: igor Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20481 | 23 July 2014, 17:22:21 UTC |
| f82d4a2 | Igor Canadi | 23 July 2014, 14:28:41 UTC | Also bump version in Makefile | 23 July 2014, 14:28:41 UTC |
| 1053358 | Igor Canadi | 23 July 2014, 14:21:54 UTC | Bump the version | 23 July 2014, 14:21:54 UTC |
| 0ff183a | Igor Canadi | 23 July 2014, 14:21:38 UTC | Move include/utilities/*.h to include/rocksdb/utilities/*.h Summary: All public headers need to be under `include/rocksdb` directory. Otherwise, clients include our header files like this: #include <rocksdb/db.h> #include <utilities/backupable_db.h> // still our public header! Also, internally, we include: #include "utilities/backupable/backupable_db.h" // internal header #include "utilities/backupable_db.h" // public header which is confusing. This way, when we install rocksdb as a system library, we can just copy `include/rocksdb` directory to system's header files. We can't really copy `utilities` directory to system's header files. Test Plan: compiles Reviewers: dhruba, ljin, yhchiang, sdong Reviewed By: sdong Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20409 | 23 July 2014, 14:21:38 UTC |
| e6de021 | sdong | 16 May 2014, 17:35:41 UTC | Add a utility function to guess optimized options based on constraints Summary: Add a function GetOptions(), where based on four parameters users give: read/write amplification threshold, memory budget for mem tables and target DB size, it picks up a compaction style and parameters for them. Background threads are not touched yet. One limit of this algorithm: since compression rate and key/value size are hard to predict, it's hard to predict level 0 file size from write buffer size. Simply make 1:1 ratio here. Sample results: https://reviews.facebook.net/P477 Test Plan: Will add some a unit test where some sample scenarios are given and see they pick the results that make sense Reviewers: yhchiang, dhruba, haobo, igor, ljin Reviewed By: ljin Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D18741 | 22 July 2014, 22:24:21 UTC |
| 250f035 | Yueh-Hsuan Chiang | 22 July 2014, 21:53:12 UTC | [Java] Optimize statistics collector, improve object dependency in RocksObjects Summary: This diff merges pull request 208. Contributor: ankgup87 [Java] Optimize statistics collector * Optimize statistics collector by collecting statistics of multiple DBs in a single thread rather than starting up a new thread for each DB. * Also, fix packaging of jnilib file on OS_X platform. * Diff review: https://reviews.facebook.net/D20265 [Java] Add documentation on interdependency of dispose call of RocksObjects * Remove transferCppRawPointersOwnershipFrom function. - This function was setting opt.filter_ and thus filter_ to be null. This way there is no one holding reference for filter object and can thus be GC'd which is not the intention. Replaced it with storeOptionsInstace which stores options instance. Options class internally holds Filter instance. Thus when Options is GC'd, filter reference will be GC'd automatically. * Added documentation explaining interdependency of Filter, Options and DB. * Diff review: https://reviews.facebook.net/D20379 Test Plan: described in their diff reviews Reviewers: haobo sdong swapnilghike zzbennett rsumbaly yhchiang Reviewed by: yhchiang | 22 July 2014, 21:53:12 UTC |
| 25682d1 | Ankit Gupta | 10 July 2014, 06:13:28 UTC | [Java] Optimize statistics collector, improve object dependency in RocksObjects Summary: This diff merges pull request #208. Contributor: ankgup87 [Java] Optimize statistics collector * Optimize statistics collector by collecting statistics of multiple DBs in a single thread rather than starting up a new thread for each DB. * Also, fix packaging of jnilib file on OS_X platform. * Diff review: https://reviews.facebook.net/D20265 [Java] Add documentation on interdependency of dispose call of RocksObjects * Remove transferCppRawPointersOwnershipFrom function. - This function was setting opt.filter_ and thus filter_ to be null. This way there is no one holding reference for filter object and can thus be GC'd which is not the intention. Replaced it with storeOptionsInstace which stores options instance. Options class internally holds Filter instance. Thus when Options is GC'd, filter reference will be GC'd automatically. * Added documentation explaining interdependency of Filter, Options and DB. * Diff review: https://reviews.facebook.net/D20379 Test Plan: described in their diff reviews Reviewers: haobo sdong swapnilghike zzbennett rsumbaly yhchiang Reviewed by: yhchiang | 22 July 2014, 21:52:50 UTC |
| 2d3d635 | Igor Canadi | 22 July 2014, 19:35:07 UTC | Fix signed-unsigned compare error | 22 July 2014, 19:35:07 UTC |
| f6272e3 | Radheshyam Balasundaram | 22 July 2014, 16:49:04 UTC | Fixing memory leaks in cuckoo_table_builder_test Summary: Fixes some memory leaks in cuckoo_builder_test.cc. This also fixed broken valgrind_check tests Test Plan: make valgrind_check ./cuckoo_builder_test Currently running make check all. I shall update once it is done. Reviewers: ljin, sdong, yhchiang, igor Reviewed By: igor Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20385 | 22 July 2014, 16:49:04 UTC |
| d19aa25 | Yueh-Hsuan Chiang | 22 July 2014, 16:48:37 UTC | [Java] Add Java support for cache sharding. Summary: Add setCacheNumShardBits() and cacheNumShardBits() to Options. This allows developers to control the number of shards for the block cache. Test Plan: make rocksdbjava cd java make db_bench ./jdb_bench.sh --cache_size=1048576 --cache_numshardbits=6 Reviewers: sdong, ljin, ankgup87 Reviewed By: ankgup87 Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19347 | 22 July 2014, 16:48:37 UTC |
| ae7743f | Yueh-Hsuan Chiang | 22 July 2014, 05:41:54 UTC | Fixed some make and linking issues of RocksDBJava Summary: Fixed some make and linking issues of RocksDBJava. Specifically: * Add JAVA_LDFLAGS, which does not include gflags * rocksdbjava library now uses JAVA_LDFLAGS instead of LDFLAGS * java/Makefile now includes build_config.mk * rearrange make rocksdbjava workflow to ensure the library file is correctly included in the jar file. Test Plan: make rocksdbjava make jdb_bench java/jdb_bench.sh Reviewers: dhruba, swapnilghike, zzbennett, rsumbaly, ankgup87 Reviewed By: ankgup87 Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20289 | 22 July 2014, 05:41:54 UTC |
| f6b7e1e | sdong | 17 July 2014, 00:39:18 UTC | Allow user to specify DB path of output file of manual compaction Summary: Add a parameter path_id to DB::CompactRange(), to indicate where the output file should be placed to. Test Plan: add a unit test Reviewers: yhchiang, ljin Reviewed By: ljin Subscribers: xjin, igor, dhruba, MarkCallaghan, leveldb Differential Revision: https://reviews.facebook.net/D20085 | 22 July 2014, 02:06:00 UTC |
| bbe2e91 | Yueh-Hsuan Chiang | 21 July 2014, 22:17:09 UTC | Fixed a compile error of cuckoo_table_builder. Summary: Fixed the following compile error. ./table/cuckoo_table_builder.h:72:22: error: private field 'key_length_' is not used [-Werror,-Wunused-private-field] const unsigned int key_length_; ^ 1 error generated. Test Plan: make Reviewers: sdong, ljin, radheshyamb, igor Reviewed By: igor Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20349 | 21 July 2014, 22:17:09 UTC |
| cf3da89 | Radheshyam Balasundaram | 21 July 2014, 20:26:09 UTC | Adding a new SST table builder based on Cuckoo Hashing Summary: Cuckoo Hashing based SST table builder. Contains: - Cuckoo Hashing logic and file storage logic. - Unit tests for logic Test Plan: make cuckoo_table_builder_test ./cuckoo_table_builder_test make check all Reviewers: yhchiang, igor, sdong, ljin Reviewed By: ljin Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D19545 | 21 July 2014, 20:26:09 UTC |
| f6f1533 | Lei Jin | 21 July 2014, 19:57:29 UTC | make internal stats independent of statistics Summary: also make it aware of column family output from db_bench ``` ** Compaction Stats [default] ** Level Files Size(MB) Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) RW-Amp W-Amp Rd(MB/s) Wr(MB/s) Rn(cnt) Rnp1(cnt) Wnp1(cnt) Wnew(cnt) Comp(sec) Comp(cnt) Avg(sec) Stall(sec) Stall(cnt) Avg(ms) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 14 956 0.9 0.0 0.0 0.0 2.7 2.7 0.0 0.0 0.0 111.6 0 0 0 0 24 40 0.612 75.20 492387 0.15 L1 21 2001 2.0 5.7 2.0 3.7 5.3 1.6 5.4 2.6 71.2 65.7 31 43 55 12 82 2 41.242 43.72 41183 1.06 L2 217 18974 1.9 16.5 2.0 14.4 15.1 0.7 15.6 7.4 70.1 64.3 17 182 185 3 241 16 15.052 0.00 0 0.00 L3 1641 188245 1.8 9.1 1.1 8.0 8.5 0.5 15.4 7.4 61.3 57.2 9 75 76 1 152 9 16.887 0.00 0 0.00 L4 4447 449025 0.4 13.4 4.8 8.6 9.1 0.5 4.7 1.9 77.8 52.7 38 79 100 21 176 38 4.639 0.00 0 0.00 Sum 6340 659201 0.0 44.7 10.0 34.7 40.6 6.0 32.0 15.2 67.7 61.6 95 379 416 37 676 105 6.439 118.91 533570 0.22 Int 0 0 0.0 1.2 0.4 0.8 1.3 0.5 5.2 2.7 59.1 65.6 3 7 9 2 20 10 2.003 0.00 0 0.00 Stalls(secs): 75.197 level0_slowdown, 0.000 level0_numfiles, 0.000 memtable_compaction, 43.717 leveln_slowdown Stalls(count): 492387 level0_slowdown, 0 level0_numfiles, 0 memtable_compaction, 41183 leveln_slowdown ** DB Stats ** Uptime(secs): 202.1 total, 13.5 interval Cumulative writes: 6291456 writes, 6291456 batches, 1.0 writes per batch, 4.90 ingest GB Cumulative WAL: 6291456 writes, 6291456 syncs, 1.00 writes per sync, 4.90 GB written Interval writes: 1048576 writes, 1048576 batches, 1.0 writes per batch, 836.0 ingest MB Interval WAL: 1048576 writes, 1048576 syncs, 1.00 writes per sync, 0.82 MB written Test Plan: ran it Reviewers: sdong, yhchiang, igor Reviewed By: igor Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19917 | 21 July 2014, 19:57:29 UTC |
| 50c2dcb | Feng Zhu | 21 July 2014, 19:01:40 UTC | add options.block_restart_interval in db_bench Summary: Add block_restart_interval in db_bench, default value 16 Test Plan: make Reviewers: sdong Reviewed By: sdong Differential Revision: https://reviews.facebook.net/D20331 | 21 July 2014, 19:01:40 UTC |
| 9c0d84d | Lei Jin | 21 July 2014, 18:03:16 UTC | improve comments for CrateRateLimiter() Summary: Suggested by @dhruba from the other diff, here is the improved comments for parameters of the function Test Plan: none Reviewers: dhruba, sdong, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D19623 | 21 July 2014, 18:03:16 UTC |
| c1a90b0 | Stanislau Hlebik | 21 July 2014, 17:31:33 UTC | Fix db_bench Summary: Adding check for zero size index Test Plan: ./build_tools/regression_build_test.sh Reviewers: yhchiang, sdong Reviewed By: sdong Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20259 | 21 July 2014, 17:31:33 UTC |
| 80a94d0 | Yueh-Hsuan Chiang | 21 July 2014, 16:19:28 UTC | [Java] Fixed the Java compilation error caused by PlainTableFactory API change. Summary: Fixed the Java compilation error caused by PlainTableFactory API change. Test Plan: make rocksdbjava make jdb_bench cd java;./jdb_bench.sh Reviewers: sdong, ljin, ankgup87, swapnilghike, zzbennett, rsumbaly, igor Reviewed By: igor Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20295 | 21 July 2014, 16:19:28 UTC |
| 3769370 | Igor Canadi | 20 July 2014, 16:08:53 UTC | Merge pull request #207 from Chilledheart/develop Fix clang compiler warnings | 20 July 2014, 16:08:53 UTC |
| 54f4e2f | Chilledheart | 20 July 2014, 14:56:40 UTC | Fix clang compiler warnings | 20 July 2014, 14:57:20 UTC |
| f4c49ae | Stanislau Hlebik | 19 July 2014, 00:23:19 UTC | Update HISTORY.md Summary: Updating HISTORY.md to reflect changes in PlainTable index Test Plan: no need Reviewers: sdong Reviewed By: sdong Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20229 | 19 July 2014, 00:23:19 UTC |
| 9d70cce | Stanislau Hlebik | 18 July 2014, 23:58:13 UTC | Adding option to save PlainTable index and bloom filter in SST file. Summary: Adding option to save PlainTable index and bloom filter in SST file. If there is no bloom block and/or index block, PlainTableReader builds new ones. Otherwise PlainTableReader just use these blocks. Test Plan: make all check Reviewers: sdong Reviewed By: sdong Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19527 | 18 July 2014, 23:58:13 UTC |
| 92d73cb | Stanislau Hlebik | 18 July 2014, 07:08:38 UTC | Add PlainTableOptions Summary: Since we have a lot of options for PlainTable, add a struct PlainTableOptions to manage them Test Plan: make all check Reviewers: sdong Reviewed By: sdong Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20175 | 18 July 2014, 07:08:38 UTC |
| 052ddbe | Yueh-Hsuan Chiang | 18 July 2014, 01:01:04 UTC | Add MaxInputLevel() to CompactionPicker Summary: Having if-then branch for different compaction strategies is considered hacky and make CompactionPicker less pluggable. This diff removes two of such if-then branches in version_set.cc by adding MaxInputLevel() to CompactionPicker. // Given the current number of levels, returns the lowest allowed level // for compaction input. virtual int MaxInputLevel(int current_num_levels) const; Test Plan: make db_test export ROCKSDB_TESTS=Compaction ./db_test Reviewers: igor, sdong, ljin Reviewed By: ljin Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19971 | 18 July 2014, 01:01:04 UTC |
| 2f289dc | Yueh-Hsuan Chiang | 18 July 2014, 00:26:12 UTC | Add -Wsign-compare to WARNING_FLAGS in Makefile Summary: Add -Wsign-compare to WARNING_FLAGS in Makefile as not all g++ compiler include -Wsign-compare in -Wall when compiling '.h' file. Test Plan: make -j32 Reviewers: ljin, igor, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D20169 | 18 July 2014, 00:26:12 UTC |
| aac941b | Yueh-Hsuan Chiang | 17 July 2014, 23:37:25 UTC | Fixed a signed and unsigned comparison in Compaction Summary: Fixed a signed and unsigned comparison in Compaction Test Plan: make db_test export ROCKSDB_TESTS=Compaction ./db_test | 17 July 2014, 23:37:25 UTC |
| 0d57e3a | Radheshyam Balasundaram | 17 July 2014, 22:07:05 UTC | Guarding files_ attribute with #ifndef NDEBUG guard in FilePicker class. Summary: Adding guards to files_ attribute of FilePicker class. This attribute is used only in DEBUG mode. This fixes build of static_lib in mac. Test Plan: make static_lib in mac make check all in devserver Reviewers: ljin, igor, sdong Reviewed By: sdong Differential Revision: https://reviews.facebook.net/D20163 | 17 July 2014, 22:07:05 UTC |
| 3178510 | Yueh-Hsuan Chiang | 17 July 2014, 21:36:41 UTC | Allow class Compaction to handle input files from multiple levels. Summary: Allow class Compaction to handle input files from multiple levels. This diff is a subset of https://reviews.facebook.net/D19263 where only db/compaction.cc and db/compaction.h are changed. Test Plan: make db_test export ROCKSDB_TESTS=Compaction ./db_test Reviewers: igor, sdong, ljin Reviewed By: ljin Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19923 | 17 July 2014, 21:36:41 UTC |
| 296e340 | Yueh-Hsuan Chiang | 17 July 2014, 01:12:17 UTC | Add struct CompactionInputFiles to manage compaction input files. Summary: Add struct CompactionInputFiles to manage compaction input files. Test Plan: export ROCKSDB_TESTS=Compact make db_test ./db_test Reviewers: ljin, igor, sdong Reviewed By: sdong Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20061 | 17 July 2014, 01:12:17 UTC |
| bc6b2ab | Feng Zhu | 17 July 2014, 00:32:30 UTC | enable kHashSearch for blocktable in db_bench Summary: add a flag called use_hash_search in db_bench Test Plan: make all check ./db_bench --use_hash_search=1 Reviewers: ljin, haobo, yhchiang, sdong Reviewed By: sdong Subscribers: igor, dhruba Differential Revision: https://reviews.facebook.net/D20067 | 17 July 2014, 00:32:30 UTC |
| 87895c6 | Feng Zhu | 16 July 2014, 23:56:49 UTC | fix bug in LOG for flush memtable Summary: One line change to fix a bug in the LOG when flush memtable Test Plan: NONE Reviewers: sdong Reviewed By: sdong Differential Revision: https://reviews.facebook.net/D20049 | 16 July 2014, 23:56:49 UTC |
| 1c9f190 | Stanislau Hlebik | 16 July 2014, 21:51:43 UTC | Fix db_test Summary: Added deletion of DBIterators in DBIterator's tests Test Plan: make valgrind_check Reviewers: igor, sdong Reviewed By: sdong Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D20043 | 16 July 2014, 21:51:43 UTC |
| 0418e66 | Radheshyam Balasundaram | 16 July 2014, 20:33:02 UTC | Refactoring Version::Get() Summary: Refactoring Version::Get() method to move file picker logic to a separate class. Test Plan: make check all Reviewers: igor, sdong, ljin Reviewed By: ljin Subscribers: leveldb Differential Revision: https://reviews.facebook.net/D19713 | 16 July 2014, 20:33:02 UTC |
| c11d604 | Feng Zhu | 16 July 2014, 18:21:30 UTC | store file_indexer info in sequential memory Summary: use arena to allocate space for next_level_index_ and level_rb_ Thus increasing data locality and make Version::Get faster. Benchmark detail Base version: commit d2a727c182338514af955bbcb1d92db4af83b41c command used: ./db_bench --db=/mnt/db/rocksdb --num_levels=6 --key_size=20 --prefix_size=20 --keys_per_prefix=0 --value_size=100 --block_size=4096 --cache_size=17179869184 --cache_numshardbits=6 --compression_type=none --compression_ratio=1 --min_level_to_compress=-1 --disable_seek_compaction=1 --hard_rate_limit=2 --write_buffer_size=134217728 --max_write_buffer_number=2 --level0_file_num_compaction_trigger=8 --target_file_size_base=2097152 --max_bytes_for_level_base=1073741824 --disable_wal=0 --sync=0 --disable_data_sync=1 --verify_checksum=1 --delete_obsolete_files_period_micros=314572800 --max_grandparent_overlap_factor=10 --max_background_compactions=4 --max_background_flushes=0 --level0_slowdown_writes_trigger=16 --level0_stop_writes_trigger=24 --statistics=0 --stats_per_interval=0 --stats_interval=1048576 --histogram=0 --use_plain_table=1 --open_files=-1 --mmap_read=1 --mmap_write=0 --memtablerep=prefix_hash --bloom_bits=10 --bloom_locality=1 --perf_level=0 --benchmarks=fillseq, readrandom,readrandom,readrandom --use_existing_db=0 --num=52428800 --threads=1 Result: cpu running percentage: Version::Get, improved from 7.98% to 7.42% FileIndexer::GetNextLevelIndex, improved from 1.18% to 0.68%. Test Plan: make all check Reviewers: ljin, haobo, yhchiang, sdong Reviewed By: sdong Subscribers: dhruba, igor Differential Revision: https://reviews.facebook.net/D19845 | 16 July 2014, 18:21:30 UTC |
| 1614284 | Igor Canadi | 16 July 2014, 13:45:49 UTC | Fix compressed cache | 16 July 2014, 13:45:49 UTC |
