| 1213413 | Yueh-Hsuan Chiang | 14 March 2015, 15:21:53 UTC | Fixed the unit-test issue in PreShutdownCompactionMiddle Summary: Fixed the unit-test issue in PreShutdownCompactionMiddle Test Plan: export ROCKSDB_TESTS=PreShutdownCompactionMiddle Reviewers: rven, sdong, igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D35061 | 14 March 2015, 15:25:27 UTC |
| fd1b3f3 | Yueh-Hsuan Chiang | 14 March 2015, 15:00:06 UTC | Fix the issue in PreShutdownMultipleCompaction Summary: Fix the issue in PreShutdownMultipleCompaction Test Plan: export ROCKSDB_TESTS=PreShutdownMultipleCompaction ./db_test Reviewers: rven, sdong, igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D35055 | 14 March 2015, 15:03:02 UTC |
| 56c4a9c | Yueh-Hsuan Chiang | 14 March 2015, 01:09:01 UTC | Fix compile warning in thread_status_util.h on Mac Summary: Fix compile warning in thread_status_util.h on Mac Test Plan: make dbg -j32 | 14 March 2015, 01:09:01 UTC |
| 417367c | Igor Canadi | 13 March 2015, 23:41:00 UTC | Fix SIGSEGV when not using cache | 13 March 2015, 23:41:00 UTC |
| e25ff03 | Venkatesh Radhakrishnan | 13 March 2015, 21:51:40 UTC | Prevent slowdowns and stalls in PreShutdown tests Summary: The preshutdown tests check for stopped compactions/flushes. Removing stalls on the write path. Test Plan: DBTests.PreShutdown* Reviewers: yhchiang, sdong, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D35037 | 13 March 2015, 21:51:40 UTC |
| f690712 | Igor Canadi | 13 March 2015, 21:45:15 UTC | Speed up db_bench shutdown Summary: See t6489044 Test Plan: compiles Reviewers: MarkCallaghan Reviewed By: MarkCallaghan Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D34977 | 13 March 2015, 21:45:15 UTC |
| c1b3cde | Yueh-Hsuan Chiang | 13 March 2015, 20:11:08 UTC | Improve the robustness of ThreadStatusSingleCompaction Summary: Improve the robustness of ThreadStatusSingleCompaction by ensuring the number of files flushed in the test. Test Plan: export ROCKSDB_TESTS=ThreadStatus ./db_test Reviewers: sdong, igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D35019 | 13 March 2015, 20:16:53 UTC |
| 8c12426 | Yueh-Hsuan Chiang | 13 March 2015, 18:59:00 UTC | Fix the deadlock issue in ThreadStatusSingleCompaction. Summary: Fix the deadlock issue in ThreadStatusSingleCompaction. In the previous version of ThreadStatusSingleCompaction, the compaction thread will wait for a SYNC_POINT while its db_mutex is held. However, if the test hasn't finished its Put cycle while a compaction is running, a deadlock will happen in the test. Test Plan: export ROCKSDB_TESTS=ThreadStatus ./db_test Reviewers: sdong, igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D35001 | 13 March 2015, 19:53:00 UTC |
| b16ead5 | sdong | 13 March 2015, 18:18:35 UTC | DBTest.DynamicLevelCompressionPerLevel should not run without snappy support Summary: The test depends on snappy to be used. Skip the test if it is not supported. Test Plan: Run the test Reviewers: meyering, igor Reviewed By: igor Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D34995 | 13 March 2015, 18:26:17 UTC |
| a5e60ba | Yueh-Hsuan Chiang | 13 March 2015, 18:18:42 UTC | Fix a typo / test failure in ThreadStatusSingleCompaction Summary: Fix a typo / test failure in ThreadStatusSingleCompaction Test Plan: export ROCKSDB_TESTS=ThreadStatus ./db_test | 13 March 2015, 18:20:17 UTC |
| cb2c918 | Igor Canadi | 13 March 2015, 18:08:50 UTC | Don't run some tests is snappy is not present Summary: Currently, we have `ifdef SNAPPY` around bunch of db_test code. Some tests that don't even use compression are also blocked when running system doesn't have snappy. This also causes hard-to-catch bugs, like D34983. We should dynamically figure out if compression is supported or not. Test Plan: compiles Reviewers: sdong, meyering Reviewed By: meyering Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D34989 | 13 March 2015, 18:08:50 UTC |
| c594b0e | Yueh-Hsuan Chiang | 13 March 2015, 17:45:40 UTC | Allow GetThreadList() to report operation stage. Summary: Allow GetThreadList() to report operation stage. Test Plan: ./thread_list_test ./db_bench --benchmarks=fillrandom --num=100000 --threads=40 \ --max_background_compactions=10 --max_background_flushes=3 \ --thread_status_per_interval=1000 --key_size=16 --value_size=1000 \ --num_column_families=10 export ROCKSDB_TESTS=ThreadStatus ./db_test Sample output ThreadID ThreadType cfName Operation OP_StartTime ElapsedTime Stage State 140116265861184 Low Pri 140116270055488 Low Pri 140116274249792 High Pri column_family_name_000005 Flush 2015/03/10-14:58:11 0 us FlushJob::WriteLevel0Table 140116400078912 Low Pri column_family_name_000004 Compaction 2015/03/10-14:58:11 0 us CompactionJob::FinishCompactionOutputFile 140116358135872 Low Pri column_family_name_000006 Compaction 2015/03/10-14:58:10 1 us CompactionJob::FinishCompactionOutputFile 140116341358656 Low Pri 140116295221312 High Pri default Flush 2015/03/10-14:58:11 0 us FlushJob::WriteLevel0Table 140116324581440 Low Pri column_family_name_000009 Compaction 2015/03/10-14:58:11 0 us CompactionJob::ProcessKeyValueCompaction 140116278444096 Low Pri 140116299415616 Low Pri column_family_name_000008 Compaction 2015/03/10-14:58:11 0 us CompactionJob::FinishCompactionOutputFile 140116291027008 High Pri column_family_name_000001 Flush 2015/03/10-14:58:11 0 us FlushJob::WriteLevel0Table 140116286832704 Low Pri column_family_name_000002 Compaction 2015/03/10-14:58:11 0 us CompactionJob::FinishCompactionOutputFile 140116282638400 Low Pri Reviewers: rven, igor, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D34683 | 13 March 2015, 17:45:40 UTC |
| 2623b2c | Igor Canadi | 13 March 2015, 17:29:32 UTC | Include chrono | 13 March 2015, 17:29:32 UTC |
| 52d8347 | Igor Canadi | 13 March 2015, 17:15:54 UTC | EventLogger Summary: Here's my proposal for making our LOGs easier to read by machines. The idea is to dump all events as JSON objects. JSON is easy to read by humans, but more importantly, it's easy to read by machines. That way, we can parse this, load into SQLite/mongo and then query or visualize. I started with table_create and table_delete events, but if everybody agrees, I'll continue by adding more events (flush/compaction/etc etc) Test Plan: Ran db_bench. Observed: 2015/01/15-14:13:25.788019 1105ef000 EVENT_LOG_v1 {"time_micros": 1421360005788015, "event": "table_file_creation", "file_number": 12, "file_size": 1909699} 2015/01/15-14:13:25.956500 110740000 EVENT_LOG_v1 {"time_micros": 1421360005956498, "event": "table_file_deletion", "file_number": 12} Reviewers: yhchiang, rven, dhruba, MarkCallaghan, lgalanis, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D31647 | 13 March 2015, 17:15:54 UTC |
| 756532d | Adam Retter | 13 March 2015, 12:45:04 UTC | Merge pull request #524 from fyrz/RocksJava-Test-Fix [RocksJava] Fix JTest issues with enabled assertions. Closes https://github.com/facebook/rocksdb/issues/523 | 13 March 2015, 12:45:04 UTC |
| 47a2b3a | Adam Retter | 13 March 2015, 12:39:36 UTC | Merge pull request #534 from fyrz/RocksJava-Fix-BrokenJacocoReport [RocksJava] Fix broken jacoco report | 13 March 2015, 12:39:36 UTC |
| 9d22a1f | Islam AbdelRahman | 13 March 2015, 03:53:18 UTC | Allow negative Wnew Summary: we are using uint64_t for Wnew this is not correct since this value can be negative https://github.com/facebook/rocksdb/issues/535 Test Plan: run db_bench and check what happens when Wnew is -ve Reviewers: sdong, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D34935 | 13 March 2015, 03:53:18 UTC |
| 2c84303 | Yueh-Hsuan Chiang | 12 March 2015, 21:12:55 UTC | Merge pull request #528 from fyrz/RocksJava-NativeLibraryLoader [RocksJava] Fix NativeLibraryLoader | 12 March 2015, 21:12:55 UTC |
| 2dc636f | fyrz | 12 March 2015, 20:27:12 UTC | [RocksJava] Fix broken jacoco report With the last folder layout change a bug was introduced which prevents Jacoco from working correctly. | 12 March 2015, 20:28:58 UTC |
| f210b0f | fyrz | 09 March 2015, 18:07:08 UTC | [RocksJava] Fix NativeLibraryLoader - Resolve problem while using a temporary data folder - Fix test | 12 March 2015, 19:13:09 UTC |
| 3ebebfc | Sameet Agarwal | 12 March 2015, 01:00:57 UTC | Prevent xxhash symbols from polluting global namespace Summary: The functions and global symbols in xxhash.h and xxhash.cc were not in any namespace. This caused issues when rocksdb library was being used along with other uses of libraries with the same name Test Plan: unit tests Reviewers: CC: Task ID: # Blame Rev: | 12 March 2015, 19:07:10 UTC |
| 5399614 | stash93 | 12 March 2015, 18:13:53 UTC | Removing unnecessary kInlineSize Summary: Remove unnecessary rocksdb::kInlineSize, since it's not used and there is rocksdb::Arena::kInlineSize. Test Plan: make all check Reviewers: igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D34905 | 12 March 2015, 18:13:53 UTC |
| b411d06 | Venkatesh Radhakrishnan | 12 March 2015, 17:49:06 UTC | Prevent stalls in preshutdown tests Summary: The tests using sync_point for intent to shutdown stop compaction and this results in stalls if too many rows are written. We now limit the number of rows written to prevent stalls, since the focus of the test is to cancel background work, which is being correctly tested. This fixes a Jenkins issue. Test Plan: DBTest.PreShutdown* Reviewers: sdong, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D34893 | 12 March 2015, 17:49:06 UTC |
| 1d43bc4 | Islam AbdelRahman | 12 March 2015, 00:57:16 UTC | Fixing segmentation fault in db_bench Summary: Fixing segmentation fault when running db_bench This seg fault happens because num_created is used without being initialized Test Plan: running db_bench using these arguments bpl=10485760;overlap=10;mcz=2;del=300000000;levels=6;ctrig=4; delay=8; stop=12; wbn=3; mbc=20; mb=67108864;wbs=134217728; dds=0; sync=0; r=1000000; t=1; vs=800; bs=65536; cs=1048576; of=500000; si=1000000; ./db_bench --benchmarks=overwrite --disable_seek_compaction=1 --mmap_read=0 --statistics=1 --histogram=1 --num=$r --threads=$t --value_size=$vs --block_size=$bs --cache_size=$cs --bloom_bits=10 --cache_numshardbits=4 --open_files=$of --verify_checksum=1 --db=/home/tec/koko/ --sync=$sync --disable_wal=1 --compression_type=zlib --stats_interval=$si --compression_ratio=0.5 --disable_data_sync=$dds --write_buffer_size=$wbs --target_file_size_base=$mb --max_write_buffer_number=$wbn --max_background_compactions=$mbc --level0_file_num_compaction_trigger=$ctrig --level0_slowdown_writes_trigger=$delay --level0_stop_writes_trigger=$stop --num_levels=$levels --delete_obsolete_files_period_micros=$del --min_level_to_compress=$mcz --max_grandparent_overlap_factor=$overlap --stats_per_interval=1 --max_bytes_for_level_base=$bpl --use_existing_db=1 Reviewers: sdong, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D34881 | 12 March 2015, 00:57:16 UTC |
| e9de8b6 | sdong | 10 March 2015, 19:35:15 UTC | Change the way options.compression_per_level is used when options.level_compaction_dynamic_level_bytes=true Summary: Change the way options.compression_per_level is used when options.level_compaction_dynamic_level_bytes=true so that options.compression_per_level[1] determines compression for the level L0 is merged to, options.compression_per_level[2] to the level after that, etc. Test Plan: run all tests Reviewers: rven, yhchiang, kradhakrishnan, igor Reviewed By: igor Subscribers: yoshinorim, leveldb, dhruba Differential Revision: https://reviews.facebook.net/D34431 | 11 March 2015, 20:14:52 UTC |
| 2b785d7 | Yueh-Hsuan Chiang | 11 March 2015, 20:06:59 UTC | Fixed a bug where CompactFiles won't delete obsolete files until flush. Summary: Fixed a bug where CompactFiles won't delete obsolete files until flush. Test Plan: ./compact_files_test export ROCKSDB_TESTS=CompactFiles ./db_test Reviewers: rven, sdong, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D34671 | 11 March 2015, 20:06:59 UTC |
| 6f55798 | Yueh-Hsuan Chiang | 11 March 2015, 20:02:46 UTC | Fixed a compile error in db_bench in mac. Summary: Fixed a compile error in db_bench in mac. Test Plan: make db_bench | 11 March 2015, 20:02:46 UTC |
| 05d92ef | Venkatesh Radhakrishnan | 11 March 2015, 19:39:53 UTC | Add convenience.cc to src.mk Summary: The build process now requires new source files to be added to src.mk. Adding convenience.cc to src.mk Test Plan: Build rocksdb Reviewers: igor, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D34815 | 11 March 2015, 19:39:53 UTC |
| 2884b10 | sdong | 10 March 2015, 23:32:12 UTC | db_bench: Better way to randomize repeated read keys in -read_random_exp_range Summary: Use a better way to map from a key with locality to a random location. Now with the same -read_random_exp_range setting, hit rate drops, which it is expected. Test Plan: ./db_bench --benchmarks=readrandom -statistics -use_existing_db -cache_size=5000000 --read_random_exp_range=<multiple_values> Reviewers: MarkCallaghan, kradhakrishnan, igor Reviewed By: igor Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D34761 | 11 March 2015, 18:46:14 UTC |
| 284be57 | Venkatesh Radhakrishnan | 11 March 2015, 17:31:02 UTC | Provide a mechanism to inform Rocksdb that it is shutting down Summary: Provide an API which enables users to infor Rocksdb that it is shutting down. Test Plan: db_test Reviewers: sdong, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D34617 | 11 March 2015, 17:31:02 UTC |
| 2ddf53b | Igor Canadi | 11 March 2015, 00:53:22 UTC | Get OptimizeFilterForHits work on Mac Summary: Got it working by some voodoo programming Test Plan: works! Reviewers: sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D34611 | 11 March 2015, 00:53:22 UTC |
| 480b284 | Venkatesh Radhakrishnan | 11 March 2015, 00:24:33 UTC | Fix make_new_version.sh Summary: One of the commands in the script make_new_version.sh was incorrect. Fixed it. Test Plan: Try the script out to verify that it works. Reviewers: igor, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D34791 | 11 March 2015, 00:24:33 UTC |
| 89597bb | Yueh-Hsuan Chiang | 10 March 2015, 21:51:28 UTC | Allow GetThreadList() to report the start time of the current operation. Summary: Allow GetThreadList() to report the start time of the current operation. Test Plan: ./db_bench --benchmarks=fillrandom --num=100000 --threads=40 \ --max_background_compactions=10 --max_background_flushes=3 \ --thread_status_per_interval=1000 --key_size=16 --value_size=1000 \ --num_column_families=10 Sample output: ThreadID ThreadType cfName Operation OP_StartTime State 140338840797248 High Pri column_family_name_000003 Flush 2015/03/09-17:49:59 140338844991552 High Pri column_family_name_000004 Flush 2015/03/09-17:49:59 140338849185856 Low Pri 140338983403584 Low Pri 140339008569408 Low Pri 140338861768768 Low Pri 140338924683328 Low Pri 140338899517504 Low Pri 140338853380160 Low Pri 140338882740288 Low Pri 140338865963072 High Pri column_family_name_000006 Flush 2015/03/09-17:49:59 140338954043456 Low Pri 140338857574464 Low Pri Reviewers: igor, rven, sdong Reviewed By: sdong Subscribers: lgalanis, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D34689 | 10 March 2015, 21:51:28 UTC |
| 37921b4 | sdong | 07 March 2015, 00:38:50 UTC | db_bench: Add Option -read_random_exp_range to allow read skewness. Summary: Introduce parameter -read_random_exp_range in db_bench to provide some key skewness in readrandom and multireadrandom benchmarks. It will helpful to cover block cache better. Test Plan: Run benchmarks with this new parameter. I can clearly see block cache hit rate change while I increase this value (DB size is about 66MB): ./db_bench --benchmarks=readrandom -statistics -use_existing_db -cache_size=5000000 --read_random_exp_range=0.0 rocksdb.block.cache.data.miss COUNT : 958418 rocksdb.block.cache.data.hit COUNT : 41582 ./db_bench --benchmarks=readrandom -statistics -use_existing_db -cache_size=5000000 --read_random_exp_range=5.0 rocksdb.block.cache.data.miss COUNT : 819518 rocksdb.block.cache.data.hit COUNT : 180482 ./db_bench --benchmarks=readrandom -statistics -use_existing_db -cache_size=5000000 --read_random_exp_range=10.0 rocksdb.block.cache.data.miss COUNT : 450479 rocksdb.block.cache.data.hit COUNT : 549521 ./db_bench --benchmarks=readrandom -statistics -use_existing_db -cache_size=5000000 --read_random_exp_range=20.0 rocksdb.block.cache.data.miss COUNT : 223192 rocksdb.block.cache.data.hit COUNT : 776808 Reviewers: MarkCallaghan, kradhakrishnan, yhchiang, rven, igor Reviewed By: igor Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D34629 | 09 March 2015, 18:34:52 UTC |
| 485ac0d | Igor Canadi | 06 March 2015, 22:21:15 UTC | Add rate_limiter to string options Summary: I want to be able to set this through mongo config. Test Plan: added unit test Reviewers: sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D34599 | 06 March 2015, 22:21:15 UTC |
| e126e0d | Leonidas Galanis | 06 March 2015, 22:12:53 UTC | Single threaded tests -> sync=0 Multi threaded tests -> sync=1 by default unless DB_BENCH_NO_SYNC is defined Summary: Single threaded tests -> sync=0 Multi threaded tests -> sync=1 by default unless DB_BENCH_NO_SYNC is defined. Also added updaterandom and mergerandom with putOperator. I am waiting for some results from udb on this. Test Plan: DB_BENCH_NO_SYNC=1 WAL_DIR=/tmp OUTPUT_DIR=/tmp/b DB_DIR=/tmp ./tools/benchmark.sh debug,bulkload,fillseq,overwrite,filluniquerandom,readrandom,readwhilewriting,updaterandom,mergerandom WAL_DIR=/tmp OUTPUT_DIR=/tmp/b DB_DIR=/tmp ./tools/benchmark.sh debug,bulkload,fillseq,overwrite,filluniquerandom,readrandom,readwhilewriting,updaterandom,mergerandom Verify sync settings Reviewers: sdong, MarkCallaghan, igor, rven Reviewed By: igor, rven Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D34185 | 06 March 2015, 22:12:53 UTC |
| dc4532c | Yueh-Hsuan Chiang | 06 March 2015, 19:22:06 UTC | Add --thread_status_per_interval to db_bench Summary: Add --thread_status_per_interval to db_bench, which allows db_bench to optionally enable print the current thread status periodically. Test Plan: ./db_bench --benchmarks=fillrandom --num=100000 --threads=40 --max_background_compactions=10 --max_background_flushes=3 --thread_status_per_interval=1000 --key_size=16 --value_size=1000 --num_column_families=10 Sample output: ThreadID ThreadType dbName cfName Operation State 140281571770432 Low Pri 140281575964736 High Pri /tmp/rocksdbtest-5297/dbbench column_family_name_000001 Flush 140281710182464 Low Pri /tmp/rocksdbtest-5297/dbbench column_family_name_000008 Compaction 140281638879296 Low Pri /tmp/rocksdbtest-5297/dbbench column_family_name_000007 Compaction 140281592741952 Low Pri 140281580159040 High Pri /tmp/rocksdbtest-5297/dbbench column_family_name_000002 Flush 140281676628032 Low Pri /tmp/rocksdbtest-5297/dbbench column_family_name_000006 Compaction 140281584353344 Low Pri 140281622102080 Low Pri /tmp/rocksdbtest-5297/dbbench column_family_name_000009 Compaction 140281605324864 Low Pri /tmp/rocksdbtest-5297/dbbench column_family_name_000004 Compaction 140281601130560 High Pri /tmp/rocksdbtest-5297/dbbench default Flush 140281596936256 Low Pri 140281588547648 Low Pri Reviewers: igor, rven, sdong Reviewed By: rven Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D34515 | 06 March 2015, 19:22:06 UTC |
| 34c75e9 | Jim Meyering | 06 March 2015, 19:03:00 UTC | fix-up patch: avoid new link error Summary: * src.mk (LIB_SOURCES): Add this file: utilities/document/json_document_builder.cc to avoid link errors. Blame Rev: D33849 Test Plan: run "make" Reviewers: ljin, igor.sugak, rven, sdong, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D34593 | 06 March 2015, 19:15:18 UTC |
| ebc647d | Jim Meyering | 21 February 2015, 01:42:16 UTC | build: fix missing dependency problems Summary: Any time one would modify a dependent of any *test*.cc file, "make" would fail to rebuild the affected test binaries, e.g., db_test. That was due to the fact that we deliberately excluded those test-related files from the definition of SOURCES and only $(SOURCES) was used to create the automatically-generated .d dependency files. The fix is to generate a .d file for every source file. * src.mk: New file. Defines LIB_SOURCES, MOCK_SOURCES and TEST_BENCH_SOURCES. * Makefile: Include src.mk. Reflect s/SOURCES/LIB_SOURCES/ renaming. * build_tools/build_detect_platform: Remove the code that was used to generate SOURCES= and MOCK_SOURCES= definitions in make_config.mk. Those lists of files are now hard-coded in src.mk. Hard-coding this list of sources is desirable, because without that, one risks including stray .cc files in a build. Not reproducible. Test Plan: Touch a file used by db_test's dependent .o files and ensure that they are all recompiled. Before, none would be: $ touch db/db_impl.h && make db_test CC db/db_test.o CC db/column_family.o CC db/db_filesnapshot.o CC db/db_impl.o CC db/db_impl_debug.o CC db/db_impl_readonly.o CC db/forward_iterator.o CC db/internal_stats.o CC db/managed_iterator.o CC db/repair.o CC db/write_batch.o CC utilities/compacted_db/compacted_db_impl.o CC utilities/ttl/db_ttl_impl.o CC util/ldb_cmd.o CC util/ldb_tool.o CC util/sst_dump_tool.o CC util/xfunc.o CCLD db_test Reviewers: ljin, igor.sugak, igor, rven, sdong Reviewed By: sdong Subscribers: yhchiang, adamretter, fyrz, dhruba Differential Revision: https://reviews.facebook.net/D33849 | 06 March 2015, 18:55:11 UTC |
| 492f6d2 | Tian Xia | 06 March 2015, 05:34:23 UTC | Fix a segfault in fbson under Mac OS X compiler Summary: The problem appears to be caused by a bug in Mac OS X compiler (http://llvm.org/bugs/show_bug.cgi?id=15337). We need explicitly construct the base object std::ostream(std::streambuf*) with nullptr. Otherwise, ostream will try to delete the underlying streambuf* which apparently is undefined in the Mac OS X compiler. https://github.com/facebook/rocksdb/issues/525 Test Plan: unit test in fbson make all check document_db_test (on mac) Reviewers: IslamAbdelRahman, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D34587 | 06 March 2015, 18:12:50 UTC |
| 0d13bbe | fyrz | 04 March 2015, 21:43:39 UTC | RocksJava] Fix ColumnFamily tests Summary: Cleaned up some tests regarding disposal order and tests which were failing when C++ assertions were enabled. Test Plan: - Enable C++ Assertions (remove e.g. -DNDebug in rocksdbjava target) - make rocksdbjava jtest Reviewers: adamretter, ankgup87, yhchiang Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D34569 | 05 March 2015, 20:13:46 UTC |
| 6753380 | fyrz | 04 March 2015, 21:30:54 UTC | [RocksJava] Fixed CompactionTest | 04 March 2015, 21:45:18 UTC |
| 1b7b997 | fyrz | 04 March 2015, 21:30:40 UTC | [RocksJava] Remove MaxValue from Statistics | 04 March 2015, 21:45:17 UTC |
| f862b38 | fyrz | 04 March 2015, 21:15:59 UTC | [RocksJava] Fix cleanup in tests | 04 March 2015, 21:45:16 UTC |
| a01b592 | fyrz | 02 March 2015, 20:58:11 UTC | [RocksJava] DefaultColumnFamily Memory Fix In the current implementation DefaultColumnFamily will not disown the native handle. As the database handles the lease on the native handle this should be the case. | 04 March 2015, 21:45:16 UTC |
| 22c73d1 | fyrz | 02 March 2015, 20:48:29 UTC | [RocksJava] Fix ColumnFamily tests Cleanup column family memory cleanup | 04 March 2015, 21:45:15 UTC |
| 694988b | Yueh-Hsuan Chiang | 03 March 2015, 20:48:12 UTC | Fix a bug in stall time counter. Improve its output format. Summary: Fix a bug in stall time counter. Improve its output format. Test Plan: export ROCKSDB_TESTS=Timeout ./db_test ./db_bench --benchmarks=fillrandom --stats_interval=10000 --statistics=true --stats_per_interval=1 --num=1000000 --threads=4 --level0_stop_writes_trigger=3 --level0_slowdown_writes_trigger=2 sample output: Uptime(secs): 35.8 total, 0.0 interval Cumulative writes: 359590 writes, 359589 keys, 183047 batches, 2.0 writes per batch, 0.04 GB user ingest, stall seconds: 1786.008 ms Cumulative WAL: 359591 writes, 183046 syncs, 1.96 writes per sync, 0.04 GB written Interval writes: 253 writes, 253 keys, 128 batches, 2.0 writes per batch, 0.0 MB user ingest, stall time: 0 us Interval WAL: 253 writes, 128 syncs, 1.96 writes per sync, 0.00 MB written Reviewers: MarkCallaghan, igor, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D34275 | 03 March 2015, 20:48:12 UTC |
| b8d23cd | Igor Canadi | 03 March 2015, 19:29:31 UTC | Revert chrono use Summary: For some reason, libstdc++ implements steady_clock::now() using syscall instead of VDSO optimized clock_gettime() when using glibc 2.16 and earlier. This leads to significant performance degradation for users with older glibcs. See bug reported here: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=59177 We observed this behavior when testing mongo on AWS hosts. Facebook hosts are unaffected since we use glibc2.17 and 2.20. Revert "Fix timing" This reverts commit 965d9d50b8cbb413de5e834b5b83ddbb682d0f1d. Revert "Use chrono for timing" This reverts commit 001ce64dc7659c65569ffb1c440e26cd23db3c94. Test Plan: make check Reviewers: MarkCallaghan, yhchiang, rven, meyering, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D34371 | 03 March 2015, 19:29:31 UTC |
| db03739 | Igor Canadi | 05 February 2015, 19:44:17 UTC | options.level_compaction_dynamic_level_bytes to allow RocksDB to pick size bases of levels dynamically. Summary: When having fixed max_bytes_for_level_base, the ratio of size of largest level and the second one can range from 0 to the multiplier. This makes LSM tree frequently irregular and unpredictable. It can also cause poor space amplification in some cases. In this improvement (proposed by Igor Kabiljo), we introduce a parameter option.level_compaction_use_dynamic_max_bytes. When turning it on, RocksDB is free to pick a level base in the range of (options.max_bytes_for_level_base/options.max_bytes_for_level_multiplier, options.max_bytes_for_level_base] so that real level ratios are close to options.max_bytes_for_level_multiplier. Test Plan: New unit tests and pass tests suites including valgrind. Reviewers: MarkCallaghan, rven, yhchiang, igor, ikabiljo Reviewed By: ikabiljo Subscribers: yoshinorim, ikabiljo, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D31437 | 03 March 2015, 06:40:41 UTC |
| f29b33c | krad | 03 March 2015, 01:07:03 UTC | Add functionality to pre-fetch blocks specified by a key range to BlockBasedTable implementation. Summary: Pre-fetching is a common operation performed by data stores for disk/flash based systems as part of database startup. This is part of task 5197184. Test Plan: Run the newly added unit test Reviewers: rven, igor, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D33933 | 03 March 2015, 01:07:03 UTC |
| c4bd03a | Mark Callaghan | 02 March 2015, 17:35:50 UTC | Fix typo in log message Summary: fix typo Task ID: # Blame Rev: Test Plan: Revert Plan: Database Impact: Memcache Impact: Other Notes: EImportant: - begin *PUBLIC* platform impact section - Bugzilla: # - end platform impact - Reviewers: igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D34251 | 02 March 2015, 17:35:50 UTC |
| 3cf7f35 | Igor Canadi | 28 February 2015, 01:06:06 UTC | Instrument memtable seeks Summary: As title Test Plan: compiles Reviewers: sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D34191 | 28 February 2015, 01:06:06 UTC |
| 216a9e1 | Igor Canadi | 27 February 2015, 22:21:16 UTC | Fix compile Summary: I was pretty sure I compiled this before landing, sorry :/ Test Plan: compiles Reviewers: sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D34173 | 27 February 2015, 22:21:16 UTC |
| b9ff6b0 | Igor Canadi | 27 February 2015, 22:03:56 UTC | Fix a bug in ReadOnlyBackupEngine Summary: This diff fixes a bug introduced by D28521. Read-only backup engine can delete a backup that is later than the latest -- we never check the condition. I also added a bunch of logging that will help with debugging cases like this in the future. See more discussion at t6218248. Test Plan: Added a unit test that was failing before the change. Also, see new LOG file contents: https://phabricator.fb.com/P19738984 Reviewers: benj, sanketh, sumeet, yhchiang, rven, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D33897 | 27 February 2015, 22:03:56 UTC |
| afa8156 | Islam AbdelRahman | 27 February 2015, 18:27:14 UTC | adding stdlib to fbson Summary: adding stdlib to fbson to fix FreeBSD compilation Test Plan: https://github.com/facebook/mysql-5.6/commit/4cfccf14f4abc5a142b25c54ce1d0f4dafe95f9c Reviewers: igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D34143 | 27 February 2015, 18:27:14 UTC |
| f9c14a4 | Igor Canadi | 27 February 2015, 17:31:29 UTC | Fix compile on Mac | 27 February 2015, 17:31:29 UTC |
| a9f0e2b | Igor Canadi | 27 February 2015, 00:56:39 UTC | Fix compile Summary: as title, we have unused variables. this is a short-term solution Test Plan: compiles Reviewers: IslamAbdelRahman, sdong, rven Reviewed By: rven Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D34125 | 27 February 2015, 00:56:39 UTC |
| e7c434c | Sameet Agarwal | 17 February 2015, 16:03:45 UTC | Add columnfamily option optimize_filters_for_hits to optimize for key hits only Summary: Summary: Added a new option to ColumnFamllyOptions - optimize_filters_for_hits. This option can be used in the case where most accesses to the store are key hits and we dont need to optimize performance for key misses. This is useful when you have a very large database and most of your lookups succeed. The option allows the store to not store and use filters in the last level (the largest level which contains data). These filters can take a large amount of space for large databases (in memory and on-disk). For the last level, these filters are only useful for key misses and not for key hits. If we are not optimizing for key misses, we can choose to not store these filters for that level. This option is only provided for BlockBasedTable. We skip the filters when we are compacting Test Plan: 1. Modified db_test toalso run tests with an additonal option (skip_filters_on_last_level) 2. Added another unit test to db_test which specifically tests that filters are being skipped Reviewers: rven, igor, sdong Reviewed By: sdong Subscribers: lgalanis, yoshinorim, MarkCallaghan, rven, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D33717 | 27 February 2015, 00:25:56 UTC |
| ba9d173 | Islam AbdelRahman | 26 February 2015, 23:19:17 UTC | RocksDB on FreeBSD support Summary: This patch will update the Makefile and source code so that we can build RocksDB successfully on FreeBSD 10 and 11 (64-bit and 32-bit) I have also encountered some problems when running tests on FreeBSD, I will try to fix them individually in different diffs Notes: - FreeBSD uses clang as it's default compiler (http://lists.freebsd.org/pipermail/freebsd-current/2012-September/036480.html) - GNU C++ compiler have C++ 11 problems on FreeBSD (https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=193528) - make is not gmake on FreeBSD (http://www.khmere.com/freebsd_book/html/ch01.html) Test Plan: Using VMWare Fusion Create 4 VM machines (FreeBSD 11 64-bit, FreeBSD 11 32-bit, FreeBSD 10 64-bit, FreeBSD 10 32-bit) - pkg install git gmake gflags archivers/snappy - git clone https://github.com/facebook/rocksdb.git - apply this patch - setenv CXX c++ - setenv CPATH /usr/local/include/ - setenv LIBRARY_PATH /usr/local/lib/ - gmake db_bench - make sure compilation is successful and db_bench is running - gmake all - make sure compilation is successful Reviewers: sdong, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D33891 | 26 February 2015, 23:19:17 UTC |
| 4ba119d | Igor Sugak | 26 February 2015, 23:12:08 UTC | rocksdb: Update path to dev clang in fbcode Summary: Current clang path in fbcode points to clang 3.5. This diff updates clang path to clang 3.7. Test Plan: Make sure clang 3.7 is in use and all tests are passing. ```lang=bash % USE_CLANG make check ``` Reviewers: meyering, sdong, rven, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D34119 | 26 February 2015, 23:12:08 UTC |
| 8984e5f | Venkatesh Radhakrishnan | 26 February 2015, 23:11:50 UTC | Fix race in sync point. Summary: The LoadDependency function does not take a lock when it runs and it could be modifying data structures while other threads are accessing it. Test Plan: Run TSAN. Reviewers: igor, sdong Reviewed By: igor, sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D34095 | 26 February 2015, 23:11:50 UTC |
| 03b432d | Igor Sugak | 26 February 2015, 22:19:51 UTC | rocksdb: Fix uninitialized use error Summary: When using latest clang (3.6 or 3.7/trunck) rocksdb is failing with many errors. Some errors are uninitialized use errors. ``` ... CC db/log_test.o util/ldb_cmd.cc:394:16: error: base class 'rocksdb::LDBCommand' is uninitialized when used here to access 'rocksdb::LDBCommand::BuildCmdLineOptions' [-Werror,-Wuninitialized] BuildCmdLineOptions({ARG_FROM, ARG_TO, ARG_HEX, ARG_KEY_HEX, ^ ... ``` ```lang=c++ CompactorCommand::CompactorCommand(const vector<string>& params, const map<string, string>& options, const vector<string>& flags) : LDBCommand(options, flags, false, BuildCmdLineOptions({ARG_FROM, ARG_TO, ARG_HEX, ARG_KEY_HEX, ARG_VALUE_HEX, ARG_TTL})), null_from_(true), null_to_(true) { . . . } ``` For the fourth parameter of the base constructor (`LDBCommand`) we call `BuildCmdLineOptions`, which is a private non-static method of `LDBCommand` base class. This diff adds missing `static` keyword for `LDBCommand::BuildCmdLineOptions` method. Test Plan: Build with trunk clang and make sure all tests are passing. ```lang=bash % # Have trunk clang present in path. % ROCKSDB_NO_FBCODE=1 CC=clang CXX=clang++ make check `` Reviewers: meyering, sdong, rven, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D34083 | 26 February 2015, 22:19:51 UTC |
| ccef2a7 | Adam Retter | 26 February 2015, 21:47:13 UTC | Merge pull request #518 from fyrz/RocksJava-Native-Library-Loader-Fix [RocksJava] RocksJava Testcases | 26 February 2015, 21:47:13 UTC |
| 9fcf1a7 | fyrz | 26 February 2015, 21:35:26 UTC | [RocksJava] RocksJava Testcases - NativeLibraryLoader Test - EnvironmentTest Bugfix | 26 February 2015, 21:35:26 UTC |
| 03bbf71 | stash93 | 26 February 2015, 21:04:14 UTC | Return fbson Summary: mac compile is fixed in fbson, so it can be returned back from 7ce1b2c Test Plan: make all check make valgrind_check Reviewers: golovachalexander, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D33855 | 26 February 2015, 21:04:14 UTC |
| 62247ff | Igor Sugak | 26 February 2015, 19:28:41 UTC | rocksdb: Add missing override Summary: When using latest clang (3.6 or 3.7/trunck) rocksdb is failing with many errors. Almost all of them are missing override errors. This diff adds missing override keyword. No manual changes. Prerequisites: bear and clang 3.5 build with extra tools ```lang=bash % USE_CLANG=1 bear make all # generate a compilation database http://clang.llvm.org/docs/JSONCompilationDatabase.html % clang-modernize -p . -include . -add-override % make format ``` Test Plan: Make sure all tests are passing. ```lang=bash % #Use default fb code clang. % make check ``` Verify less error and no missing override errors. ```lang=bash % # Have trunk clang present in path. % ROCKSDB_NO_FBCODE=1 CC=clang CXX=clang++ make ``` Reviewers: igor, kradhakrishnan, rven, meyering, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D34077 | 26 February 2015, 19:28:41 UTC |
| 1e06a40 | Mark Callaghan | 26 February 2015, 18:50:03 UTC | Support builds for MongoDB+RocksDB Summary: This changes the RocksDB build to share headers provided by MongoDB. It is invoked as: ROCKSDB_FOR_MONGO=/path/to/mongodb/root make static_lib Task ID: # Blame Rev: Test Plan: Revert Plan: Database Impact: Memcache Impact: Other Notes: EImportant: - begin *PUBLIC* platform impact section - Bugzilla: # - end platform impact - Reviewers: igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D34071 | 26 February 2015, 18:50:03 UTC |
| b74ad66 | Adam Retter | 26 February 2015, 13:40:05 UTC | Merge pull request #508 from fyrz/RocksJava-Final-Patch [RocksJava] Final usage correction | 26 February 2015, 13:40:05 UTC |
| d9f4875 | krad | 26 February 2015, 00:34:26 UTC | Disable pre-fetching of index and filter blocks for sst_dump_tool. Summary: BlockBasedTable pre-fetches the filter and index blocks on Open call. This is an optimistic optimization targeted for runtime scenario. The optimization is unnecessary for sst_dump_tool - Added a provision to disable pre-fetching of index and filter blocks in BlockBasedTable - Disabled pre-fetching for the sst_dump tool Stack for reference : #01 0x00000000005ed944 in snappy::InternalUncompress<snappy::SnappyArrayWriter> () from /home/engshare/third-party2/snappy/1.0.3/src/snappy-1.0.3/snappy.cc:148 #02 0x00000000005edeee in snappy::RawUncompress () from /home/engshare/third-party2/snappy/1.0.3/src/snappy-1.0.3/snappy.cc:947 #03 0x00000000004e0b4d in rocksdb::UncompressBlockContents () from /data/users/paultuckfield/rocksdb/./util/compression.h:69 #04 0x00000000004e145c in rocksdb::ReadBlockContents () from /data/users/paultuckfield/rocksdb/table/format.cc:334 #05 0x00000000004ca424 in rocksdb::(anonymous namespace)::ReadBlockFromFile () from /data/users/paultuckfield/rocksdb/table/block_based_table_reader.cc:70 #06 0x00000000004cccad in rocksdb::BlockBasedTable::CreateIndexReader () from /data/users/paultuckfield/rocksdb/table/block_based_table_reader.cc:173 #07 0x00000000004d17e5 in rocksdb::BlockBasedTable::Open () from /data/users/paultuckfield/rocksdb/table/block_based_table_reader.cc:553 #08 0x00000000004c8184 in rocksdb::BlockBasedTableFactory::NewTableReader () from /data/users/paultuckfield/rocksdb/table/block_based_table_factory.cc:51 #09 0x0000000000598463 in rocksdb::SstFileReader::NewTableReader () from /data/users/paultuckfield/rocksdb/util/sst_dump_tool.cc:69 #10 0x00000000005986c2 in rocksdb::SstFileReader::SstFileReader () from /data/users/paultuckfield/rocksdb/util/sst_dump_tool.cc:26 #11 0x0000000000599047 in rocksdb::SSTDumpTool::Run () from /data/users/paultuckfield/rocksdb/util/sst_dump_tool.cc:332 #12 0x0000000000409b06 in main () from /data/users/paultuckfield/rocksdb/tools/sst_dump.cc:12 Test Plan: - Added a unit test to trigger the code. - Also did some manual verification. - Passed all unit tests task #6296048 Reviewers: igor, rven, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D34041 | 26 February 2015, 00:34:26 UTC |
| 182b4ce | Mark Callaghan | 25 February 2015, 23:53:45 UTC | Limit key range to number of keys, not number of writes Summary: An old commit (482401) changed DoWrite to use the value of --writes rather than --num to determine the range for keys. This restores the old and correct behavior which is to limit it using --num. Task ID: #6353043 Blame Rev: Test Plan: run db_bench Revert Plan: Database Impact: Memcache Impact: Other Notes: EImportant: - begin *PUBLIC* platform impact section - Bugzilla: # - end platform impact - Reviewers: igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D34065 | 25 February 2015, 23:53:45 UTC |
| a360bb6 | Igor Canadi | 25 February 2015, 22:40:15 UTC | Merge pull request #516 from fyrz/RocksJava-Update-Statistics-To-3.10 [RocksJava] Updated TickerTypes and Histogram to 3.10 | 25 February 2015, 22:40:15 UTC |
| 8c76844 | fyrz | 25 February 2015, 22:34:10 UTC | [RocksJava] Updated TickerTypes and Histogram to 3.10 Updated tickers and histogram types to 3.10. | 25 February 2015, 22:34:10 UTC |
| 7f0c77c | fyrz | 24 February 2015, 20:24:26 UTC | [RocksJava] Integrated changes from D33417 | 25 February 2015, 20:55:32 UTC |
| 819e787 | fyrz | 13 February 2015, 20:30:08 UTC | [RocksJava] Final usage correction Summary: Introduced final keyword to parameters with immutable values and classes which should not be derived. Test Plan: make rocksdbjava make jtest Reviewers: yhchiang, adamretter, ankgup87 Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D33417 | 25 February 2015, 20:55:31 UTC |
| 5139e67 | Igor Canadi | 25 February 2015, 01:19:25 UTC | Upgrade compiler in Travis | 25 February 2015, 01:19:58 UTC |
| 4ade899 | Venkatesh Radhakrishnan | 25 February 2015, 00:24:53 UTC | Fix compile error on MacOS. Summary: In a release build, a member was not being accessed. This member was only being accessed in a debug build. We now add an accessor function for this member and the buid succeeds. Test Plan: build release/unity/debug on linux/mac Reviewers: sdong, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D34035 | 25 February 2015, 00:24:53 UTC |
| ace3d85 | Igor Canadi | 24 February 2015, 21:27:41 UTC | Revert "Unused managed iterator" This reverts commit bd339a97984d58d1cbf1bde95196a74d1d404c31. Conflicts: db/managed_iterator.cc | 24 February 2015, 21:27:41 UTC |
| 7b8f348 | Igor Canadi | 24 February 2015, 20:20:43 UTC | Attempt at fixing travis issue | 24 February 2015, 20:20:43 UTC |
| d859939 | Adam Retter | 24 February 2015, 20:11:42 UTC | Merge pull request #506 from fyrz/RocksJava-Raw-Use [RocksJava] Raw use of parametrized class | 24 February 2015, 20:11:42 UTC |
| 30e93c9 | Adam Retter | 24 February 2015, 20:03:00 UTC | Merge pull request #505 from fyrz/RocksJava-Redundant-Modifier [RocksJava] Redundant access-modifier in interfaces | 24 February 2015, 20:03:00 UTC |
| 217854d | Igor Canadi | 24 February 2015, 17:52:28 UTC | Introduce DISABLE_WARNING_AS_ERROR in Makefile Summary: See https://www.facebook.com/groups/rocksdb.dev/permalink/718536351578215/. We should enable people to run without warning-as-error, since we break this so often for different compilers. Test Plan: introduce an warning, compile sucessfully Reviewers: yhchiang, meyering, igor.sugak, rven, sdong Reviewed By: sdong Subscribers: adamretter, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D33963 | 24 February 2015, 17:52:28 UTC |
| bd339a9 | Igor Canadi | 24 February 2015, 17:51:52 UTC | Unused managed iterator Summary: This causes warnings on OS X Test Plan: compiles Reviewers: rven, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D33969 | 24 February 2015, 17:51:52 UTC |
| 174a79c | stash93 | 24 February 2015, 06:05:06 UTC | LevelDb include guard replaced with #pragma once Summary: Replaced LevelDb include guards with #pragma once Test Plan: make all check Reviewers: igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D33939 | 24 February 2015, 06:05:06 UTC |
| 6fdda8a | Igor Sugak | 24 February 2015, 01:45:25 UTC | rocksdb: changes to support 'make analyze' in Jenkins Summary: * Updated Makefile to exit `make analyze` with status 1 if scan-build detected any bugs. * scan-build automatically detects which c++ compiler to use, and some times is uses wrong ones (from $CPP). Added implicit parameters to use $CC and $CXX. * Added `scan_build_report` directory to .gitignore file. * Added `scan_build_report` directory to clean target. Test Plan: Run `make analyze` and verify that exit status is 1, if there are scan-build bugs detected. Run `make clean` and verify that files in `scan_build_report` directory are deleted. After running `make analyze; git status` and verify that no untracked files in `scan_build_report` directory. Reviewers: meyering, sdong, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D33831 | 24 February 2015, 01:45:25 UTC |
| 96d989f | Jinfu Leng | 24 February 2015, 00:08:27 UTC | catch config errors with L0 file count triggers Test Plan: Run "make clean && make all check" Reviewers: rven, igor, yhchiang, kradhakrishnan, MarkCallaghan, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D33627 | 24 February 2015, 00:08:27 UTC |
| 62f7a1b | Igor Sugak | 23 February 2015, 22:10:09 UTC | rocksdb: Fixed 'Dead assignment' and 'Dead initialization' scan-build warnings Summary: This diff contains trivial fixes for 6 scan-build warnings: **db/c_test.c** `db` variable is never read. Removed assignment. scan-build report: http://home.fburl.com/~sugak/latest20/report-9b77d2.html#EndPath **db/db_iter.cc** `skipping` local variable is assigned to false. Then in the next switch block the only "non return" case assign `skipping` to true, the rest cases don't use it and all do return. scan-build report: http://home.fburl.com/~sugak/latest20/report-13fca7.html#EndPath **db/log_reader.cc** In `bool Reader::SkipToInitialBlock()` `offset_in_block` local variable is assigned to 0 `if (offset_in_block > kBlockSize - 6)` and then never used. Removed the assignment and renamed it to `initial_offset_in_block` to avoid confusion. scan-build report: http://home.fburl.com/~sugak/latest20/report-a618dd.html#EndPath In `bool Reader::ReadRecord(Slice* record, std::string* scratch)` local variable `in_fragmented_record` in switch case `kFullType` block is assigned to false and then does `return` without use. In the other switch case `kFirstType` block the same `in_fragmented_record` is assigned to false, but later assigned to true without prior use. Removed assignment for both cases. scan-build reprots: http://home.fburl.com/~sugak/latest20/report-bb86b0.html#EndPath http://home.fburl.com/~sugak/latest20/report-a975be.html#EndPath **table/plain_table_key_coding.cc** Local variable `user_key_size` is assigned when declared. But then in both places where it is used assigned to `static_cast<uint32_t>(key.size() - 8)`. Changed to initialize the variable to the proper value in declaration. scan-build report: http://home.fburl.com/~sugak/latest20/report-9e6b86.html#EndPath **tools/db_stress.cc** Missing `break` in switch case block. This seems to be a bug. Added missing `break`. Test Plan: Make sure all tests are passing and scan-build does not report 'Dead assignment' and 'Dead initialization' bugs. ```lang=bash % make check % make analyze ``` Reviewers: meyering, igor, kradhakrishnan, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D33795 | 23 February 2015, 22:10:09 UTC |
| 5636c4a | Igor Canadi | 23 February 2015, 20:32:31 UTC | Verbose build in travis | 23 February 2015, 20:32:31 UTC |
| a047409 | krad | 23 February 2015, 19:59:39 UTC | Fixed a bug in the test case Summary: The unit test was supposed to check that the old file and the new file contains the header message. Test Plan: Run the unit test. Reviewers: sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D33705 | 23 February 2015, 19:59:39 UTC |
| 4f514a5 | Jim Meyering | 20 February 2015, 00:45:32 UTC | build: enable more compiler warnings Summary: * Makefile (WARNING_FLAGS): Add -W and -Wextra, and at least for now, -Wno-unused-parameter. Without that latter one, there would be many errors/warnings about unused parameters. Test Plan: run these and ensure there is no warning: make clean && make make clean && USE_CLANG=1 make Reviewers: ljin, sdong, igor.sugak, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D33819 | 20 February 2015, 23:02:40 UTC |
| a2b911b | Jim Meyering | 20 February 2015, 19:21:19 UTC | inputs: restore "const" attribute removed by D33759 Summary: The "const" attribute applies to the type, and placing it before that return type retains the desired semantics, yet avoids the compiler error/warning. Test Plan: Run make Reviewers: ljin, sdong, igor.sugak, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D33789 | 20 February 2015, 19:52:20 UTC |
| 1b40825 | Jim Meyering | 19 February 2015, 23:55:06 UTC | mark as unused some variables with cpp-derived names Summary: as above Test Plan: Run "make EXTRA_CXXFLAGS='-W -Wextra'" and see fewer errors. Reviewers: ljin, sdong, igor.sugak, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D33753 | 20 February 2015, 19:07:21 UTC |
| c6d54b5 | Jim Meyering | 20 February 2015, 00:44:36 UTC | fix erroneous assert: cast kBlockSize (of type unsigned int) to "int" Summary: Otherwise, we would assert that an unsigned expression is always >= 0. The intent was to form a possibly negative number, and to assert that that value is always >= 0, but since one variable in the computation was unsigned, the result was guaranteed to be unsigned, too, rendering the assertion useless. Cast that unsigned variable to "int", so that all operands are signed, and thus so that the result can be negative. Test Plan: Run "make EXTRA_CXXFLAGS='-W -Wextra'" and see fewer errors. Reviewers: ljin, sdong, igor.sugak, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D33771 | 20 February 2015, 19:07:17 UTC |
| aa5d8e6 | Jim Meyering | 20 February 2015, 00:42:22 UTC | table_test.cc: add missing 5th arg in TestArgs initializer Summary: Adding -W and -Wextra to CXXFLAGS provoked this failure: table/table_test.cc:1854:56: error: missing initializer for member ‘rocksdb::TestArgs::format_version’ [-Werror=missing-field-initializers] TestArgs args = { DB_TEST, false, 16, kNoCompression }; ^ Add the missing, 5th value (format_version). Test Plan: Run "make EXTRA_CXXFLAGS='-W -Wextra'" and see fewer errors. Reviewers: ljin, sdong, igor.sugak, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D33765 | 20 February 2015, 19:07:11 UTC |
| c37937a | Jim Meyering | 20 February 2015, 00:41:38 UTC | maint: remove extraneous "const" attribute from return type Summary: The "const" attribute does not make sense on a return type, and provokes a warning/error from gcc -W -Wextra. Test Plan: Run "make EXTRA_CXXFLAGS='-W -Wextra'" and see fewer errors. Reviewers: ljin, sdong, igor.sugak, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D33759 | 20 February 2015, 19:07:07 UTC |
| 9283c7a | Jim Meyering | 19 February 2015, 23:54:19 UTC | build: remove always-true assertions Summary: Remove some always-true assertions. They provoke these compilation failures: table/plain_table_key_coding.cc:279:20: error: comparison of unsigned expression >= 0 is always true [-Werror=type-limits] db/version_set.cc:336:15: error: comparison of unsigned expression >= 0 is always true [-Werror=type-limits] * table/plain_table_key_coding.cc (rocksdb): Remove assertion that unsigned type variable is >= 0. * db/version_set.cc (DoGenerateLevelFilesBrief): Likewise. Test Plan: Run "make EXTRA_CXXFLAGS='-W -Wextra'" and see fewer errors. Reviewers: ljin, sdong, igor.sugak, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D33747 | 20 February 2015, 19:07:03 UTC |
| 06a766d | Islam AbdelRahman | 20 February 2015, 18:58:29 UTC | Adding Flush to AutoRollLogger Summary: During running AutoRollLoggerTest on FreeBSD we have found out that AutoRollLogger is not flushing correctly (test fails on FreeBSD) This diff add Flush to AutoRollLogger to fix this problem Test Plan: [My machine] make all check ( all tests pass) [FreeBSD VM] running AutoRollLoggerTest ( all tests pass ) Reviewers: sdong, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D33633 | 20 February 2015, 18:58:29 UTC |
| 92416fa | Igor Canadi | 20 February 2015, 03:26:38 UTC | Fix mac build | 20 February 2015, 03:26:38 UTC |
| 96ab15d | Igor Canadi | 20 February 2015, 03:11:14 UTC | GetOptionsFromString + fixes to block_based_table_options Summary: In mongo, we currently have a single column family and I'd like to support setting rocksdb::Options from string. This diff provides an option to GetOptionsFromString() There's one more problem. Currently GetColumnFamilyOptionsFromString() overwrites block_based options. In mongo I set default values for block_cache and some other values of BlockBasedTableOptions and I don't want them reset to default with GetOptionsFromString(). Test Plan: added unit test Reviewers: yhchiang, rven, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D33729 | 20 February 2015, 03:11:14 UTC |
| 73711f9 | Igor Sugak | 19 February 2015, 22:27:48 UTC | rocksdb: Fix scan-build bug 'Memory leak' in db/db_bench.cc Summary: The bug is detected by scan-build. In `void WriteSeqSeekSeq(ThreadState* thread)` memory is allocated in line 3118 `Slice key = AllocateKey();` but `Slice` is not responsible deleting `Slice::data()`. Added `std::unique_ptr<const char[]>*` parameter to ` AllocateKey()`, so that it requires caller to not forget about Slice::data() management. scan-build bug report: http://home.fburl.com/~sugak/latest6/report-6e9754.html#EndPath Test Plan: Make sure scan-build does not report 'Memory leak' in db/db_bench.cc and all tests are passing. ```lang=bash % make analyze % make check ``` Reviewers: lgalanis, igor, meyering, sdong Reviewed By: meyering, sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D33501 | 19 February 2015, 22:27:48 UTC |
| 98870c7 | Igor Sugak | 19 February 2015, 22:07:38 UTC | rocksdb: Fix scan-build memory warning in table/block_based_table_reader.cc Summary: scan-build is reporting two memory leak bugs in `table/block_based_table_reader.cc`. They are both false positives. In both cases we allocate memory in `ReadBlockFromFile` if `s.ok()`. Then after the function `ReadBlockFromFile` returns we check for the same variable if `s.ok()` and then use the memory that was allocated. The bugs reported by scan-build is if `ReadBlockFromFile` allocates memory and returns, but for some reason status `s` is not the same and `s.ok() != true`. In this case scan-build is concerned that memory owner transfer is not explicit. I modified `ReadBlockFromFile` to accept `std::unique_ptr<Block>*` as a parameter, instead of raw pointer. scan-build reports: http://home.fburl.com/~sugak/latest2/report-a4b3fa.html#EndPath http://home.fburl.com/~sugak/latest2/report-29adbf.html#EndPath Test Plan: Make sure scan-build does not report these bugs and all tests are passing. ```lang=bash % make check % make analyze ``` Reviewers: sdong, lgalanis, meyering, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D33681 | 19 February 2015, 22:07:38 UTC |
