| 149ef19 | Venkatesh Radhakrishnan | 20 April 2015, 18:21:29 UTC | Bump up patch to 3.10.1 Summary: Bump up patch in preparation for 3.10.1 Test Plan: make_release.py Reviewers: igor, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D37395 | 20 April 2015, 18:56:01 UTC |
| 8893171 | sdong | 15 April 2015, 02:58:52 UTC | Bug of trivial move of dynamic level Summary: D36669 introduces a bug that trivial moved data is not going to specific level but the next level, which will incorrectly be level 1 for level 0 compaciton if base level is not level 1. Fixing it by appreciating the output level Test Plan: Run all tests Reviewers: MarkCallaghan, rven, yhchiang, igor Reviewed By: igor Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D37119 Conflicts: db/db_test.cc | 17 April 2015, 17:33:27 UTC |
| 82e5c84 | Igor Canadi | 13 April 2015, 22:58:45 UTC | Fix bad performance in debug mode Summary: See github issue 574: https://github.com/facebook/rocksdb/issues/574 Basically when we're running in DEBUG mode we're calling `usleep(0)` on every mutex lock. I bisected the issue to https://reviews.facebook.net/D36963. Instead of calling sleep(0), this diff just avoids calling SleepForMicroseconds() when delay is not set. Test Plan: bpl=10485760;overlap=10;mcz=2;del=300000000;levels=2;ctrig=10000000; delay=10000000; stop=10000000; wbn=30; mbc=20; mb=1073741824;wbs=268435456; dds=1; sync=0; r=100000; t=1; vs=800; bs=65536; cs=1048576; of=500000; si=1000000; ./db_bench --benchmarks=fillrandom --disable_seek_compaction=1 --mmap_read=0 --statistics=1 --histogram=1 --num=$r --threads=$t --value_size=$vs --block_size=$bs --cache_size=$cs --bloom_bits=10 --cache_numshardbits=4 --open_files=$of --verify_checksum=1 --db=/tmp/rdb10test --sync=$sync --disable_wal=1 --compression_type=snappy --stats_interval=$si --compression_ratio=0.5 --disable_data_sync=$dds --write_buffer_size=$wbs --target_file_size_base=$mb --max_write_buffer_number=$wbn --max_background_compactions=$mbc --level0_file_num_compaction_trigger=$ctrig --level0_slowdown_writes_trigger=$delay --level0_stop_writes_trigger=$stop --num_levels=$levels --delete_obsolete_files_period_micros=$del --min_level_to_compress=$mcz --max_grandparent_overlap_factor=$overlap --stats_per_interval=1 --max_bytes_for_level_base=$bpl --memtablerep=vector --use_existing_db=0 --disable_auto_compactions=1 --source_compaction_factor=10000000 | grep ops Before: fillrandom : 117.525 micros/op 8508 ops/sec; 6.6 MB/s After: fillrandom : 1.283 micros/op 779502 ops/sec; 606.6 MB/s Reviewers: rven, yhchiang, sdong Reviewed By: sdong Subscribers: meyering, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D36963 | 13 April 2015, 22:59:29 UTC |
| be55065 | Yueh-Hsuan Chiang | 10 April 2015, 04:05:09 UTC | Fixed xfunc related compile errors in ROCKSDB_LITE Summary: Fixed xfunc related compile errors in ROCKSDB_LITE Now make OPT=-DROCKSDB_LITE shared_lib -j32 would work Test Plan: make clean make OPT=-DROCKSDB_LITE shared_lib -j32 make clean make OPT=-DROCKSDB_LITE static_lib -j32 Reviewers: sdong, igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D36825 | 10 April 2015, 04:32:03 UTC |
| 159a99f | Yueh-Hsuan Chiang | 10 April 2015, 00:05:17 UTC | Fix a compile error in ROCKSDB_LITE in db/db_impl.cc Summary: Fix a compile error in ROCKSDB_LITE in db/db_impl.cc related to internal_stats. Test Plan: make OPT=-DROCKSDB_LITE shared_lib Reviewers: sdong, igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D36819 | 10 April 2015, 04:31:44 UTC |
| b573aa8 | Yueh-Hsuan Chiang | 09 April 2015, 23:39:03 UTC | Fix a compilation error in ROCKSDB_LITE in db/internal_stats.h Summary: Fix a compilation error in ROCKSDB_LITE in db/internal_stats.h Other compilation errors will be fixed in a separate diff. Test Plan: make OPT=-DROCKSDB_LITE Reviewers: sdong, igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D36807 | 10 April 2015, 04:31:08 UTC |
| 3e32a88 | sdong | 03 April 2015, 05:24:50 UTC | Fix level size overflow for options_.level_compaction_dynamic_level_bytes=true Summary: Int is used for level size targets when options_.level_compaction_dynamic_level_bytes=true, which will cause overflow when database grows big. Fix it. Test Plan: Add a new unit test which fails without the fix. Reviewers: rven, yhchiang, MarkCallaghan, igor Reviewed By: igor Subscribers: leveldb, dhruba, yoshinorim Differential Revision: https://reviews.facebook.net/D36453 | 03 April 2015, 18:09:05 UTC |
| 56b9e4a | Venkatesh Radhakrishnan | 01 April 2015, 23:55:08 UTC | Fix crash caused by opening an empty DB in readonly mode Summary: This diff fixes a crash found when an empty database is opened in readonly mode. We now check the number of levels before we open the DB as a compacted DB. Test Plan: DBTest.EmptyCompactedDB Reviewers: igor, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D36327 Conflicts: db/db_test.cc | 03 April 2015, 18:05:19 UTC |
| ff1ff7c | Jim Meyering | 25 March 2015, 21:40:41 UTC | TSAN: avoid new link failure with -pg Summary: * Makefile (COMPILE_WITH_TSAN): Avoid a link failure by disabling -pg when building with TSAN enabled. Now that "make check" builds all $(PROGRAMS), it is linking a few programs that were not normally linked before. For example, this would fail to link with the following diagnostic: COMPILE_WITH_TSAN=1 make -j40 log_and_apply_bench CCLD log_and_apply_bench ld: /usr/lib/../lib64/gcrt1.o: relocation R_X86_64_32S against `__libc_csu_fini' can not be used when making a shared object; recompile with -fPIC /usr/lib/../lib64/gcrt1.o: error adding symbols: Bad value collect2: error: ld returned 1 exit status Makefile:511: recipe for target 'log_and_apply_bench' failed make: *** [log_and_apply_bench] Error 1 Since removing -pg is sufficient to get past this link failure, and no one cares about profiling TSAN-enabled binaries anyway, we will refrain from linking with -pg when TSAN testing is enabled. Use a new variable, "pg" which is set to "-pg" in most cases, but that is made empty when COMPILE_WITH_TSAN is set. Test Plan: Now, this succeeds: rm -f log_and_apply_bench COMPILE_WITH_TSAN=1 make -j40 log_and_apply_bench Reviewers: igor.sugak, rven, sdong, ljin, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D35943 | 25 March 2015, 21:40:41 UTC |
| 39d508e | Yueh-Hsuan Chiang | 25 March 2015, 21:14:26 UTC | Add a missing section title in HISTORY.md Summary: Add a missing section title in HISTORY.md Test Plan: no code change | 25 March 2015, 21:14:26 UTC |
| 727684b | Yueh-Hsuan Chiang | 25 March 2015, 18:08:08 UTC | Fixed a typo in RocksDBSample.java Summary: Fixed a typo in RocksDBSample.java Test Plan: make clean make rocksdbjava -j32 make jtest | 25 March 2015, 18:09:30 UTC |
| ca25e86 | Jim Meyering | 25 March 2015, 05:09:40 UTC | build: cause the "check" rule to depend on $(PROGRAMS) Summary: * Makefile (check): Cause "make check" to build all $(PROGRAMS), so that it verifies that the few benchmark-only source files that not already built via "make check" do compile and link successfully. Test Plan: run "make clean; make check", and verify that table/table_reader_bench.cc is now compiled. Before, it was not, which led to an incomplete fix for a build break. Reviewers: ljin, igor.sugak, rven, igor, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D35883 | 25 March 2015, 05:09:40 UTC |
| 986bdc6 | Anurag Indu | 25 March 2015, 00:00:41 UTC | Merge branch 'master' of github.com:facebook/rocksdb | 25 March 2015, 00:00:41 UTC |
| 1e57f2b | Anurag Indu | 24 March 2015, 23:53:49 UTC | Fix build Test Plan: Running make all Reviewers: sdong Reviewed By: sdong Subscribers: rven, yhchiang, igor, meyering, dhruba Differential Revision: https://reviews.facebook.net/D35889 | 25 March 2015, 00:00:28 UTC |
| 2d417e5 | Yueh-Hsuan Chiang | 24 March 2015, 23:39:39 UTC | Update HISTORY.md for 3.10.0 Summary: Update HISTORY.md for 3.10.0 Test Plan: no code chagne. Reviewers: sdong, rven, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D35871 | 24 March 2015, 23:39:39 UTC |
| cbd6a20 | Anurag Indu | 24 March 2015, 23:27:39 UTC | Merge branch 'master' of github.com:facebook/rocksdb | 24 March 2015, 23:27:39 UTC |
| 211ca26 | Anurag Indu | 24 March 2015, 22:06:09 UTC | Fixing build issue Summary: Fixing issues with get context function. Test Plan: Run make commit-prereq Reviewers: sdong, meyering, yhchiang Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D35853 | 24 March 2015, 23:27:24 UTC |
| 2495f93 | Yueh-Hsuan Chiang | 24 March 2015, 22:39:42 UTC | Merge pull request #555 from pshareghi/staticLZ4 [RocksJava] Added static lz4 support for roccksjavastatic | 24 March 2015, 22:39:42 UTC |
| f06de5f | Yueh-Hsuan Chiang | 24 March 2015, 21:58:00 UTC | Merge pull request #556 from fyrz/RocksJava-Maven-Fix [RocksJava] Maven build fix | 24 March 2015, 21:58:00 UTC |
| cd987c3 | Yueh-Hsuan Chiang | 24 March 2015, 21:50:32 UTC | Fix compile error when NROCKSDB_THREAD_STATUS is not used. Summary: Fix compile error when NROCKSDB_THREAD_STATUS is not used. Test Plan: make dbg OPT=-DNROCKSDB_THREAD_STATUS -j32 Reviewers: sdong, igor, rven Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D35847 | 24 March 2015, 21:52:59 UTC |
| 8f104ae | fyrz | 24 March 2015, 21:52:30 UTC | [RocksJava] Maven build fix | 24 March 2015, 21:52:30 UTC |
| 3d1a924 | Anurag Indu | 03 March 2015, 18:59:36 UTC | Adding stats for the merge and filter operation Summary: We have addded new stats and perf_context for measuring the merge and filter operation time consumption. We have bounded all the merge operations within the GUARD statment and collected the total time for these operations in the DB. Test Plan: WIP Reviewers: rven, yhchiang, kradhakrishnan, igor, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D34377 | 24 March 2015, 21:42:04 UTC |
| afc5164 | Yueh-Hsuan Chiang | 24 March 2015, 21:01:12 UTC | Merge pull request #546 from fyrz/RocksJava-MemEnv [RocksJava] Expose MemEnv in RocksJava | 24 March 2015, 21:01:12 UTC |
| 4806cc1 | Pooya Shareghi | 24 March 2015, 17:40:00 UTC | Added static lz4 support for roccksjavastatic Usage: make clean rocksdbjavastatic Dependency: LZ4 To install LZ4: git clone https://github.com/Cyan4973/lz4.git && \ cd lz4/lib && \ make install Alternativel installation method: curl -O https://codeload.github.com/Cyan4973/lz4/tar.gz/r127 mv r127 lz4-r127.tar.gz tar xvzf lz4-r127.tar.gz cd lz4-r127/lib && \ make install Signed-off-by: Pooya Shareghi <shareghi@gmail.com> | 24 March 2015, 20:48:53 UTC |
| 6284eef | Yueh-Hsuan Chiang | 24 March 2015, 19:14:09 UTC | Merge pull request #545 from fyrz/RocksJava-Level-Compression [RocksJava] Add compression per level to API | 24 March 2015, 19:14:09 UTC |
| 248c063 | Yueh-Hsuan Chiang | 23 March 2015, 23:35:04 UTC | Report elapsed time in micros in ThreadStatus instead of start time. Summary: Report elapsed time of a thread operation in micros in ThreadStatus instead of start time of a thread operation in seconds since the Epoch, 1970-01-01 00:00:00 (UTC). Test Plan: ./db_bench --benchmarks=fillrandom --num=100000 --threads=40 \ --max_background_compactions=10 --max_background_flushes=3 \ --thread_status_per_interval=1000 --key_size=16 --value_size=1000 \ --num_column_families=10 Sample Output: ThreadID ThreadType cfName Operation ElapsedTime Stage State 140667724562496 High Pri column_family_name_000002 Flush 772.419 ms FlushJob::WriteLevel0Table 140667728756800 High Pri default Flush 617.845 ms FlushJob::WriteLevel0Table 140667732951104 High Pri column_family_name_000005 Flush 772.078 ms FlushJob::WriteLevel0Table 140667875557440 Low Pri column_family_name_000008 Compaction 1409.216 ms CompactionJob::Install 140667737145408 Low Pri 140667749728320 Low Pri 140667816837184 Low Pri column_family_name_000007 Compaction 1071.815 ms CompactionJob::ProcessKeyValueCompaction 140667787477056 Low Pri column_family_name_000009 Compaction 772.516 ms CompactionJob::ProcessKeyValueCompaction 140667741339712 Low Pri 140667758116928 Low Pri column_family_name_000004 Compaction 620.739 ms CompactionJob::ProcessKeyValueCompaction 140667753922624 Low Pri 140667842003008 Low Pri column_family_name_000006 Compaction 1260.079 ms CompactionJob::ProcessKeyValueCompaction 140667745534016 Low Pri Reviewers: sdong, igor, rven Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D35769 | 24 March 2015, 18:32:25 UTC |
| 315abac | Igor Canadi | 24 March 2015, 16:42:38 UTC | Undeprecate GetLiveFiles() Summary: There is no alternative to GetLiveFiles() function Test Plan: none Reviewers: sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D35805 | 24 March 2015, 16:42:38 UTC |
| a057bb2 | Yueh-Hsuan Chiang | 23 March 2015, 22:05:58 UTC | Improve ThreadStatusSingleCompaction Summary: Improve ThreadStatusSingleCompaction in two ways: 1. Use SYNC_POINT to ensure compaction won't happen before the test finishes its "Put Phase" instead of using sleep. 2. In Put Phase, it continues until we have sufficient number of L0 files. Note that during the put phase, there won't be any compaction that consumes L0 files because of item 1. Test Plan: ./db_test --gtest_filter="*ThreadStatusSingleCompaction*" Reviewers: sdong, igor, rven Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D35727 | 23 March 2015, 22:30:45 UTC |
| 868968c | krad | 23 March 2015, 22:04:48 UTC | Merge branch 'master' of github.com:facebook/rocksdb | 23 March 2015, 22:04:48 UTC |
| 6893914 | krad | 23 March 2015, 20:38:17 UTC | Make SSTDumpTest.GetProperties less noisy Summary: Limiting verbose printing to "command=scan" Test Plan: Run make check and manual testing of sst_dump_test Reviewers: sdong CC: leveldb Task ID: #6575982 Blame Rev: | 23 March 2015, 21:30:11 UTC |
| 8d86562 | Yueh-Hsuan Chiang | 23 March 2015, 20:25:37 UTC | Merge pull request #551 from fyrz/RocksJava-JavaDoc-Fix [RocksJava] Add missing JavaDoc annotations | 23 March 2015, 20:25:37 UTC |
| f8c505b | Yueh-Hsuan Chiang | 23 March 2015, 20:24:37 UTC | Merge pull request #550 from fyrz/RocksJava-Sample-Fix [RocksJava] Java sample bugfix | 23 March 2015, 20:24:37 UTC |
| 46443bf | fyrz | 23 March 2015, 20:17:20 UTC | [RocksJava] Add missing JavaDoc annotations | 23 March 2015, 20:17:20 UTC |
| 864b7e8 | fyrz | 23 March 2015, 20:03:28 UTC | [RocksJava] Java sample bugfix One of the latest commits broke the sample code. This resolves the introduced bug. | 23 March 2015, 20:03:28 UTC |
| 2b2394c | fyrz | 22 March 2015, 18:02:36 UTC | [RocksJava] DBBenchmark option for RocksMemEnv Extended DBBenchmark with option to enable RocksMemEnv. | 23 March 2015, 17:59:32 UTC |
| fd8804f | fyrz | 22 March 2015, 17:30:51 UTC | [RocksJava] Expose MemEnv in RocksJava Summary: In 3.10 the C++ code was extended with a MemEnv implementation. This is now also available in RocksJava. Changes: - Extraced abstract super class Env - Introduced RocksMemEnv - Remove unnecessary disposeInternal method. The disposal of the default environment is managed by C++ so there needs to be no disposeInternal method in Java. - Introduced a RocksMemEnvTest, which is aligned with the C++ equivalent. Test Plan: make rocksdbjava make jtest Reviewers: adamretter, yhchiang, ankgup87 Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D35619 | 23 March 2015, 17:59:31 UTC |
| 004b89f | fyrz | 19 March 2015, 22:50:45 UTC | [RocksJava] Add compression per level to API Summary: RocksDB offers the possibility to set different compression types on a per level basis. This shall be also available using RocksJava. Test Plan: make rocksdbjava make jtest Reviewers: adamretter, yhchiang, ankgup87 Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D35577 | 23 March 2015, 17:58:56 UTC |
| 51da3aa | Adam Retter | 22 March 2015, 23:33:35 UTC | Merge pull request #536 from fyrz/RocksJava-32-Bit-adjustment [RocksJava] 32-Bit adjustments | 22 March 2015, 23:33:35 UTC |
| b7e88cf | Adam Retter | 22 March 2015, 23:16:02 UTC | Merge pull request #543 from fyrz/RocksJava-Logger-Comment [RocksJava] Enhanced Logger comment | 22 March 2015, 23:16:02 UTC |
| 38d286f | sdong | 20 March 2015, 19:13:18 UTC | Clean-up WAL directory before running db_test Summary: DBTest doesn't clean up wal directory. It might cause failure after a failure test run. Fix it. Test Plan: Run unit tests Try open DB with non-empty db_path/wal. Reviewers: rven, yhchiang, kradhakrishnan, igor Reviewed By: igor Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D35559 | 20 March 2015, 19:34:24 UTC |
| 28bc6de | Igor Sugak | 20 March 2015, 00:32:43 UTC | rocksdb: print status error message when (ASSERT|EXPECT)_OK fails Summary: Modified rocksdb status assertions ASSERT_OK and EXPECT_OK to print error message from Status::ToString() when failed. Test Plan: Modify a test to fail status assertions ASSERT_OK and EXPECT_OK and notice an error message that came from Status::ToString() Reviewers: meyering, sdong, yhchiang, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D35469 | 20 March 2015, 00:32:43 UTC |
| 9405b5e | Igor Sugak | 20 March 2015, 00:29:37 UTC | rocksdb: Remove #include "util/string_util.h" from util/testharness.h Summary: 1. Manually deleted #include "util/string_util.h" from util/testharness.h 2. ``` % USE_CLANG=1 make all -j55 -k 2> build.log % perl -naF: -E 'say $F[0] if /: error:/' build.log | sort -u | xargs sed -i '/#include "util\/testharness.h"/i #include "util\/string_util.h"' ``` Test Plan: Make sure make all completes with no errors. ``` % make all -j55 ``` Reviewers: meyering, igor, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D35493 | 20 March 2015, 00:29:37 UTC |
| 220d0df | Igor Sugak | 20 March 2015, 00:06:02 UTC | rocksdb: Remove #include "util/random.h" from util/testharness.h Summary: Cleaning util/testharness.h Test Plan: Make completes with no errors. ``` % make all ``` Reviewers: meyering, igor, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D35487 | 20 March 2015, 00:06:02 UTC |
| b088c83 | Igor Canadi | 20 March 2015, 00:04:29 UTC | Don't delete files when column family is dropped Summary: To understand the bug read t5943287 and check out the new test in column_family_test (ReadDroppedColumnFamily), iter 0. RocksDB contract allowes you to read a drop column family as long as there is a live reference. However, since our iteration ignores dropped column families, AddLiveFiles() didn't mark files of a dropped column families as live. So we deleted them. In this patch I no longer ignore dropped column families in the iteration. I think this behavior was confusing and it also led to this bug. Now if an iterator client wants to ignore dropped column families, he needs to do it explicitly. Test Plan: Added a new unit test that is failing on master. Unit test succeeds now. Reviewers: sdong, rven, yhchiang Reviewed By: yhchiang Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D32535 | 20 March 2015, 00:04:29 UTC |
| 17ae3fc | Igor Sugak | 19 March 2015, 23:52:59 UTC | rocksdb: initial util/testharness clean up Summary: Deleted some redundant code. More comming. Test Plan: ```lang=bash % USE_CLANG=1 make check ``` Reviewers: meyering, sdong, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D35463 | 19 March 2015, 23:52:59 UTC |
| 39f4271 | fyrz | 19 March 2015, 20:58:03 UTC | [RocksJava] Enhanced Logger comment Added information about performance penalties using a custom logger implementation. | 19 March 2015, 21:01:00 UTC |
| 5615e23 | fyrz | 12 March 2015, 21:25:57 UTC | [RocksJava] 32-Bit adjustments Summary: Before this change overflowing size_t values led to a checked Exception. After that change: size_t overflows on 32-Bit architecture throw now an IllegalArgumentException, which removes the necessity for a developer to catch these Exceptions explicitly. This is especially an advantage for developers targeting 64-Bit systems because it is not necessary anymore to catch exceptions which are never thrown on a 64-Bit system. Test Plan: make clean jclean rocksdbjava jtest mvn -f rocksjni.pom package Reviewers: adamretter, yhchiang, ankgup87 Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D34923 | 19 March 2015, 19:45:55 UTC |
| 836bcc2 | Adam Retter | 19 March 2015, 12:51:40 UTC | Merge pull request #532 from fyrz/RocksJava-LevelCompactionDynamicLevelBytes [RocksJava] Added LevelCompactionDynamicLevelBytes to Options | 19 March 2015, 12:51:40 UTC |
| 67d9958 | Igor Sugak | 19 March 2015, 02:12:23 UTC | rocksdb: fix make unity Summary: changed make unity target to use $LIB_SOURCES as a source of library source code. In the old way (using find) table/mock_table.h was added to a list of library objects and this was a course of `make unity` break. `build_tools/unity` contains some redundant code, I deleted it and moved the functionality in the Makefile. Test Plan: Make sure unity completes with no errors. ```lang=bash % make unity ``` Reviewers: sdong, rven, igor, meyering Reviewed By: igor, meyering Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D35385 | 19 March 2015, 02:12:23 UTC |
| 52e0f33 | Igor Canadi | 19 March 2015, 01:25:15 UTC | Clean up compactions_in_progress_ Summary: Suprisingly, the only way we use this vector is to keep track of level0 compactions. Thus, I simplified it. Test Plan: make check Reviewers: rven, yhchiang, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D35313 | 19 March 2015, 01:25:15 UTC |
| 6b626ff | Igor Sugak | 19 March 2015, 01:18:12 UTC | rocksdb: change db_test::MultiThreadedDBTest as value parameterized test. Summary: This is a simple change to make db_test::MultiThreadedDBTest as value parameterized test. There is a value of creating a separate set of such tests later. Test Plan: ```lang=bash % make db_test % ./make db_test ``` Also with the following command I can execute all db_test in 2:37.87 on my box ``` % ./db_test --gtest_list_tests | sed 's/\# GetParam.*//' | tr -d ' ' | env time parallel --gnu --eta --joblog=LOG -- 'TEST_TMPDIR=/dev/shm/rocksdb-{} ./db_test --gtest_filter="*{}"' ``` Reviewers: igor, rven, meyering, sdong Reviewed By: meyering Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D35361 | 19 March 2015, 01:18:12 UTC |
| 9720ea4 | sdong | 19 March 2015, 00:26:29 UTC | A build option to run through all check-in requirements. Summary: Make it easier for people to run all the tests. Test Plan: Run it. Reviewers: rven, yhchiang, igor, MarkCallaghan, IslamAbdelRahman, igor.sugak, anthony, kradhakrishnan, meyering Reviewed By: meyering Subscribers: meyering, leveldb, dhruba Differential Revision: https://reviews.facebook.net/D35319 | 19 March 2015, 00:57:42 UTC |
| 0831a35 | sdong | 18 March 2015, 23:11:02 UTC | Add a DB Property For Number of Deletions in Memtables Summary: Add a DB property for number of deletions in memtables. It can sometimes help people debug slowness because of too many deletes. Test Plan: Add test cases. Reviewers: rven, yhchiang, kradhakrishnan, igor Reviewed By: igor Subscribers: leveldb, dhruba, yoshinorim Differential Revision: https://reviews.facebook.net/D35247 | 19 March 2015, 00:03:59 UTC |
| f7ed654 | Yueh-Hsuan Chiang | 18 March 2015, 22:40:27 UTC | Fix RocksJava test failure due to deprecation of table_cache_remove_scan_count_limit Summary: table_cache_remove_scan_count_limit is marked as deprecated in RocksDB C++. (see rocksdb/options.h). This patch made necessary changes on RocksJava side. Test Plan: make rocksdbjava -j32 make jtest Reviewers: rven, igor, fyrz, adamretter, ankgup87, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D35355 | 18 March 2015, 22:40:27 UTC |
| 51301b8 | Igor Canadi | 18 March 2015, 22:35:55 UTC | Enable dynamic changing of rate limiter's bytes_per_second Summary: This feature is going to be useful for mongodb+rocksdb. I'll expose it through mongo's API. Test Plan: added new unit test. also will run TSAN on the new unit test Reviewers: meyering, sdong Reviewed By: meyering, sdong Subscribers: meyering, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D35307 | 18 March 2015, 22:35:55 UTC |
| 652db51 | Yueh-Hsuan Chiang | 18 March 2015, 22:08:31 UTC | Fix compilation error in rocksjni/write_batch_test.cc Summary: Fix compilation error in rocksjni/write_batch_test.cc Test Plan: make rocksdbjava -j32 Reviewers: sdong, igor, rven Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D35343 | 18 March 2015, 22:10:57 UTC |
| dfccc7b | Mark Callaghan | 18 March 2015, 20:50:52 UTC | Add readwhilemerging benchmark Summary: This is like readwhilewriting but uses Merge rather than Put in the writer thread. I am using it for in-progress benchmarks. I don't think the other benchmarks for Merge cover this behavior. The purpose for this test is to measure read performance when readers might have to merge results. This will also benefit from work-in-progress to add skewed key generation. Task ID: # Blame Rev: Test Plan: Revert Plan: Database Impact: Memcache Impact: Other Notes: EImportant: - begin *PUBLIC* platform impact section - Bugzilla: # - end platform impact - Reviewers: igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D35115 | 18 March 2015, 20:50:52 UTC |
| c345d1e | fyrz | 18 March 2015, 20:37:46 UTC | [RocksJava] Integrated changes for D34857 | 18 March 2015, 20:38:17 UTC |
| 1235011 | fyrz | 11 March 2015, 20:50:10 UTC | [RocksJava] Added LevelCompactionDynamicLevelBytes to Options Summary: Added LevelCompactionDynamicLevelBytes to Options. Test Plan: make clean jclean rocksdbjava jtest mvn -f rocksjni.pom package Reviewers: adamretter, ankgup87, yhchiang Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D34857 | 18 March 2015, 20:38:17 UTC |
| 230e687 | Venkatesh Radhakrishnan | 18 March 2015, 18:40:46 UTC | Fix TSAN failue in env_test Summary: Check for state of task before deleting it. Test Plan: Run env_test with TSAN Reviewers: igor, sdong Reviewed By: sdong Subscribers: meyering, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D35283 | 18 March 2015, 18:40:46 UTC |
| 155d468 | Islam AbdelRahman | 18 March 2015, 18:26:10 UTC | Using chrono as a fallback Summary: Right now if they system we are compiling on is not Linux and not Mac we will get a compilation error this diff use chrono as a fallback when we are compiling on something other than Linux/FreeBSD/Mac Test Plan: compile on CentOS/FreeBSD ./db_test (still running) Reviewers: igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D35277 | 18 March 2015, 18:26:10 UTC |
| 81345b9 | agiardullo | 24 February 2015, 01:49:23 UTC | Create an abstract interface for write batches Summary: WriteBatch and WriteBatchWithIndex now both inherit from a common abstract base class. This makes it easier to write code that is agnostic toward the implementation of the particular write batch. In particular, I plan on utilizing this abstraction to allow transactions to support using either implementation of a write batch. Test Plan: modified existing WriteBatchWithIndex tests to test new functions. Running all tests. Reviewers: igor, rven, yhchiang, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D34017 | 18 March 2015, 02:23:08 UTC |
| 46214df | Igor Canadi | 17 March 2015, 22:24:16 UTC | Only run db_test in Travis | 17 March 2015, 22:24:16 UTC |
| c88ff4c | Igor Canadi | 17 March 2015, 22:04:37 UTC | Deprecate removeScanCountLimit in NewLRUCache Summary: It is no longer used by the implementation, so we should also remove it from the public API. Test Plan: make check Reviewers: sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D34971 | 17 March 2015, 22:04:37 UTC |
| b4b69e4 | Igor Sugak | 17 March 2015, 21:08:00 UTC | rocksdb: switch to gtest Summary: Our existing test notation is very similar to what is used in gtest. It makes it easy to adopt what is different. In this diff I modify existing [[ https://code.google.com/p/googletest/wiki/Primer#Test_Fixtures:_Using_the_Same_Data_Configuration_for_Multiple_Te | test fixture ]] classes to inherit from `testing::Test`. Also for unit tests that use fixture class, `TEST` is replaced with `TEST_F` as required in gtest. There are several custom `main` functions in our existing tests. To make this transition easier, I modify all `main` functions to fallow gtest notation. But eventually we can remove them and use implementation of `main` that gtest provides. ```lang=bash % cat ~/transform #!/bin/sh files=$(git ls-files '*test\.cc') for file in $files do if grep -q "rocksdb::test::RunAllTests()" $file then if grep -Eq '^class \w+Test {' $file then perl -pi -e 's/^(class \w+Test) {/${1}: public testing::Test {/g' $file perl -pi -e 's/^(TEST)/${1}_F/g' $file fi perl -pi -e 's/(int main.*\{)/${1}::testing::InitGoogleTest(&argc, argv);/g' $file perl -pi -e 's/rocksdb::test::RunAllTests/RUN_ALL_TESTS/g' $file fi done % sh ~/transform % make format ``` Second iteration of this diff contains only scripted changes. Third iteration contains manual changes to fix last errors and make it compilable. Test Plan: Build and notice no errors. ```lang=bash % USE_CLANG=1 make check -j55 ``` Tests are still testing. Reviewers: meyering, sdong, rven, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D35157 | 17 March 2015, 21:08:00 UTC |
| 413e352 | Igor Canadi | 17 March 2015, 17:33:21 UTC | Merge pull request #540 from dalgaaf/wip-da-fix-elif Fix '#elif with no expression' | 17 March 2015, 17:33:21 UTC |
| 969aa80 | Danny Al-Gaaf | 17 March 2015, 11:04:05 UTC | util/xfunc.h: fix #elif check for NDEBUG Fix '#elif with no expression', add defined() to check. Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de> | 17 March 2015, 11:04:05 UTC |
| 87c7d49 | Danny Al-Gaaf | 17 March 2015, 11:03:11 UTC | util/env_posix.cc: fix #elif check for __MACH__ Fix '#elif with no expression' add defined() to check. Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de> | 17 March 2015, 11:03:11 UTC |
| c86e5d7 | Danny Al-Gaaf | 17 March 2015, 11:00:55 UTC | stack_trace.cc: fix #elif check for OS_MACOSX Fix '#elif with no expression' add defined() to check. Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de> | 17 March 2015, 11:00:55 UTC |
| 98c37fd | Venkatesh Radhakrishnan | 17 March 2015, 04:07:54 UTC | Remove unused parameter in CancelAllBackgroundWork Summary: Some suggestions for cleanup from Igor. Test Plan: Regression tests. Reviewers: igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D35169 | 17 March 2015, 04:07:54 UTC |
| 9fd6edf | Igor Sugak | 17 March 2015, 03:52:32 UTC | rocksdb: Replace ASSERT* with EXPECT* in functions that does not return void value Summary: gtest does not use exceptions to fail a unit test by design, and `ASSERT*`s are implemented using `return`. As a consequence we cannot use `ASSERT*` in a function that does not return `void` value ([[ https://code.google.com/p/googletest/wiki/AdvancedGuide#Assertion_Placement | 1]]), and have to fix our existing code. This diff does this in a generic way, with no manual changes. In order to detect all existing `ASSERT*` that are used in functions that doesn't return void value, I change the code to generate compile errors for such cases. In `util/testharness.h` I defined `EXPECT*` assertions, the same way as `ASSERT*`, and redefined `ASSERT*` to return `void`. Then executed: ```lang=bash % USE_CLANG=1 make all -j55 -k 2> build.log % perl -naF: -e 'print "-- -number=".$F[1]." ".$F[0]."\n" if /: error:/' \ build.log | xargs -L 1 perl -spi -e 's/ASSERT/EXPECT/g if $. == $number' % make format ``` After that I reverted back change to `ASSERT*` in `util/testharness.h`. But preserved introduced `EXPECT*`, which is the same as `ASSERT*`. This will be deleted once switched to gtest. This diff is independent and contains manual changes only in `util/testharness.h`. Test Plan: Make sure all tests are passing. ```lang=bash % USE_CLANG=1 make check ``` Reviewers: igor, lgalanis, sdong, yufei.zhu, rven, meyering Reviewed By: meyering Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D33333 | 17 March 2015, 03:52:32 UTC |
| d4d42c0 | Venkatesh Radhakrishnan | 17 March 2015, 02:51:25 UTC | Fixed clang build in env.h Summary: Mark function as override. Test Plan: USE_CLANG=1 make -j32 check Reviewers: sdong, yhchiang, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D35163 | 17 March 2015, 02:51:25 UTC |
| b2b3086 | Venkatesh Radhakrishnan | 17 March 2015, 01:49:14 UTC | Speed up rocksDB close call. Summary: On RocksDB, when there are multiple instances doing flushes/compactions in the background, the close call takes a long time because the flushes/compactions need to complete before the database can shut down. If another instance is using the background threads and the compaction for this instance is in the queue since it has been scheduled, we still cannot shutdown. We now remove the scheduled background tasks which have not yet started running, so that shutdown is speeded up. Test Plan: DB Test added. Reviewers: yhchiang, igor, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D33741 | 17 March 2015, 01:49:14 UTC |
| a7aba2e | Igor Sugak | 17 March 2015, 01:27:30 UTC | rocksdb: Add gtest Summary: Adds gtest fused source code into `third-party` directory. No manual changes. gtest latest released 1.7 has clang dev compilation errors. Trunk version requires only one disabled warning (-Wno-missing-field-initializers) Fused code is made as described here https://fburl.com/90806322 Details about why we need gtest source code instead of precompiled library https://fburl.com/90805763 Source used from http://googletest.googlecode.com/svn/trunk Test Plan: Build and notice no errors. Also check in logs that gtest-all.o being compiled gtest-all.o. ```lang=bash % USE_CLANG=1 make all ``` Reviewers: lgalanis, yufei.zhu, rven, sdong, igor, meyering Reviewed By: meyering Subscribers: meyering, yhchiang, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D33345 | 17 March 2015, 01:27:30 UTC |
| 9534434 | Igor Sugak | 17 March 2015, 01:08:59 UTC | rocksdb: Small refactoring before migrating to gtest Summary: These changes are necessary to make tests look more generic, and avoid feature conflicts with gtest. Test Plan: Make sure no build errors, and all test are passing. ``` % make check ``` Reviewers: igor, meyering Reviewed By: meyering Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D35145 | 17 March 2015, 01:08:59 UTC |
| bd4963e | Igor Sugak | 17 March 2015, 00:08:05 UTC | rocksdb: update reference to the latest version of clang dev in fbcode Summary: Updated reference to the latest clang dev in fbcode. Since RocksDB already uses recent version of clang dev no code changes need. Test Plan: Make sure can be build with clang. ```lang=bash % USE_CLANG=1 make all ``` Reviewers: meyering, sdong, rven, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D35121 | 17 March 2015, 00:08:05 UTC |
| 56337fa | Mark Callaghan | 16 March 2015, 18:50:23 UTC | Fix compaction IO stats to handle large file counts Summary: The output did not have space for 6-digit file counts or for 3-digit counts of files being compacted. This adds space for that while preserving existing alignment. See https://gist.github.com/mdcallag/0a61c6a18dd467224c11 Task ID: # Blame Rev: Test Plan: run db_bench, look at output Revert Plan: Database Impact: Memcache Impact: Other Notes: EImportant: - begin *PUBLIC* platform impact section - Bugzilla: # - end platform impact - Reviewers: igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D35091 | 16 March 2015, 18:50:23 UTC |
| eafa1bf | Adam Retter | 16 March 2015, 10:38:15 UTC | Merge pull request #529 from fyrz/RocksJava-Logger [RockJava] Custom logger addition | 16 March 2015, 10:38:15 UTC |
| ac03c4d | Yueh-Hsuan Chiang | 16 March 2015, 04:32:06 UTC | Merge pull request #522 from fyrz/RocksJava-Optimize-Filters-For-Hits [RocksJava] OptimizeFiltersForHits | 16 March 2015, 04:32:06 UTC |
| c6967a1 | Igor Canadi | 14 March 2015, 22:12:41 UTC | Make RecordIn/RecordOut human readable Summary: I had hard time understanding these big numbers. Here's how the output looks like now: https://gist.github.com/igorcanadi/4c39c17685049584a992 Test Plan: db_bench Reviewers: sdong, MarkCallaghan Reviewed By: MarkCallaghan Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D35073 | 14 March 2015, 22:12:41 UTC |
| c8da670 | Mark Callaghan | 14 March 2015, 22:01:43 UTC | Stop printing per-level stall times. Summary: Per-level stall times are the suggested stall time, not the actual stall time so this change stops printing them both in the per-level output lines and in the summary. Also changed output for total stall time to include units in all cases. The new output looks like: Level Files Size(MB) Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) Comp(cnt) Avg(sec) Stall(cnt) RecordIn RecordDrop ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 4/1 7 0.8 0.0 0.0 0.0 0.6 0.6 0.0 0.0 0.0 12.9 50 352 0.141 882 0 0 L1 5/0 9 0.9 0.0 0.0 0.0 0.0 0.0 0.6 0.0 0.0 0.0 0 0 0.000 0 0 0 L2 54/0 99 1.0 0.0 0.0 0.0 0.0 0.0 0.6 0.0 0.0 0.0 0 0 0.000 0 0 0 L3 289/0 527 0.5 0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.0 0.0 0 0 0.000 0 0 0 Sum 352/1 642 0.0 0.0 0.0 0.0 0.6 0.6 1.7 1.0 0.0 12.9 50 352 0.141 882 0 0 Int 0/0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 15.5 0 3 0.118 7 0 0 Flush(GB): accumulative 0.627, interval 0.005 Stalls(count): 0 level0_slowdown, 0 level0_numfiles, 882 memtable_compaction, 0 leveln_slowdown_soft, 0 leveln_slowdown_hard Task ID: #6493861 Blame Rev: Test Plan: run db_bench, look at output Revert Plan: Database Impact: Memcache Impact: Other Notes: EImportant: - begin *PUBLIC* platform impact section - Bugzilla: # - end platform impact - Reviewers: igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D35085 | 14 March 2015, 22:01:43 UTC |
| 04778a9 | fyrz | 28 February 2015, 21:38:58 UTC | [RocksJava] OptimizeFiltersForHits Summary: Added optimize_filters_for_hits option. Test Plan: make clean jclean rocksdbjava jtest mvn -f rocksjni.pom package Reviewers: adamretter, yhchiang, ankgup87 Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D35013 | 14 March 2015, 20:00:15 UTC |
| 57f2a00 | fyrz | 10 March 2015, 22:16:21 UTC | RocksJava - JNI Logger callback Summary: Within this commit a new AbstractLogger was introduced which allows to handle log messages at an application level. Log messages are passed up to Java using a JNI callback. This allows a Java-Developer to use common Java APIs for log messages e.g. SLF4J, LOG4J, etc. Within this commit no new dependencies were introduced, which keeps the RocksDB API clean and doesn`t force a developer to use a predefined high-level Java API. Another feature is to dynamically set a custom loggers verbosity at runtime using its public method `setInfoLogLevel` and to retrieve the currently active level using the `infoLogLevel` method. Test Plan: make clean jclean rocksdbjava jtest mvn -f rocksjni.pom package Reviewers: adamretter, ankgup87, yhchiang Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D34755 | 14 March 2015, 19:57:18 UTC |
| 814627a | fyrz | 10 March 2015, 21:57:15 UTC | [RocksJava] Change log level at runtime in custom logger This commit introduces the possibility to change the log level of a used custom logger at runtime. | 14 March 2015, 19:57:18 UTC |
| a3bd414 | fyrz | 10 February 2015, 20:59:40 UTC | [RocksJava] Logging JNI callback Within this commit a new AbstractLogger was introduced which pushes info log messages all the way up to Java. | 14 March 2015, 19:57:18 UTC |
| 58878f1 | Mark Callaghan | 14 March 2015, 15:36:57 UTC | Switch to use_existing_db=1 for updaterandom and mergerandom Summary: Without this change about half of the updaterandom reads and merge puts will be for keys that don't exist. I think it is better for these tests to start with a full database and use fillseq to fill it. Task ID: # Blame Rev: Test Plan: Revert Plan: Database Impact: Memcache Impact: Other Notes: EImportant: - begin *PUBLIC* platform impact section - Bugzilla: # - end platform impact - Reviewers: igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D35043 | 14 March 2015, 15:36:57 UTC |
| 1213413 | Yueh-Hsuan Chiang | 14 March 2015, 15:21:53 UTC | Fixed the unit-test issue in PreShutdownCompactionMiddle Summary: Fixed the unit-test issue in PreShutdownCompactionMiddle Test Plan: export ROCKSDB_TESTS=PreShutdownCompactionMiddle Reviewers: rven, sdong, igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D35061 | 14 March 2015, 15:25:27 UTC |
| fd1b3f3 | Yueh-Hsuan Chiang | 14 March 2015, 15:00:06 UTC | Fix the issue in PreShutdownMultipleCompaction Summary: Fix the issue in PreShutdownMultipleCompaction Test Plan: export ROCKSDB_TESTS=PreShutdownMultipleCompaction ./db_test Reviewers: rven, sdong, igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D35055 | 14 March 2015, 15:03:02 UTC |
| 56c4a9c | Yueh-Hsuan Chiang | 14 March 2015, 01:09:01 UTC | Fix compile warning in thread_status_util.h on Mac Summary: Fix compile warning in thread_status_util.h on Mac Test Plan: make dbg -j32 | 14 March 2015, 01:09:01 UTC |
| 417367c | Igor Canadi | 13 March 2015, 23:41:00 UTC | Fix SIGSEGV when not using cache | 13 March 2015, 23:41:00 UTC |
| e25ff03 | Venkatesh Radhakrishnan | 13 March 2015, 21:51:40 UTC | Prevent slowdowns and stalls in PreShutdown tests Summary: The preshutdown tests check for stopped compactions/flushes. Removing stalls on the write path. Test Plan: DBTests.PreShutdown* Reviewers: yhchiang, sdong, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D35037 | 13 March 2015, 21:51:40 UTC |
| f690712 | Igor Canadi | 13 March 2015, 21:45:15 UTC | Speed up db_bench shutdown Summary: See t6489044 Test Plan: compiles Reviewers: MarkCallaghan Reviewed By: MarkCallaghan Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D34977 | 13 March 2015, 21:45:15 UTC |
| c1b3cde | Yueh-Hsuan Chiang | 13 March 2015, 20:11:08 UTC | Improve the robustness of ThreadStatusSingleCompaction Summary: Improve the robustness of ThreadStatusSingleCompaction by ensuring the number of files flushed in the test. Test Plan: export ROCKSDB_TESTS=ThreadStatus ./db_test Reviewers: sdong, igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D35019 | 13 March 2015, 20:16:53 UTC |
| 8c12426 | Yueh-Hsuan Chiang | 13 March 2015, 18:59:00 UTC | Fix the deadlock issue in ThreadStatusSingleCompaction. Summary: Fix the deadlock issue in ThreadStatusSingleCompaction. In the previous version of ThreadStatusSingleCompaction, the compaction thread will wait for a SYNC_POINT while its db_mutex is held. However, if the test hasn't finished its Put cycle while a compaction is running, a deadlock will happen in the test. Test Plan: export ROCKSDB_TESTS=ThreadStatus ./db_test Reviewers: sdong, igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D35001 | 13 March 2015, 19:53:00 UTC |
| b16ead5 | sdong | 13 March 2015, 18:18:35 UTC | DBTest.DynamicLevelCompressionPerLevel should not run without snappy support Summary: The test depends on snappy to be used. Skip the test if it is not supported. Test Plan: Run the test Reviewers: meyering, igor Reviewed By: igor Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D34995 | 13 March 2015, 18:26:17 UTC |
| a5e60ba | Yueh-Hsuan Chiang | 13 March 2015, 18:18:42 UTC | Fix a typo / test failure in ThreadStatusSingleCompaction Summary: Fix a typo / test failure in ThreadStatusSingleCompaction Test Plan: export ROCKSDB_TESTS=ThreadStatus ./db_test | 13 March 2015, 18:20:17 UTC |
| cb2c918 | Igor Canadi | 13 March 2015, 18:08:50 UTC | Don't run some tests is snappy is not present Summary: Currently, we have `ifdef SNAPPY` around bunch of db_test code. Some tests that don't even use compression are also blocked when running system doesn't have snappy. This also causes hard-to-catch bugs, like D34983. We should dynamically figure out if compression is supported or not. Test Plan: compiles Reviewers: sdong, meyering Reviewed By: meyering Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D34989 | 13 March 2015, 18:08:50 UTC |
| c594b0e | Yueh-Hsuan Chiang | 13 March 2015, 17:45:40 UTC | Allow GetThreadList() to report operation stage. Summary: Allow GetThreadList() to report operation stage. Test Plan: ./thread_list_test ./db_bench --benchmarks=fillrandom --num=100000 --threads=40 \ --max_background_compactions=10 --max_background_flushes=3 \ --thread_status_per_interval=1000 --key_size=16 --value_size=1000 \ --num_column_families=10 export ROCKSDB_TESTS=ThreadStatus ./db_test Sample output ThreadID ThreadType cfName Operation OP_StartTime ElapsedTime Stage State 140116265861184 Low Pri 140116270055488 Low Pri 140116274249792 High Pri column_family_name_000005 Flush 2015/03/10-14:58:11 0 us FlushJob::WriteLevel0Table 140116400078912 Low Pri column_family_name_000004 Compaction 2015/03/10-14:58:11 0 us CompactionJob::FinishCompactionOutputFile 140116358135872 Low Pri column_family_name_000006 Compaction 2015/03/10-14:58:10 1 us CompactionJob::FinishCompactionOutputFile 140116341358656 Low Pri 140116295221312 High Pri default Flush 2015/03/10-14:58:11 0 us FlushJob::WriteLevel0Table 140116324581440 Low Pri column_family_name_000009 Compaction 2015/03/10-14:58:11 0 us CompactionJob::ProcessKeyValueCompaction 140116278444096 Low Pri 140116299415616 Low Pri column_family_name_000008 Compaction 2015/03/10-14:58:11 0 us CompactionJob::FinishCompactionOutputFile 140116291027008 High Pri column_family_name_000001 Flush 2015/03/10-14:58:11 0 us FlushJob::WriteLevel0Table 140116286832704 Low Pri column_family_name_000002 Compaction 2015/03/10-14:58:11 0 us CompactionJob::FinishCompactionOutputFile 140116282638400 Low Pri Reviewers: rven, igor, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D34683 | 13 March 2015, 17:45:40 UTC |
| 2623b2c | Igor Canadi | 13 March 2015, 17:29:32 UTC | Include chrono | 13 March 2015, 17:29:32 UTC |
| 52d8347 | Igor Canadi | 13 March 2015, 17:15:54 UTC | EventLogger Summary: Here's my proposal for making our LOGs easier to read by machines. The idea is to dump all events as JSON objects. JSON is easy to read by humans, but more importantly, it's easy to read by machines. That way, we can parse this, load into SQLite/mongo and then query or visualize. I started with table_create and table_delete events, but if everybody agrees, I'll continue by adding more events (flush/compaction/etc etc) Test Plan: Ran db_bench. Observed: 2015/01/15-14:13:25.788019 1105ef000 EVENT_LOG_v1 {"time_micros": 1421360005788015, "event": "table_file_creation", "file_number": 12, "file_size": 1909699} 2015/01/15-14:13:25.956500 110740000 EVENT_LOG_v1 {"time_micros": 1421360005956498, "event": "table_file_deletion", "file_number": 12} Reviewers: yhchiang, rven, dhruba, MarkCallaghan, lgalanis, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D31647 | 13 March 2015, 17:15:54 UTC |
