https://github.com/facebook/rocksdb
- HEAD
- refs/heads/2.2.fb.branch
- refs/heads/2.3.fb.branch
- refs/heads/2.4.fb.branch
- refs/heads/2.5.fb.branch
- refs/heads/2.6.fb.branch
- refs/heads/2.7
- refs/heads/2.7.fb.branch
- refs/heads/2.8.1.fb
- refs/heads/2.8.fb
- refs/heads/2.8.fb.trunk
- refs/heads/3.0.fb
- refs/heads/3.0.fb.branch
- refs/heads/3.1.fb
- refs/heads/3.10.fb
- refs/heads/3.11.fb
- refs/heads/3.12.fb
- refs/heads/3.13.fb
- refs/heads/3.2.fb
- refs/heads/3.3.fb
- refs/heads/3.4.fb
- refs/heads/3.5.fb
- refs/heads/3.6.fb
- refs/heads/3.7.fb
- refs/heads/3.8.fb
- refs/heads/3.9.fb
- refs/heads/4.0.fb
- refs/heads/4.1.fb
- refs/heads/4.10.fb
- refs/heads/4.11.fb
- refs/heads/4.12.fb
- refs/heads/4.13.fb
- refs/heads/4.2.fb
- refs/heads/4.3.fb
- refs/heads/4.4.fb
- refs/heads/4.5.fb
- refs/heads/4.6.fb
- refs/heads/4.7.fb
- refs/heads/4.8.fb
- refs/heads/4.9.fb
- refs/heads/5.0.fb
- refs/heads/5.1.fb
- refs/heads/5.10.fb
- refs/heads/5.11.fb
- refs/heads/5.12.fb
- refs/heads/5.13.fb
- refs/heads/5.13.fb.myrocks
- refs/heads/5.14.fb
- refs/heads/5.14.fb.myrocks
- refs/heads/5.15.fb
- refs/heads/5.16.fb
- refs/heads/5.17.fb
- refs/heads/5.17.fb.myrocks
- refs/heads/5.18.fb
- refs/heads/5.2.fb
- refs/heads/5.3.fb
- refs/heads/5.4.fb
- refs/heads/5.5.fb
- refs/heads/5.6.fb
- refs/heads/5.7.fb
- refs/heads/5.7.fb.myrocks
- refs/heads/5.8.3
- refs/heads/5.8.fb
- refs/heads/5.9.fb
- refs/heads/5.9.fb.myrocks
- refs/heads/6.0.fb
- refs/heads/6.0.fb.myrocks
- refs/heads/6.1.fb
- refs/heads/6.1.fb.myrocks
- refs/heads/6.1.fb.prod201905
- refs/heads/6.10.fb
- refs/heads/6.11.fb
- refs/heads/6.12.fb
- refs/heads/6.13.fb
- refs/heads/6.13.fb.laser
- refs/heads/6.14.fb
- refs/heads/6.14.fb.laser
- refs/heads/6.15.fb
- refs/heads/6.16.fb
- refs/heads/6.17.fb
- refs/heads/6.17.fb.laser
- refs/heads/6.18.fb
- refs/heads/6.19.fb
- refs/heads/6.2.fb
- refs/heads/6.20.fb
- refs/heads/6.21.fb
- refs/heads/6.22-history.md-fixup
- refs/heads/6.22.fb
- refs/heads/6.23.fb
- refs/heads/6.24.fb
- refs/heads/6.25.fb
- refs/heads/6.26.fb
- refs/heads/6.27.fb
- refs/heads/6.28.fb
- refs/heads/6.29.fb
- refs/heads/6.3.fb
- refs/heads/6.3.fb.myrocks
- refs/heads/6.3.fb.myrocks2
- refs/heads/6.3fb
- refs/heads/6.4.fb
- refs/heads/6.5.fb
- refs/heads/6.6.fb
- refs/heads/6.7.fb
- refs/heads/6.8.fb
- refs/heads/6.9.fb
- refs/heads/7.0.fb
- refs/heads/7.1.fb
- refs/heads/7.10.fb
- refs/heads/7.2.fb
- refs/heads/7.3.fb
- refs/heads/7.4.fb
- refs/heads/7.5.fb
- refs/heads/7.6.fb
- refs/heads/7.7.fb
- refs/heads/7.8.fb
- refs/heads/7.9.fb
- refs/heads/8.0.fb
- refs/heads/8.1.fb
- refs/heads/8.10.fb
- refs/heads/8.11.2_zippydb
- refs/heads/8.11.fb
- refs/heads/8.11.fb_zippydb
- refs/heads/8.2.fb
- refs/heads/8.3.fb
- refs/heads/8.4.fb
- refs/heads/8.5.fb
- refs/heads/8.6.fb
- refs/heads/8.7.fb
- refs/heads/8.8.fb
- refs/heads/8.9.fb
- refs/heads/9.0.fb
- refs/heads/9.1.fb
- refs/heads/adaptive
- refs/heads/ajkr-patch-1
- refs/heads/ajkr-patch-2
- refs/heads/blob_shadow
- refs/heads/bottom-pri-level
- refs/heads/bugfix-build-detect
- refs/heads/checksum_readahead_mmap_fix
- refs/heads/draft-myrocks-and-fbcode-8.0.fb
- refs/heads/feature/debug-rocksdbjavastatic
- refs/heads/feature/travis-arm64
- refs/heads/fix-release-notes
- refs/heads/fix-win2022-build
- refs/heads/fix-write-batch-comment
- refs/heads/format_compatible_4
- refs/heads/getmergeops
- refs/heads/gh-pages-old
- refs/heads/history-update
- refs/heads/hotfix/lambda-capture
- refs/heads/improve-support
- refs/heads/jijiew-patch-1
- refs/heads/katherinez-patch-1
- refs/heads/katherinez-patch-2
- refs/heads/main
- refs/heads/master
- refs/heads/mdcallag_benchmark_oct22
- refs/heads/nvm_cache_proto
- refs/heads/pr-sanity-check-as-GHAction
- refs/heads/pr/11267
- refs/heads/pr/6062
- refs/heads/ramvadiv-patch-1
- refs/heads/release_fix
- refs/heads/revert-10606-7.6.1
- refs/heads/ribbon_bloom_hybrid
- refs/heads/scaffold
- refs/heads/siying-patch-1
- refs/heads/siying-patch-10
- refs/heads/siying-patch-2
- refs/heads/siying-patch-3
- refs/heads/siying-patch-4
- refs/heads/siying-patch-5
- refs/heads/siying-patch-6
- refs/heads/siying-patch-7
- refs/heads/siying-patch-8
- refs/heads/skip_memtable_flush
- refs/heads/testing_ppc_build
- refs/heads/tests
- refs/heads/unschedule_issue_test_base
- refs/heads/unused-var
- refs/heads/v6.6.4
- refs/heads/xxhash_merge_base
- refs/heads/yiwu_stackable
- refs/heads/yuslepukhin
- refs/remotes/origin/5.13.fb
- refs/tags/2.5.fb
- refs/tags/2.6.fb
- refs/tags/3.0.fb
- refs/tags/do-not-use-me2
- refs/tags/rocksdb-3.1
- refs/tags/rocksdb-3.10.2
- refs/tags/rocksdb-3.11
- refs/tags/rocksdb-3.11.1
- refs/tags/rocksdb-3.11.2
- refs/tags/rocksdb-3.2
- refs/tags/rocksdb-3.3
- refs/tags/rocksdb-3.4
- refs/tags/rocksdb-3.5
- refs/tags/rocksdb-3.5.1
- refs/tags/rocksdb-3.6.1
- refs/tags/rocksdb-3.6.2
- refs/tags/rocksdb-3.7
- refs/tags/rocksdb-3.8
- refs/tags/rocksdb-3.9
- refs/tags/rocksdb-3.9.1
- refs/tags/rocksdb-4.1
- refs/tags/rocksdb-5.10.2
- refs/tags/rocksdb-5.10.3
- refs/tags/rocksdb-5.10.4
- refs/tags/rocksdb-5.11.2
- refs/tags/rocksdb-5.11.3
- refs/tags/rocksdb-5.14.3
- refs/tags/rocksdb-5.2.1
- refs/tags/rocksdb-5.3.3
- refs/tags/rocksdb-5.3.4
- refs/tags/rocksdb-5.3.5
- refs/tags/rocksdb-5.3.6
- refs/tags/rocksdb-5.4.10
- refs/tags/rocksdb-5.4.5
- refs/tags/rocksdb-5.4.6
- refs/tags/rocksdb-5.5.2
- refs/tags/rocksdb-5.5.3
- refs/tags/rocksdb-5.5.4
- refs/tags/rocksdb-5.5.5
- refs/tags/rocksdb-5.5.6
- refs/tags/rocksdb-5.6.1
- refs/tags/rocksdb-5.6.2
- refs/tags/rocksdb-5.7.1
- refs/tags/rocksdb-5.7.2
- refs/tags/rocksdb-5.7.3
- refs/tags/rocksdb-5.7.5
- refs/tags/rocksdb-5.8.6
- refs/tags/rocksdb-5.8.7
- refs/tags/rocksdb-5.8.8
- refs/tags/rocksdb-5.9.2
- refs/tags/v4.0
- refs/tags/v4.1
- refs/tags/v5.10.2
- refs/tags/v5.10.3
- refs/tags/v5.10.4
- refs/tags/v5.11.2
- refs/tags/v5.11.3
- refs/tags/v5.13.3
- refs/tags/v5.14.3
- refs/tags/v5.15.10
- refs/tags/v5.18.3
- refs/tags/v5.2.1
- refs/tags/v5.3.3
- refs/tags/v5.3.4
- refs/tags/v5.3.5
- refs/tags/v5.3.6
- refs/tags/v5.4.10
- refs/tags/v5.4.5
- refs/tags/v5.4.6
- refs/tags/v5.5.2
- refs/tags/v5.5.3
- refs/tags/v5.5.4
- refs/tags/v5.5.5
- refs/tags/v5.5.6
- refs/tags/v5.6.1
- refs/tags/v5.6.2
- refs/tags/v5.7.1
- refs/tags/v5.7.2
- refs/tags/v5.7.3
- refs/tags/v5.7.5
- refs/tags/v5.8.6
- refs/tags/v5.8.7
- refs/tags/v5.8.8
- refs/tags/v5.9.2
- refs/tags/v6.0.1
- refs/tags/v6.0.2
- refs/tags/v6.1.1
- refs/tags/v6.1.2
- refs/tags/v6.10.1
- refs/tags/v6.10.2
- refs/tags/v6.11.4
- refs/tags/v6.11.6
- refs/tags/v6.12.6
- refs/tags/v6.12.7
- refs/tags/v6.13.2
- refs/tags/v6.13.3
- refs/tags/v6.14.5
- refs/tags/v6.14.6
- refs/tags/v6.15.4
- refs/tags/v6.15.5
- refs/tags/v6.16.3
- refs/tags/v6.16.4
- refs/tags/v6.17.3
- refs/tags/v6.2.2
- refs/tags/v6.2.4
- refs/tags/v6.20.3
- refs/tags/v6.22.1
- refs/tags/v6.25.3
- refs/tags/v6.26.1
- refs/tags/v6.28.2
- refs/tags/v6.29.3
- refs/tags/v6.29.4
- refs/tags/v6.29.5
- refs/tags/v6.3.6
- refs/tags/v6.4.6
- refs/tags/v6.5.2
- refs/tags/v6.5.3
- refs/tags/v6.6.3
- refs/tags/v6.6.4
- refs/tags/v6.7.3
- refs/tags/v6.8.1
- refs/tags/v7.0.1
- refs/tags/v7.0.2
- refs/tags/v7.0.4
- refs/tags/v7.2.0
- refs/tags/v7.2.2
- refs/tags/v7.5.3
- refs/tags/v7.7.2
- refs/tags/v7.9.2
- refs/tags/v8.0.0
- refs/tags/v8.11.4
- refs/tags/v8.3.2
- refs/tags/v8.3.3
- refs/tags/v8.4.4
- refs/tags/v8.5.3
- refs/tags/v8.6.7
- refs/tags/v8.7.3
- v9.0.0
- v8.9.1
- v8.8.1
- v8.5.4
- v8.11.3
- v8.10.2
- v8.10.0
- v8.1.1
- v7.8.3
- v7.7.8
- v7.7.3
- v7.6.0
- v7.4.5
- v7.4.4
- v7.4.3
- v7.3.1
- v7.10.2
- v7.1.2
- v7.1.1
- v7.0.3
- v6.27.3
- v6.26.0
- v6.25.1
- v6.24.2
- v6.23.3
- v6.23.2
- v6.19.3
- v6.15.2
- v5.8
- v5.5.1
- v5.4.7
- v5.18.4
- v5.17.2
- v5.16.6
- v5.14.2
- v5.13.4
- v5.13.2
- v5.13.1
- v5.12.5
- v5.12.4
- v5.12.3
- v5.12.2
- v5.1.4
- v5.1.3
- v5.1.2
- v5.0.2
- v5.0.1
- v4.9
- v4.8
- v4.6.1
- v4.5.1
- v4.4.1
- v4.4
- v4.3.1
- v4.3
- v4.2
- v4.13.5
- v4.13
- v4.11.2
- v3.9
- v3.8
- v3.7
- v3.6.1
- v3.5
- v3.4
- v3.3
- v3.2
- v3.13.1
- v3.13
- v3.12.1
- v3.12
- v3.11
- v3.10
- v3.1
- v3.0
- v2.8
- v2.7
- v2.6
- v2.5
- v2.4
- v2.3
- v2.2
- v2.1
- v2.0
- v1.5.9.1
- v1.5.8.2
- v1.5.8.1
- v1.5.8
- v1.5.7
- rocksdb-5.8
- rocksdb-5.4.7
- rocksdb-5.1.4
- rocksdb-5.1.3
- rocksdb-5.1.2
- rocksdb-5.0.2
- rocksdb-5.0.1
- rocksdb-4.9
- rocksdb-4.8
- rocksdb-4.6.1
- rocksdb-4.5.1
- rocksdb-4.4.1
- rocksdb-4.4
- rocksdb-4.3.1
- rocksdb-4.3
- rocksdb-4.2
- rocksdb-4.13.5
- rocksdb-4.13
- rocksdb-4.11.2
- rocksdb-3.10.1
- blob_st_lvl-pre
- 2.8.fb
- 2.7.fb
- 2.4.fb
- 2.3.fb
- 2.2.fb
- 2.1.fb
- 2.0.fb
- 1.5.9.fb
- 1.5.9.2.fb
- 1.5.9.1.fb
- 1.5.8.fb
- 1.5.8.2.fb
- 1.5.8.1.fb
- 1.5.7.fb
Take a new snapshot of a software origin
If the archived software origin currently browsed is not synchronized with its upstream version (for instance when new commits have been issued), you can explicitly request Software Heritage to take a new snapshot of it.
Use the form below to proceed. Once a request has been submitted and accepted, it will be processed as soon as possible. You can then check its processing state by visiting this dedicated page.Processing "take a new snapshot" request ...
Permalinks
To reference or cite the objects present in the Software Heritage archive, permalinks based on SoftWare Hash IDentifiers (SWHIDs) must be used.
Select below a type of object currently browsed in order to display its associated SWHID and permalink.
Revision | Author | Date | Message | Commit Date |
---|---|---|---|---|
2b90ba0 | sdong | 05 November 2019, 17:25:25 UTC | Add one more #include<functional> | 05 November 2019, 17:29:21 UTC |
acb6d82 | sdong | 31 October 2019, 21:42:34 UTC | Disable warning as error | 31 October 2019, 21:44:06 UTC |
ab4f72f | sdong | 31 October 2019, 21:17:26 UTC | Add some include<functional> | 31 October 2019, 21:44:05 UTC |
57db794 | sdong | 31 October 2019, 21:10:18 UTC | [FB Internal] Point to the latest tool chain. | 31 October 2019, 21:44:05 UTC |
15d9f3a | sdong | 18 July 2017, 05:17:47 UTC | [fb only] revert unintended change of USE_SSE The previuos change that use gcc-5 set USE_SSE to wrong flag by mistake. Fix it. | 18 July 2017, 05:20:08 UTC |
61cac5a | sdong | 17 July 2017, 22:39:37 UTC | [FB Only] use gcc-5 | 18 July 2017, 04:26:30 UTC |
354d692 | Yueh-Hsuan Chiang | 13 October 2015, 18:06:48 UTC | Merge pull request #764 from dmittendorf/fix-java-static-packaging Fix crossbuild jar packaging Conflicts: Makefile java/crossbuild/build-linux-centos.sh | 13 October 2015, 18:21:39 UTC |
e44957c | Yueh-Hsuan Chiang | 10 October 2015, 03:03:39 UTC | Merge pull request #759 from jwlent55/statically-load-compression-libraries Ensure that the compression libraries are statically linked into dyna… | 10 October 2015, 03:08:16 UTC |
3e0700e | sdong | 14 September 2015, 21:01:13 UTC | DBImpl::FindObsoleteFiles() shouldn't release mutex between getting min_pending_output and scanning files Summary: Releasing mutex between getting min_pending_output and scanning files may cause min_pending_output to be max but some non-final files are found in file scanning, ending up with deleting wrong files. As a recent regression, mutex can be released while waiting for log sync. We move it to after file scanning. Test Plan: Run all existing tests. Don't think it is easy to write a unit test. Maybe we should find a way to assert lock not released so that we can have some test verification for similar cases. Reviewers: igor, anthony, IslamAbdelRahman, kradhakrishnan, yhchiang, kolmike, rven Reviewed By: rven Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D46899 | 24 September 2015, 18:39:15 UTC |
0ee2ee8 | Yueh-Hsuan Chiang | 15 September 2015, 17:57:31 UTC | Fixed a compile warning in Java Summary: Fixed the following compile warning in java: ==> linux64: java/rocksjni/loggerjnicallback.cc: In constructor ‘rocksdb::LoggerJniCallback::LoggerJniCallback(JNIEnv*, jobject)’: ==> linux64: java/rocksjni/loggerjnicallback.cc:19:14: warning: unused variable ‘rs’ [-Wunused-variable] ==> linux64: const jint rs = env->GetJavaVM(&m_jvm); ==> linux64: ^ Test Plan: make rocksdbjavastaticrelease Reviewers: igor, sdong, anthony, IslamAbdelRahman Subscribers: | 15 September 2015, 17:57:31 UTC |
d66a90f | agiardullo | 03 September 2015, 00:52:52 UTC | Disable some asserts in 3.13 tests Summary: tp2_build fails for rocksdb due to test failures. I can only occasionally repro these test failures when running the tests in parallel. This diff disables the failing asserts in 3.13 branch only to unblock search team. Will follow up on fixing these tests in master Test Plan: test is test Reviewers: sdong, igor, kradhakrishnan, yhchiang Reviewed By: yhchiang Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D46083 | 03 September 2015, 02:21:21 UTC |
7e10e1e | Yueh-Hsuan Chiang | 01 September 2015, 18:07:43 UTC | Fixed the build issue of rocksdbjavastaticrelease Summary: This patch fixed couple build issues of rocksdbjavastaticrelease. Test Plan: make rocksdbjavastaticrelease Reviewers: sdong, anthony, IslamAbdelRahman, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D45915 | 01 September 2015, 18:17:46 UTC |
6162170 | Yueh-Hsuan Chiang | 01 September 2015, 01:36:17 UTC | Fixed compile warning in rocksdbjava Summary: Fixed the following compile warning in rocksdbjava: java/rocksjni/comparatorjnicallback.cc:20:14: warning: unused variable ‘rs’ [-Wunused-variable] const jint rs = env->GetJavaVM(&m_jvm); ^ java/rocksjni/comparatorjnicallback.cc: In member function ‘JNIEnv* rocksdb::BaseComparatorJniCallback::getJniEnv() const’: java/rocksjni/comparatorjnicallback.cc:45:8: warning: unused variable ‘rs’ [-Wunused-variable] jint rs = m_jvm->AttachCurrentThread(reinterpret_cast<void **>(&env), NULL); ^ java/rocksjni/loggerjnicallback.cc: In constructor ‘rocksdb::LoggerJniCallback::LoggerJniCallback(JNIEnv*, jobject)’: java/rocksjni/loggerjnicallback.cc:19:14: warning: unused variable ‘rs’ [-Wunused-variable] const jint rs = env->GetJavaVM(&m_jvm); ^ java/rocksjni/loggerjnicallback.cc: In member function ‘JNIEnv* rocksdb::LoggerJniCallback::getJniEnv() const’: java/rocksjni/loggerjnicallback.cc:33:8: warning: unused variable ‘rs’ [-Wunused-variable] jint rs = m_jvm->AttachCurrentThread(reinterpret_cast<void **>(&env), NULL); ^ Test Plan: make rocksdbjava -j32 Reviewers: sdong, anthony, IslamAbdelRahman, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D45891 | 01 September 2015, 18:17:12 UTC |
f5561ed | Yueh-Hsuan Chiang | 01 September 2015, 01:35:12 UTC | Fixed a compile warning in linux32 environment. Summary: Fixed the following compile warning in linux32 environment. ==> linux32: util/sst_dump_tool.cc: In member function ‘int rocksdb::SstFileReader::ShowAllCompressionSizes(size_t)’: ==> linux32: util/sst_dump_tool.cc:167:50: warning: format ‘%lu’ expects argument of type ‘long unsigned int’, but argument 3 has type ‘size_t {aka unsigned int}’ [-Wformat=] ==> linux32: fprintf(stdout, "Block Size: %lu\n", block_size); Test Plan: make sst_dump Reviewers: anthony, IslamAbdelRahman, sdong, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D45885 | 01 September 2015, 18:13:09 UTC |
35b43ee | sdong | 31 August 2015, 18:23:39 UTC | Bump up version to RocksDB 3.13.1 | 31 August 2015, 18:23:39 UTC |
297241e | Andres Noetzli | 29 August 2015, 01:06:32 UTC | Fix deadlock in WAL sync Summary: MarkLogsSynced() was doing `logs_.erase(it++);`. The standard is saying: ``` all iterators and references are invalidated, unless the erased members are at an end (front or back) of the deque (in which case only iterators and references to the erased members are invalidated) ``` Because `it` is an iterator to the first element of the container, it is invalidated, only one iteration is executed and `log.getting_synced = false;` is not being done, so `while (logs_.front().getting_synced)` in `WriteImpl()` is not terminating. Test Plan: make db_bench && ./db_bench --benchmarks=fillsync Reviewers: igor, rven, IslamAbdelRahman, anthony, kradhakrishnan, yhchiang, sdong, tnovak Reviewed By: tnovak Subscribers: kolmike, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D45807 | 29 August 2015, 01:07:47 UTC |
254c4fb | sdong | 06 August 2015, 18:11:36 UTC | In HISTORY.md Switch unreleased notes to 3.13 Summary: Prepare release notes for 3.13. Test Plan: Text only Reviewers: yhchiang, IslamAbdelRahman, rven, anthony Reviewed By: anthony Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D43665 | 06 August 2015, 21:02:44 UTC |
40f893f | Islam AbdelRahman | 06 August 2015, 17:56:00 UTC | Fix delete_scheduler_test valgrind error Summary: Use shared_ptr instead of deleting in destructor Test Plan: DISABLE_JEMALLOC=1 make delete_scheduler_test -j64 && valgrind --error-exitcode=2 --leak-check=full ./delete_scheduler_test Reviewers: yhchiang, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D43659 | 06 August 2015, 17:56:00 UTC |
6a4aaad | sdong | 06 August 2015, 03:51:27 UTC | Avoid type unique_ptr in LogWriterNumber::writer for Windows build break Summary: Visual Studio complains about deque<LogWriterNumber> because LogWriterNumber is non-copyable for its unique_ptr member writer. Move away from it, and do explit free. It is less safe but I can't think of a better way to unblock it. Test Plan: valgrind check test Reviewers: anthony, IslamAbdelRahman, kolmike, rven, yhchiang Reviewed By: yhchiang Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D43647 | 06 August 2015, 17:52:41 UTC |
d7314ba | Andres Noetzli | 06 August 2015, 17:43:28 UTC | Fixing endless loop if seeking to end of key with seq num 0 Summary: When seeking to the last occurrence of a key with sequence number 0, db_iter ends up in an endless loop because it seeks to type kValueTypeForSeek which is larger than kTypeDeletion/kTypeValue. Added test case that triggers the behavior. Test Plan: make clean all check Reviewers: igor, rven, anthony, yhchiang, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D43653 | 06 August 2015, 17:43:28 UTC |
48e6e9a | sdong | 06 August 2015, 03:44:27 UTC | Add util/delete_scheduler_impl.cc to CMakeLists.txt Summary: util/delete_scheduler_impl.cc is missing CMakeLists.txt, which fails Windows build. Add it. Test Plan: Build in both of Windows and Linux Reviewers: anthony, rven, yhchiang, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D43641 | 06 August 2015, 03:56:04 UTC |
c774245 | sdong | 05 August 2015, 20:14:28 UTC | Add Statistics.getHistogramString() to print more detailed outputs of a histogram Summary: Provide a way for users to know more detailed ditribution of a histogram metrics. Example outputs: Manually add statement fprintf(stdout, "%s\n", dbstats->getHistogramString(SST_READ_MICROS).c_str()); Will print out something like: Count: 989151 Average: 1.7659 StdDev: 1.52 Min: 0.0000 Median: 1.2071 Max: 860.0000 Percentiles: P50: 1.21 P75: 1.70 P99: 5.12 P99.9: 13.67 P99.99: 21.70 ------------------------------------------------------ [ 0, 1 ) 390839 39.513% 39.513% ######## [ 1, 2 ) 500918 50.641% 90.154% ########## [ 2, 3 ) 79358 8.023% 98.177% ## [ 3, 4 ) 6297 0.637% 98.813% [ 4, 5 ) 1712 0.173% 98.986% [ 5, 6 ) 1134 0.115% 99.101% [ 6, 7 ) 1222 0.124% 99.224% [ 7, 8 ) 1529 0.155% 99.379% [ 8, 9 ) 1264 0.128% 99.507% [ 9, 10 ) 988 0.100% 99.607% [ 10, 12 ) 1378 0.139% 99.746% [ 12, 14 ) 1828 0.185% 99.931% [ 14, 16 ) 410 0.041% 99.972% [ 16, 18 ) 72 0.007% 99.980% [ 18, 20 ) 67 0.007% 99.986% [ 20, 25 ) 106 0.011% 99.997% [ 25, 30 ) 24 0.002% 99.999% [ 30, 35 ) 1 0.000% 100.000% [ 250, 300 ) 2 0.000% 100.000% [ 300, 350 ) 1 0.000% 100.000% [ 800, 900 ) 1 0.000% 100.000% Test Plan: Manually add a print in db_bench and make sure it prints out as expected. Will add some codes to cover the function Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D43611 | 06 August 2015, 03:05:56 UTC |
29b028b | Islam AbdelRahman | 06 August 2015, 02:16:52 UTC | Make DeleteScheduler tests more reliable Summary: Update DeleteScheduler tests so that they verify the used penalties for waiting instead of measuring the time spent which is not reliable Test Plan: make -j64 delete_scheduler_test && ./delete_scheduler_test COMPILE_WITH_TSAN=1 make -j64 delete_scheduler_test && ./delete_scheduler_test COMPILE_WITH_ASAN=1 make -j64 delete_scheduler_test && ./delete_scheduler_test make -j64 db_test && ./db_test --gtest_filter="DBTest.RateLimitedDelete:DBTest.DeleteSchedulerMultipleDBPaths" COMPILE_WITH_TSAN=1 make -j64 db_test && ./db_test --gtest_filter="DBTest.RateLimitedDelete:DBTest.DeleteSchedulerMultipleDBPaths" COMPILE_WITH_ASAN=1 make -j64 db_test && ./db_test --gtest_filter="DBTest.RateLimitedDelete:DBTest.DeleteSchedulerMultipleDBPaths" Reviewers: yhchiang, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D43635 | 06 August 2015, 02:16:52 UTC |
fca88f8 | sdong | 05 August 2015, 23:04:18 UTC | valgrind_check to exit on test failures Summary: Currently, valgrind_check doesn't fail on test failures, which creates confusion. valgrind_check should fail if test fails. Test Plan: Manually change tests to return test failure or cause memory leak and see valgrind_check has the correct behavior. Reviewers: anthony, yhchiang, IslamAbdelRahman, igor, kradhakrishnan, rven Reviewed By: rven Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D43629 | 06 August 2015, 00:46:09 UTC |
7d364d0 | Poornima Chozhiyath Raman | 05 August 2015, 23:38:12 UTC | Fix build failure Summary: fix the build failure Test Plan: make all Reviewers: sdong, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D43623 | 05 August 2015, 23:38:12 UTC |
960d936 | Poornima Chozhiyath Raman | 05 August 2015, 23:16:46 UTC | Add function 'GetInfoLogList()' Summary: The list of info log files of a db can be obtained using the new function. Test Plan: New test in db_test.cc passed. Reviewers: yhchiang, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: IslamAbdelRahman, leveldb, dhruba Differential Revision: https://reviews.facebook.net/D41715 | 05 August 2015, 23:16:46 UTC |
7ccd1c8 | sdong | 05 August 2015, 18:56:19 UTC | Add two unit tests for SyncWAL() Summary: Add two unit tests for SyncWAL(). One makes sure SyncWAL() doesn't block writes in the other thread. Another one makes sure SyncWAL() doesn't wait ongoing writes to finish before being executed. Create a new test file db_wal_test and move two WAL related tests from db_test to here. Test Plan: Run the new tests Reviewers: IslamAbdelRahman, rven, kradhakrishnan, kolmike, tnovak, yhchiang Reviewed By: yhchiang Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D43605 | 05 August 2015, 21:27:02 UTC |
3ae386e | sdong | 05 August 2015, 19:11:30 UTC | Add statistic histogram "rocksdb.sst.read.micros" Summary: Measure read latency histogram and put in statistics. Compaction inputs are excluded from it when possible (unfortunately usually no possible as we usually take table reader from table cache. Test Plan: Run db_bench and it shows the stats, like: rocksdb.sst.read.micros statistics Percentiles :=> 50 : 1.238522 95 : 2.529740 99 : 3.912180 Reviewers: kradhakrishnan, rven, anthony, IslamAbdelRahman, MarkCallaghan, yhchiang Reviewed By: yhchiang Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D43275 | 05 August 2015, 20:02:33 UTC |
8ecb51a | sdong | 05 August 2015, 18:15:54 UTC | "make commit-prereq" should clean up rocksjava properly Summary: "make commit-prereq" fails to clean up java, which can cause rocksjava failure. Test Plan: Run commit-prepreq Reviewers: IslamAbdelRahman, rven, kradhakrishnan, yhchiang Reviewed By: yhchiang Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D43575 | 05 August 2015, 19:47:24 UTC |
9aec75f | Islam AbdelRahman | 05 August 2015, 18:47:07 UTC | Enable DBTest.FlushSchedule under TSAN Summary: This patch will fix the false positive of DBTest.FlushSchedule under TSAN, we dont need to disable this test Test Plan: COMPILE_WITH_TSAN=1 make -j64 db_test && ./db_test --gtest_filter="DBTest.FlushSchedule" Reviewers: yhchiang, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D43599 | 05 August 2015, 18:47:07 UTC |
bd2fc5f | Islam AbdelRahman | 05 August 2015, 18:45:31 UTC | Fix TSAN for delete_scheduler_test Summary: Fixing TSAN false positive and relaxing the conditions when we are running under TSAN Test Plan: COMPILE_WITH_TSAN=1 make -j64 delete_scheduler_test && ./delete_scheduler_test Reviewers: yhchiang, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D43593 | 05 August 2015, 18:45:31 UTC |
8e01bd1 | sdong | 04 August 2015, 23:50:40 UTC | Fix misplaced position for reversing iterator direction while current key is a merge Summary: While doing forward iterating, if current key is merge, internal iterator position is placed to the next key. If Prev() is called now, needs to do extra Prev() to recover the location. This is second attempt of fixing after reverting ec70fea4c4025351190eba7a02bd09bb5f083790. This time shrink the fix to only merge key is the current key and avoid the reseeking logic for max_iterating skipping Test Plan: enable the two disabled tests and make sure they pass Reviewers: rven, IslamAbdelRahman, kradhakrishnan, tnovak, yhchiang Reviewed By: yhchiang Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D43557 | 05 August 2015, 18:08:50 UTC |
c465071 | Andres Notzli | 05 August 2015, 14:33:27 UTC | Removing duplicate code Summary: While working on https://reviews.facebook.net/D43179 , I found duplicate code in the tests. This patch removes it. Test Plan: make clean all check Reviewers: igor, sdong, rven, anthony, yhchiang Reviewed By: yhchiang Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D43263 | 05 August 2015, 14:33:27 UTC |
e06cf1a | Mike Kolupaev | 05 August 2015, 13:06:39 UTC | [wal changes 3/3] method in DB to sync WAL without blocking writers Summary: Subj. We really need this feature. Previous diff D40899 has most of the changes to make this possible, this diff just adds the method. Test Plan: `make check`, the new test fails without this diff; ran with ASAN, TSAN and valgrind. Reviewers: igor, rven, IslamAbdelRahman, anthony, kradhakrishnan, tnovak, yhchiang, sdong Reviewed By: sdong Subscribers: MarkCallaghan, maykov, hermanlee4, yoshinorim, tnovak, dhruba Differential Revision: https://reviews.facebook.net/D40905 | 05 August 2015, 13:06:39 UTC |
5dc3e68 | Ari Ekmekji | 05 August 2015, 05:19:07 UTC | Update Tests To Enable Subcompactions Summary: Updated DBTest DBCompactionTest and CompactionJobStatsTest to run compaction-related tests once with subcompactions enabled and once disabled using the TEST_P test type in the Google Test suite. Test Plan: ./db_test ./db_compaction-test ./compaction_job_stats_test Reviewers: sdong, igor, anthony, yhchiang Reviewed By: yhchiang Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D43443 | 05 August 2015, 05:19:07 UTC |
c45a57b | Islam AbdelRahman | 05 August 2015, 03:45:27 UTC | Support delete rate limiting Summary: Introduce DeleteScheduler that allow enforcing a rate limit on file deletion Instead of deleting files immediately, files are moved to trash directory and deleted in a background thread that apply sleep penalty between deletes if needed. I have updated PurgeObsoleteFiles and PurgeObsoleteWALFiles to use the delete_scheduler instead of env_->DeleteFile Test Plan: added delete_scheduler_test existing unit tests Reviewers: kradhakrishnan, anthony, rven, yhchiang, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D43221 | 05 August 2015, 03:45:27 UTC |
102ac11 | Yueh-Hsuan Chiang | 05 August 2015, 01:12:58 UTC | Update JAVA-HISTORY.md for v3.13 Summary: Update JAVA-HISTORY.md for v3.13 Test Plan: no code change. Reviewers: igor, anthony, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D43539 | 05 August 2015, 01:12:58 UTC |
3a1d4e6 | Yueh-Hsuan Chiang | 05 August 2015, 00:33:35 UTC | Merge pull request #670 from skunkwerks/fix_osx_shared_library_names Fix shared library names on OSX | 05 August 2015, 00:33:35 UTC |
f5d072a | Yueh-Hsuan Chiang | 05 August 2015, 00:01:08 UTC | Fixed RocksJava test failure of shouldSetTestCappedPrefixExtractor Summary: Fixed RocksJava test failure of shouldSetTestCappedPrefixExtractor by adding the missing native implementation of useCappedPrefixExtractor. Test Plan: make jclean make rocksdbjava -j32 make jtest Reviewers: igor, anthony, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D43551 | 05 August 2015, 00:01:08 UTC |
f39cbcb | Yueh-Hsuan Chiang | 04 August 2015, 23:54:14 UTC | Merge pull request #654 from adamretter/remove-emptyvalue-compactionfilter RemoveEmptyValueCompactionFilter | 04 August 2015, 23:54:14 UTC |
f0b5bcc | ashishn | 29 July 2015, 05:15:15 UTC | add support for capped prefix extractor in java | 04 August 2015, 23:46:23 UTC |
18ba58a | Yoshinori Matsunobu | 04 August 2015, 15:05:06 UTC | Upgrading jemalloc from 3.6.0 to the latest for fbcode+gcc 4.8.1 Summary: MyRocks is using jemalloc latest version, not 3.6.0. Combining multiple versions (3.6.0 in RocksDB and latest in MyRocks) broke some features -- for example, getting SIGSEGV when heap profiling was enabled. This diff switches to use jemalloc latest, if env variable ROCKSDB_FBCODE_BUILD_WITH_481=1 was set. My understanding is this env was used by MyRocks only so it would be safe to change. Test Plan: building MyRocks then verified jemalloc heap profiling worked Reviewers: igor, rven, yhchiang, jtolmer, maykov, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D43479 | 04 August 2015, 23:35:26 UTC |
ce21afd | Yueh-Hsuan Chiang | 04 August 2015, 23:07:17 UTC | Expose the BackupEngine from the Java API Summary: Merge pull request #665 by adamretter Exposes BackupEngine from C++ to the Java API. Previously only BackupableDB was available Test Plan: BackupEngineTest.java Reviewers: fyrz, igor, ankgup87, yhchiang Reviewed By: yhchiang Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D42873 | 04 August 2015, 23:08:54 UTC |
b0d12a1 | Yueh-Hsuan Chiang | 04 August 2015, 22:51:45 UTC | Merge pull request #569 from adamretter/travis-java-api Another attempt at adding the Java API and tests to the travis build | 04 August 2015, 22:51:45 UTC |
241bb2a | Yueh-Hsuan Chiang | 04 August 2015, 22:47:05 UTC | Make DBCompactionTest.SkipStatsUpdateTest more stable. Summary: Make DBCompactionTest.SkipStatsUpdateTest more stable by removing flaky but unnecessary assertion on the size of db as simply checking the random file open count is suffice. Test Plan: db_compaction_test Reviewers: igor, anthony, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D43533 | 04 August 2015, 22:47:05 UTC |
3424eeb | Yueh-Hsuan Chiang | 04 August 2015, 22:04:19 UTC | Polish HISTORY.md Summary: Polish HISTORY.md Test Plan: no code change. Reviewers: igor, anthony, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D43527 | 04 August 2015, 22:04:19 UTC |
cf3e053 | sdong | 04 August 2015, 19:20:38 UTC | crash_test cleans up directory before testing if TEST_TMPDIR is set Summary: In a recent change, crash_test can put data under TEST_TMPDIR. However, the directory is not cleaned before running the test, which may cause unexpected results. Clean it. Test Plan: Run white and black box crash test against non-existing, or non-empty but not compactible DBs, and make sure it works as expected. Reviewers: kradhakrishnan, rven, yhchiang, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D43515 | 04 August 2015, 21:59:28 UTC |
24daff6 | Yueh-Hsuan Chiang | 04 August 2015, 20:51:05 UTC | Fix a typo and update HISTORY.md for NewCompactOnDeletionCollectorFactory(). Summary: Fix a typo and update HISTORY.md for NewCompactOnDeletionCollectorFactory(). Test Plan: no code change. Reviewers: igor, anthony, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D43521 | 04 August 2015, 20:51:05 UTC |
14d0bfa | Yueh-Hsuan Chiang | 04 August 2015, 20:48:16 UTC | Add DBOptions::skip_sats_update_on_db_open Summary: UpdateAccumulatedStats() is used to optimize compaction decision esp. when the number of deletion entries are high, but this function can slowdown DBOpen esp. in disk environment. This patch adds DBOptions::skip_sats_update_on_db_open, which skips UpdateAccumulatedStats() in DB::Open() time when it's set to true. Test Plan: Add DBCompactionTest.SkipStatsUpdateTest Reviewers: igor, anthony, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: tnovak, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D42843 | 04 August 2015, 20:48:16 UTC |
e2a3bfe | sdong | 04 August 2015, 01:11:14 UTC | First half of whitebox_crash_test to keep crashing the same DB Summary: Currently, whitebox crash test is not really executed, because the DB is destroyed after each crash. With this fix, in the first half of the time, DB will keep opening the crashed DB and continue from there. Test Plan: "make whitebox_crash_test" and see the same DB keeps crashing and being reopened. Reviewers: IslamAbdelRahman, yhchiang, rven, kradhakrishnan Reviewed By: kradhakrishnan Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D43503 | 04 August 2015, 19:16:44 UTC |
2e73bd4 | sdong | 04 August 2015, 18:35:44 UTC | crash_test to put DB under TEST_TMPDIR Summary: Currently crash_test only puts data under /tmp. It is less flexible if we want to cover different file systems or media. Make crash_test to appreciate TEST_TMPDIR so that users can run it against another file system. Test Plan: Run blackbox_crash_test and whitebox_crash_test with or without TEST_TMPDIR set and make sure DBs are put in the right place Reviewers: kradhakrishnan, yhchiang, rven, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D43509 | 04 August 2015, 19:10:55 UTC |
1205bdb | sdong | 04 August 2015, 00:23:34 UTC | crash_test to cover simply cases Summary: crash_test now only runs complicated options, multiple column families, prefix hash, frequently changing options, many compaction threads, etc. These options are good to cover new features but we loss coverage in most common use cases. Furthermore, by running only for multiple column families, we are not able to create LSM trees that are large enough to cover some stress cases. Make half of crash_test runs the simply tests: single column family, default mem table, one compaction thread, no change options. Test Plan: Run crash_test Reviewers: rven, yhchiang, IslamAbdelRahman, kradhakrishnan Reviewed By: kradhakrishnan Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D43461 | 04 August 2015, 19:08:38 UTC |
d5c0a6d | Boyang Zhang | 04 August 2015, 17:56:49 UTC | Merge branch 'master' of github.com:facebook/rocksdb Fixed memory leak error | 04 August 2015, 17:56:49 UTC |
2d41403 | Boyang Zhang | 04 August 2015, 15:42:34 UTC | Made change to fix the memory leak Summary: So I took a look and I used a pointer to TableBuilder. Changed it to a unique_ptr. I think this should work, but I cannot run valgrind correctly on my local machine to test it. Test Plan: Run valgrind, but it's not working locally. It says I'm executing an unrecognized instruction. Reviewers: yhchiang Subscribers: dhruba, sdong Differential Revision: https://reviews.facebook.net/D43485 | 04 August 2015, 17:55:42 UTC |
92f7039 | sdong | 04 August 2015, 17:36:08 UTC | fix memory corruption issue in sst_dump --show_compression_sizes Summary: In "sst_dump --show_compression_sizes", a reference of CompressionOptions is kept in TableBuilderOptions, which is destroyed later, causing a memory issue. Test Plan: Run valgrind against SSTDumpToolTest.CompressedSizes and make sure it is fixed Reviewers: IslamAbdelRahman, yhchiang, kradhakrishnan, rven Reviewed By: rven Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D43497 | 04 August 2015, 17:48:44 UTC |
be8621f | Yueh-Hsuan Chiang | 04 August 2015, 03:51:34 UTC | Fix compile warning in compact_on_deletion_collector in some environment Summary: Fix compile warning in compact_on_deletion_collector some environment Test Plan: make Reviewers: igor, sdong, anthony, IslamAbdelRahman Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D43467 | 04 August 2015, 03:53:15 UTC |
2689430 | Yueh-Hsuan Chiang | 04 August 2015, 03:42:55 UTC | Add CompactOnDeletionCollector in utilities/table_properties_collectors. Summary: This diff adds CompactOnDeletionCollector in utilities/table_properties_collectors, which applies a sliding window to a sst file and mark this file as need-compaction when it observe enough deletion entries within the consecutive keys covered by the sliding window. Test Plan: compact_on_deletion_collector_test Reviewers: igor, anthony, IslamAbdelRahman, kradhakrishnan, yoshinorim, sdong Reviewed By: sdong Subscribers: maykov, dhruba Differential Revision: https://reviews.facebook.net/D41175 | 04 August 2015, 03:42:55 UTC |
20b244f | Venkatesh Radhakrishnan | 03 August 2015, 22:53:22 UTC | Fix CompactFiles by adding all necessary files Summary: The compact files API had a bug where some overlapping files are not added. These are files which overlap with files which were added to the compaction input files, but not to the original set of input files. This happens only when there are more than two levels involved in the compaction. An example will illustrate this better. Level 2 has 1 input file 1.sst which spans [20,30]. Level 3 has added file 2.sst which spans [10,25] Level 4 has file 3.sst which spans [35,40] and input file 4.sst which spans [46,50]. The existing code would not add 3.sst to the set of input_files because it only becomes an overlapping file in level 4 and it wasn't one in level 3. When installing the results of the compaction, 3.sst would overlap with output file from the compact files and result in the assertion in version_set.cc:1130 // Must not overlap assert(level <= 0 || level_files->empty() || internal_comparator_->Compare( (*level_files)[level_files->size() - 1]->largest, f->smallest) < 0); This change now adds overlapping files from the current level to the set of input files also so that we don't hit the assertion above. Test Plan: d=/tmp/j; rm -rf $d; seq 1000 | parallel --gnu --eta 'd=/tmp/j/d-{}; mkdir -p $d; TEST_TMPDIR=$d ./db_compaction_test --gtest_filter=*CompactilesOnLevel* --gtest_also_run_disabled_tests >& '$d'/log-{}' Reviewers: igor, yhchiang, sdong Reviewed By: yhchiang Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D43437 | 03 August 2015, 22:53:22 UTC |
87df629 | Venkatesh Radhakrishnan | 03 August 2015, 22:52:52 UTC | Make SuggestCompactRangeNoTwoLevel0Compactions deterministic Summary: Made SuggestCompactRangeNoTwoLevel0Compactions by forcing a flush after generating a file and waiting for compaction at the end. Test Plan: Run SuggestCompactRangeNoTwoLevel0Compactions Reviewers: yhchiang, igor, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D43449 | 03 August 2015, 22:52:52 UTC |
40c6443 | Ari Ekmekji | 03 August 2015, 18:32:14 UTC | Parallelize L0-L1 Compaction: Restructure Compaction Job Summary: As of now compactions involving files from Level 0 and Level 1 are single threaded because the files in L0, although sorted, are not range partitioned like the other levels. This means that during L0-L1 compaction each file from L1 needs to be merged with potentially all the files from L0. This attempt to parallelize the L0-L1 compaction assigns a thread and a corresponding iterator to each L1 file that then considers only the key range found in that L1 file and only the L0 files that have those keys (and only the specific portion of those L0 files in which those keys are found). In this way the overlap is minimized and potentially eliminated between different iterators focusing on the same files. The first step is to restructure the compaction logic to break L0-L1 compactions into multiple, smaller, sequential compactions. Eventually each of these smaller jobs will be run simultaneously. Areas to pay extra attention to are # Correct aggregation of compaction job statistics across multiple threads # Proper opening/closing of output files (make sure each thread's is unique) # Keys that span multiple L1 files # Skewed distributions of keys within L0 files Test Plan: Make and run db_test (newer version has separate compaction tests) and compaction_job_stats_test Reviewers: igor, noetzli, anthony, sdong, yhchiang Reviewed By: yhchiang Subscribers: MarkCallaghan, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D42699 | 03 August 2015, 18:32:14 UTC |
47316c2 | sdong | 03 August 2015, 18:01:24 UTC | dump_manifest supports DB with more number of levels Summary: Now ldb dump_manifest refuses to work if there are 20 levels. Extend the limit to 64. Test Plan: Run the tool with 20 number of levels Reviewers: kradhakrishnan, anthony, IslamAbdelRahman, yhchiang Reviewed By: yhchiang Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D42879 | 03 August 2015, 18:02:09 UTC |
bd852bf | Andres Noetzli | 31 July 2015, 21:11:43 UTC | Fixed typos in db_stress Summary: Fixed typos. Test Plan: None Reviewers: igor, yhchiang, sdong, anthony, rven Reviewed By: rven Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D43365 | 31 July 2015, 21:11:43 UTC |
544be63 | Andres Noetzli | 31 July 2015, 00:46:47 UTC | Fixing fprintf of non string literal Summary: sst_dump_tool contains two instances of `fprintf`s where the `format` argument is not a string literal. This prevents the code from compiling with some compilers/compiler options because of the potential security risks associated with printing non-literals. Test Plan: make all Reviewers: rven, igor, yhchiang, sdong, anthony Reviewed By: anthony Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D43305 | 31 July 2015, 00:46:47 UTC |
193dc97 | Andres Notzli | 30 July 2015, 23:59:03 UTC | Fixing dead code in table_properties_collector_test Summary: There was a bug in table_properties_collector_test that this patch is fixing: `!backward_mode && !test_int_tbl_prop_collector` in TestCustomizedTablePropertiesCollector was never true, so the code in the if-block never got executed. The reason is that the CustomizedTablePropertiesCollector test was skipping tests with `!backward_mode_ && !encode_as_internal`. The reason for skipping the tests is unknown. Test Plan: make table_properties_collector_test && ./table_properties_collector_test Reviewers: rven, igor, yhchiang, anthony, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D43281 | 30 July 2015, 23:59:03 UTC |
05d4265 | Boyang Zhang | 30 July 2015, 00:48:52 UTC | Merge branch 'master' of github.com:facebook/rocksdb | 30 July 2015, 00:48:52 UTC |
4be6d44 | Boyang Zhang | 24 July 2015, 00:05:33 UTC | Compression sizes option for sst_dump_tool Summary: Added a new feature to sst_dump_tool.cc to allow a user to see the sizes of the different compression algorithms on an .sst file. Usage: ./sst_dump --file=<filename> --show_compression_sizes ./sst_dump --file=<filename> --show_compression_sizes --set_block_size=<block_size> Note: If you do not set a block size, it will default to 16kb Test Plan: manual test and the write a unit test Reviewers: IslamAbdelRahman, anthony, yhchiang, rven, kradhakrishnan, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D42963 | 30 July 2015, 00:42:13 UTC |
8161bdb | agiardullo | 11 July 2015, 03:15:45 UTC | WriteBatch Save Points Summary: Support RollbackToSavePoint() in WriteBatch and WriteBatchWithIndex. Support for partial transaction rollback is needed for MyRocks. An alternate implementation of Transaction::RollbackToSavePoint() exists in D40869. However, the other implementation is messier because it is implemented outside of WriteBatch. This implementation is much cleaner and also exposes a potentially useful feature to WriteBatch. Test Plan: Added unit tests Reviewers: IslamAbdelRahman, kradhakrishnan, maykov, yoshinorim, hermanlee4, spetrunia, sdong, yhchiang Reviewed By: yhchiang Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D42723 | 29 July 2015, 23:54:23 UTC |
7bfae3a | sdong | 29 July 2015, 01:31:58 UTC | tools/db_crashtest2.py should run on the same DB Summary: Crash tests are supposed to restart the same DB after crashing, but it is now opening a different DB. Fix it. It's probably a leftover of https://reviews.facebook.net/D17073 Test Plan: Run the test and make sure the same Db is opened. Reviewers: kradhakrishnan, rven, igor, IslamAbdelRahman, yhchiang, anthony Reviewed By: anthony Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D43197 | 29 July 2015, 22:50:37 UTC |
d06c82e | Andres Notzli | 29 July 2015, 02:21:55 UTC | Further cleanup of CompactionJob and MergeHelper Summary: Simplified logic in CompactionJob and removed unused parameter in MergeHelper. Test Plan: make && make check Reviewers: rven, igor, sdong, yhchiang Reviewed By: sdong Subscribers: aekmekji, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D42687 | 29 July 2015, 02:21:55 UTC |
e95c59c | Andres Notzli | 28 July 2015, 23:41:40 UTC | Count number of corrupt keys during compaction Summary: For task #7771355, we would like to log the number of corrupt keys during a compaction. This patch implements and tests the count as part of CompactionJobStats. Test Plan: make && make check Reviewers: rven, igor, yhchiang, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D42921 | 28 July 2015, 23:41:40 UTC |
221a94a | Adam Retter | 07 April 2015, 11:30:45 UTC | Another attempt at adding the Java API and tests to the travis build Summary: Adds the Java build and tests to Travis Test Plan: Make sure that Travis still runs (does currently) Reviewers: igor, fyrz, sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D43173 | 28 July 2015, 10:52:11 UTC |
1bdfcef | Poornima Chozhiyath Raman | 27 July 2015, 21:25:57 UTC | Fix when output level is 0 of universal compaction with trivial move Summary: Fix for universal compaction with trivial move, when the ouput level is 0. The tests where failing. Fixed by allowing normal compaction when output level is 0. Test Plan: modified test cases run successfully. Reviewers: sdong, yhchiang, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: anthony, kradhakrishnan, leveldb, dhruba Differential Revision: https://reviews.facebook.net/D42933 | 27 July 2015, 21:25:57 UTC |
6a82fba | Andres Noetzli | 25 July 2015, 00:07:19 UTC | Add missing hashCode() implementation Summary: Whenever a Java class implements equals(), it has to implement hashCode(), otherwise there might be weird behavior when inserting instances of the class in a hash map for example. This adds two missing hashCode() implementations and extends tests to test the hashCode() implementations. Test Plan: make jtest Reviewers: rven, igor, sdong, yhchiang Reviewed By: yhchiang Subscribers: anthony, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D43017 | 25 July 2015, 00:07:19 UTC |
f73c801 | Andres Noetzli | 24 July 2015, 23:56:26 UTC | Fixing Java tests. Summary: While working on https://reviews.facebook.net/D43017 , I realized that some Java tests are failing due to a deprecated option. This patch removes the offending tests, adds @Deprecated annotations to the Java interface and removes the corresponding functions in rocksjni Test Plan: make jtest (all tests are passing now) Reviewers: rven, igor, sdong, anthony, yhchiang Reviewed By: yhchiang Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D43035 | 24 July 2015, 23:56:26 UTC |
14f4137 | Yueh-Hsuan Chiang | 24 July 2015, 23:29:05 UTC | Correct the comment of DB::GetApproximateSizes Summary: Correct the comment of DB::GetApproximateSizes Test Plan: no code change Reviewers: igor, anthony, IslamAbdelRahman, kradhakrishnan, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D42939 | 24 July 2015, 23:29:05 UTC |
8279d41 | Siying Dong | 23 July 2015, 18:00:43 UTC | Merge pull request #667 from yuslepukhin/fix_now_microsec_win Fix WinEnv::NowMicros | 23 July 2015, 18:00:43 UTC |
eb8e3b4 | Dave Cottlehuber | 23 July 2015, 13:08:24 UTC | Fix shared library names on OSX - OSX format is libname.major.minor.patch.dylib - closes #666 | 23 July 2015, 13:28:11 UTC |
555ca3e | Dmitri Smirnov | 22 July 2015, 21:36:43 UTC | Fix WinEnv::NowMicrosec * std::chrono does not provide enough granularity for microsecs and periodically emits duplicates * the bug is manifested in log rotation logic where we get duplicate log file names and loose previous log content * msvc does not imlement COW on std::strings adjusted the test to use refs in the loops as auto does not retain ref info * adjust auto_log rotation test with Windows specific command to remove a folder. The test previously worked because we have unix utils installed in house but this may not be the case for everyone. | 22 July 2015, 21:36:43 UTC |
82f148e | sdong | 22 July 2015, 20:45:08 UTC | Fix test DBCompactionTest.PartialCompactionFailure undeterministic failure Summary: DBCompactionTest.PartialCompactionFailure has a risk that one flush job writes out two mem tables into one file, so that the total files flushed are less than expected. Fix it by writing for flush to finish after every write. Test Plan: Run the test Reviewers: IslamAbdelRahman, kradhakrishnan, yhchiang, anthony Reviewed By: anthony Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D42831 | 22 July 2015, 20:46:56 UTC |
6002801 | agiardullo | 22 July 2015, 19:56:56 UTC | Abandon ROCKSDB_WARNING Summary: My latest fix to pragma_error.h caused compilation errors for another internal project. I am now unable to figure out how to get pragma_error working on all platforms and build environments (nor am I able to test any other options). Seems like the best option is to get rid of this macro. include/utilities has been deprecated for a year now, so lets just deal with a breaking change in 3.13 to remove these files. And I guess we'll have to live with having an extra convenience.h. Thoughts? Test Plan: build Reviewers: igor, yhchiang, kradhakrishnan, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D42597 | 22 July 2015, 19:56:56 UTC |
4922af6 | Mike Kolupaev | 22 July 2015, 19:37:49 UTC | fixed DBTest.GetPropertiesOfAllTablesTest and DBTest.GetUserDefinedTablaProperties flakiness Summary: These tests used to fail if a compaction happened between flushing tables and enumerating them to get properties. Test Plan: this reports occasional failures without this diff and no failures with it: `for i in {1..10000}; do echo $i; done | parallel --gnu -j100 'TEST_TMPDIR=`TMPDIR=/dev/shm/rockstemp mktemp -d -t` ./db_test --gtest_filter=DBTest.GetUserDefinedTablaProperties >&/dev/null || echo {} failed'` Reviewers: sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D42861 | 22 July 2015, 19:37:49 UTC |
3bf9f9a | Mike Kolupaev | 22 July 2015, 19:27:39 UTC | cleaned up PosixMmapFile a little Summary: https://reviews.facebook.net/D42321 has left PosixMmapFile in some weird state. This diff removes pending_sync_ that was now unused, fixes indentation and prevents Fsync() from calling both fsync() and fdatasync(). Test Plan: `make -j check` Reviewers: sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D42885 | 22 July 2015, 19:27:39 UTC |
fe09a6d | Mike Kolupaev | 22 July 2015, 10:28:08 UTC | [wal changes 2/3] write with sync=true syncs previous unsynced wals to prevent illegal data loss Summary: I'll just copy internal task summary here: " This sequence will cause data loss in the middle after an sync write: non-sync write key 1 flush triggered, not yet scheduled sync write key 2 system crash After rebooting, users might see key 2 but not key 1, which violates the API of sync write. This can be reproduced using unit test FaultInjectionTest::DISABLED_WriteOptionSyncTest. One way to fix it is for a sync write, if there is outstanding unsynced log files, we need to syc them too. " This diff should be considered together with the next diff D40905; in isolation this fix probably could be a little simpler. Test Plan: `make check`; added a test for that (DBTest.SyncingPreviousLogs) before noticing FaultInjectionTest.WriteOptionSyncTest (keeping both since mine asserts a bit more); both tests fail without this diff; for D40905 stacked on top of this diff, ran tests with ASAN, TSAN and valgrind Reviewers: rven, yhchiang, IslamAbdelRahman, anthony, kradhakrishnan, igor, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D40899 | 22 July 2015, 10:28:08 UTC |
06aebca | Andres Notzli | 22 July 2015, 04:33:20 UTC | Report live data size estimate Summary: Fixes T6548822. Added a new function for estimating the size of the live data as proposed in the task. The value can be accessed through the property rocksdb.estimate-live-data-size. Test Plan: There are two unit tests in version_set_test and a simple test in db_test. make version_set_test && ./version_set_test; make db_test && ./db_test gtest_filter=GetProperty Reviewers: rven, igor, yhchiang, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D41493 | 22 July 2015, 04:33:20 UTC |
85ac655 | sdong | 22 July 2015, 00:22:49 UTC | Tests to avoid to use TMPDIR directly Summary: Directly using TMPDIR can cause problems when running tests using parallel option. Fix them. Test Plan: Run all tests in parallel Reviewers: kradhakrishnan, yhchiang, IslamAbdelRahman, anthony Reviewed By: anthony Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D42807 | 22 July 2015, 02:07:34 UTC |
f09d456 | Siying Dong | 22 July 2015, 02:03:44 UTC | Merge pull request #664 from yuslepukhin/add_tests_fix_sanity Add missing tests, fix db_sanity | 22 July 2015, 02:03:44 UTC |
31b35c9 | Dmitri Smirnov | 22 July 2015, 01:04:28 UTC | Add missing tests, fix db_sanity Add heap_test, merge_helper_test Fix uninitialized pointers in db_sanity_test that cause SIGSEV when DB::Open fails in case compression is not linked. | 22 July 2015, 01:04:28 UTC |
66a3cbc | Siying Dong | 22 July 2015, 00:29:02 UTC | Merge pull request #663 from yuslepukhin/fix_windows_build_refactor Fix windows build after refactoring | 22 July 2015, 00:29:02 UTC |
ac5e441 | Dmitri Smirnov | 22 July 2015, 00:20:57 UTC | Fix windows build after refactoring Missing and duplicate files in CMake Missing definition of port::Crash | 22 July 2015, 00:20:57 UTC |
02b635f | sdong | 22 July 2015, 00:04:00 UTC | Fix undeterministic failure of DBTest.GetPropertiesOfAllTablesTest Summary: DBTest.GetPropertiesOfAllTablesTest generates four files and expects four files there, but a L0->L1 comapction can trigger to compact to one single file. Fix it by raising level 0 number of file compaction trigger Test Plan: Run it many times and see it never fails. Reviewers: kradhakrishnan, IslamAbdelRahman, yhchiang, anthony Reviewed By: anthony Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D42789 | 22 July 2015, 00:13:23 UTC |
3dbf4ba | Siying Dong | 20 July 2015, 21:46:15 UTC | RangeSync not to sync last 1MB of the file Summary: From other ones' investigation: "sync_file_range() behavior highly depends on kernel version and filesystem. xfs does neighbor page flushing outside of the specified ranges. For example, sync_file_range(fd, 8192, 16384) does not only trigger flushing page #3 to #4, but also flushing many more dirty pages (i.e. up to page#16)... Ranges of the sync_file_range() should be far enough from write() offset (at least 1MB)." Test Plan: make all check Reviewers: igor, rven, kradhakrishnan, yhchiang, IslamAbdelRahman, anthony Reviewed By: anthony Subscribers: yoshinorim, MarkCallaghan, sumeet, domas, dhruba, leveldb, ljin Differential Revision: https://reviews.facebook.net/D15807 | 21 July 2015, 23:22:40 UTC |
7219088 | Yueh-Hsuan Chiang | 21 July 2015, 10:05:57 UTC | Move general compaction tests from db_test.cc to db_compaction_test.cc Summary: Move general compaction tests from db_test.cc to db_compaction_test.cc Test Plan: db_test db_compaction_test Reviewers: igor, sdong, IslamAbdelRahman, anthony Reviewed By: anthony Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D42651 | 21 July 2015, 10:05:57 UTC |
0adecd9 | Yueh-Hsuan Chiang | 21 July 2015, 00:19:37 UTC | Add db_inplace_update_test back to Makefile Summary: Add db_inplace_update_test back to Makefile Test Plan: make all check Reviewers: igor, sdong, anthony, IslamAbdelRahman Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D42717 | 21 July 2015, 01:17:11 UTC |
0642940 | agiardullo | 21 July 2015, 00:20:40 UTC | Improved FileExists API Summary: Add new CheckFileExists method. Considered changing the FileExists api but didn't want to break anyone's builds. Test Plan: unit tests Reviewers: yhchiang, igor, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D42003 | 21 July 2015, 00:20:40 UTC |
9f1de95 | Yueh-Hsuan Chiang | 20 July 2015, 23:40:41 UTC | Revert Makefile Summary: Revert Makefile Test Plan: make clean Reviewers: sdong, anthony Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D42711 | 20 July 2015, 23:42:02 UTC |
6867fb1 | Yueh-Hsuan Chiang | 20 July 2015, 23:18:35 UTC | Revert "Add missing db_log_iter_test in the test list" Summary: This reverts commit 03467bdd4d568b221107d35056594decfc2440c6. It seems the previous git merge remove all the tabs in Makefile and cause the Makefile not runnable. This patch revert that incorrect merge. Test Plan: make Reviewers: sdong, anthony Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D42705 Conflicts: CMakeLists.txt Makefile | 20 July 2015, 23:18:35 UTC |
443c664 | Yueh-Hsuan Chiang | 20 July 2015, 23:07:54 UTC | Move remaining universal compaction tests from db_test.cc to db_universal_compaction_test.cc Summary: Move remaining universal compaction tests from db_test.cc to db_universal_compaction_test.cc Test Plan: db_test db_universal_compaction_test Reviewers: igor, anthony, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D42645 | 20 July 2015, 23:07:54 UTC |
7462286 | Yueh-Hsuan Chiang | 20 July 2015, 23:05:28 UTC | Move in-place-update related tests from db_test.cc to db_inplace_update_test.cc Summary: Move in-place-update related tests from db_test.cc to db_inplace_update_test.cc Test Plan: db_test db_inplace_update_test Reviewers: igor, anthony, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D42657 | 20 July 2015, 23:05:28 UTC |
03467bd | Yueh-Hsuan Chiang | 20 July 2015, 22:51:33 UTC | Add missing db_log_iter_test in the test list Summary: Add missing db_log_iter_test in the test list Test Plan: make all check Reviewers: igor, anthony, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D42663 | 20 July 2015, 22:51:33 UTC |
331954a | Yueh-Hsuan Chiang | 20 July 2015, 20:53:41 UTC | Fixed DBTestUniversalManualCompactionOutputPathId test Summary: Fixed DBTestUniversalManualCompactionOutputPathId test by changing the expected number of files when setting up the test as flushes no-longer preempt compactions in patch https://reviews.facebook.net/D41931. Also, include db_universal_copaction_test in make all check. Test Plan: db_universal_copaction_test Reviewers: igor, sdong, IslamAbdelRahman, anthony Reviewed By: anthony Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D42639 | 20 July 2015, 20:53:41 UTC |