Skip to main content

Browse the archive

https://github.com/facebook/rocksdb
13 April 2024, 05:39:59 UTC
sort by:
Revision Author Date Message Commit Date
4844d69 sdong 05 November 2019, 18:57:56 UTC Disable error as warning 05 November 2019, 19:05:17 UTC
157da07 sdong 05 November 2019, 17:25:25 UTC Add one more #include<functional> 05 November 2019, 19:05:16 UTC
951ae63 sdong 31 October 2019, 21:17:26 UTC Add some include<functional> 31 October 2019, 21:21:04 UTC
8df5fde sdong 31 October 2019, 21:10:18 UTC [FB Internal] Point to the latest tool chain. 31 October 2019, 21:21:03 UTC
3c56f03 sdong 18 July 2017, 05:17:47 UTC [fb only] revert unintended change of USE_SSE The previuos change that use gcc-5 set USE_SSE to wrong flag by mistake. Fix it. 18 July 2017, 05:20:53 UTC
368afc9 sdong 17 July 2017, 22:39:37 UTC [FB Only] use gcc-5 18 July 2017, 04:59:53 UTC
5339a5e krad 30 September 2016, 01:02:13 UTC Add factory method for creating persistent cache that is accessible from public Summary: Currently there is no mechanism to create persistent cache from headers. Adding a simple factory method to create a simple persistent cache with default or NVM optimized settings. note: Any idea to test this factory is appreciated. Test Plan: None Reviewers: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D64527 03 October 2016, 23:14:03 UTC
24c3b2b Islam AbdelRahman 28 September 2016, 22:42:06 UTC Fix conflict between AddFile() and CompactRange() Summary: Fix the conflict bug between AddFile() and CompactRange() by - Make sure that no AddFile calls are running when asking CompactionPicker to pick compaction for manual compaction - If AddFile() run after we pick the compaction for the manual compaction it will be aware of it since we will add the manual compaction to running_compactions_ after picking it This will solve these 2 scenarios - If AddFile() is running, we will wait for it to finish before we pick a compaction for the manual compaction - If we already picked a manual compaction and then AddFile() started ... we ensure that it never ingest a file in a level that will overlap with the manual compaction Test Plan: unit tests Reviewers: sdong Reviewed By: sdong Subscribers: andrewkr, yoshinorim, jkedgar, dhruba Differential Revision: https://reviews.facebook.net/D64449 28 September 2016, 23:16:10 UTC
f0b8816 Islam AbdelRahman 26 September 2016, 21:44:48 UTC Fix AddFile() conflict with compaction output [WaitForAddFile()] Summary: Since AddFile unlock/lock the mutex inside LogAndApply() we need to ensure that during this period other compactions cannot run since such compactions are not aware of the file we are ingesting and could create a compaction that overlap wit this file this diff add - WaitForAddFile() call that will ensure that no AddFile() calls are being processed right now - Call `WaitForAddFile()` in 3 locations -- When doing manual Compaction -- When starting automatic Compaction -- When doing CompactFiles() Test Plan: unit test Reviewers: lightmark, yiwu, andrewkr, sdong Reviewed By: sdong Subscribers: andrewkr, yoshinorim, jkedgar, dhruba Differential Revision: https://reviews.facebook.net/D64383 27 September 2016, 17:37:29 UTC
8ee2ee8 Islam AbdelRahman 26 September 2016, 17:39:07 UTC Fix CompactFilesTest.ObsoleteFiles timeout (#1353) 26 September 2016, 19:03:39 UTC
af28d11 Aaron Gao 24 September 2016, 00:16:38 UTC not cut compaction output when compact to level 0 Summary: we should not call ShouldStopBefore() in compaction when the compaction targets level 0. Otherwise, CheckConsistency will fail the assertion of seq number check on level 0. Test Plan: make all check -j64 I also manully test that using db_bench to compact files to level 0. Without this line change, the assertion files and multiple files are generated on level 0 after compaction. Reviewers: yhchiang, andrewkr, yiwu, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D64269 26 September 2016, 03:08:34 UTC
a34c1e3 yiwu-arbug 23 September 2016, 23:15:14 UTC Recover same sequence id from WAL (#1350) Summary: Revert the behavior where we don't read sequence id from WAL, but increase it as we replay the log. We still keep the behave for 2PC for now but will fix later. This change fixes github issue 1339, where some writes come with WAL disabled and we may recover records with wrong sequence id. Test Plan: Added unit test. Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D64275 26 September 2016, 03:08:24 UTC
165cec6 Yi Wu 15 September 2016, 20:44:33 UTC Fix DBWALTest.RecoveryWithLogDataForSomeCFs with mac Summary: Seems there's no std::array on mac+clang. Use raw array instead. Test Plan: run ./db_wal_test on mac. Reviewers: andrewkr Reviewed By: andrewkr Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D64005 19 September 2016, 21:29:29 UTC
d7d6a9a Andrew Kryczka 15 September 2016, 18:40:48 UTC Fix recovery for WALs without data for all CFs Summary: if one or more CFs had no data in the WAL, the log number that's used by FindObsoleteFiles() wasn't updated. We need to treat this case the same as if the data for that WAL had been flushed. Test Plan: new unit test Reviewers: IslamAbdelRahman, yiwu, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D63963 19 September 2016, 19:51:54 UTC
55ab150 Andrew Kryczka 15 September 2016, 16:55:02 UTC Fix GetSortedWalFiles when log recycling enabled Summary: Previously the sequence number was mistakenly passed in an argument where the log number should go. This caused the reader to assume the old WAL format was used, which is incompatible with the WAL recycling format. Test Plan: new unit test, verified it fails before this change and passes afterwards. Reviewers: yiwu, lightmark, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D63987 19 September 2016, 19:51:48 UTC
cba883c xh931076284 14 September 2016, 17:17:34 UTC Fix bug in UnScSigned-off-by: xh931076284 <931076284@qq.com> (#1336) Fix HdfsEnv::UnSchedule() API error 19 September 2016, 19:50:12 UTC
c4ffd74 Yi Wu 16 September 2016, 17:54:31 UTC Fix java makefile dependencies Summary: Fix dependencies in java makefile, to avoid java build failure. Test Plan: run "make rocksdbjava" multiple times. Reviewers: IslamAbdelRahman, sdong, yhchiang Reviewed By: yhchiang Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D64023 19 September 2016, 19:49:47 UTC
ac7e52c Yi Wu 14 September 2016, 20:12:55 UTC Rename jvalue to jval in rocksjni Summary: jvalue shadows a global name in <jni.h>. Rename it to jval to fix java build. Test Plan: JAVA_HOME=/usr/local/jdk-7u10-64 make rocksdbjava -j64 Reviewers: adamretter, yhchiang, IslamAbdelRahman Reviewed By: yhchiang, IslamAbdelRahman Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D63981 19 September 2016, 19:49:38 UTC
28d8501 Adam Retter 12 September 2016, 18:51:08 UTC Allow an offset as well as a length to be specified for byte[] operations in RocksJava JNI (#1264) Test Plan: Execute the Java test suite Reviewers: yhchiang Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D61971 19 September 2016, 19:47:40 UTC
e8bbb13 Islam AbdelRahman 12 September 2016, 19:34:08 UTC Release RocksDB 4.12 Summary: Release 4.12 Test Plan: none Reviewers: andrewkr, yiwu, lightmark, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D63885 12 September 2016, 19:56:56 UTC
b06b191 zhangjinpeng1987 09 September 2016, 17:11:30 UTC add C api for set wal_recovery_mode (#1327) * add C api for set wal recovery mode * add test 09 September 2016, 17:11:30 UTC
1cca091 Islam AbdelRahman 08 September 2016, 21:45:32 UTC Temporarily revert Prev() prefix support Summary: Temporarily revert commits for supporting prefix Prev() to unblock MyRocks and RocksDB release These are the commits reverted - 6a14d55bd913490dbd61d682567e6e0625756c0d - b18f9c9eace89d63f37432ce1a3dba48bddbcef0 - db74b1a21905336e2c178ff1f2ffd12c7852b7b8 - 2482d5fb45d8f01b0b065d649d01f43dacad799c Test Plan: make check -j64 Reviewers: sdong, lightmark Reviewed By: lightmark Subscribers: andrewkr, dhruba, yoshinorim Differential Revision: https://reviews.facebook.net/D63789 08 September 2016, 21:45:32 UTC
de28a25 Andrew Kryczka 08 September 2016, 21:02:19 UTC Update HISTORY.md for thread-local stats Summary: as titled Test Plan: doitlive Reviewers: sdong, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D63777 08 September 2016, 21:02:19 UTC
0fcb6db ammongit 08 September 2016, 18:31:06 UTC Remove extraneous function prototypes from c.h (#1326) * Fix function prototypes from upstream commit 32149059. * Fix removed function. * Readd removed function. 08 September 2016, 18:31:06 UTC
52ee07b Islam AbdelRahman 07 September 2016, 22:41:54 UTC Move AddFile() tests to external_sst_file_test.cc Summary: Simply move the tests Test Plan: make check -j64 Reviewers: andrewkr, lightmark, yiwu, yhchiang, kradhakrishnan, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D62529 07 September 2016, 22:41:54 UTC
66a91e2 Edouard A 07 September 2016, 19:37:45 UTC Add NoSpace subcode to IOError (#1320) Add a sub code to distinguish "out of space" errors from regular I/O errors 07 September 2016, 19:37:45 UTC
67036c0 sdong 06 September 2016, 19:57:46 UTC Fix Flaky ColumnFamilyTest.FlushCloseWALFiles Summary: In ColumnFamilyTest.FlushCloseWALFiles, there is a small window in which the flush has finished but the log writer is not yet closed, causing the assert failure. Fix it by explicitly waiting the flush job to finish. Test Plan: Run the test many times in high parallelism. Subscribers: leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D63423 07 September 2016, 18:19:15 UTC
0e2da49 wenduo 07 September 2016, 08:36:16 UTC fix typo in option.h's comment (#1321) 07 September 2016, 08:36:16 UTC
6d61358 Joel Marcey 07 September 2016, 04:10:14 UTC Add real Google Analytics ID Summary: This adds the actual RocksDB Google Analytics ID that will be used when we switch over from WP to GH-pages Test Plan: visual Reviewers: IslamAbdelRahman, lgalanis, sdong Reviewed By: sdong Subscribers: sdong, andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D63429 07 September 2016, 04:10:14 UTC
2d9d36e Joel Marcey 07 September 2016, 04:08:53 UTC Have "Edit on GitHub" point to master instead of gh-pages Summary: If someone clicks on `Edit on GitHub` to edit a doc for a pull request, for example, then we should point to `master` instead of `gh-pages`. > Also fixed some Windows-based line endings. From CRLF to LF. Test Plan: https://www.facebook.com/pxlcld/pvV2 Reviewers: IslamAbdelRahman, lgalanis, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D63435 07 September 2016, 04:08:53 UTC
9377518 Joel Marcey 07 September 2016, 04:07:47 UTC Update landing page content Summary: Changes based on comments in D62985 Test Plan: Visual https://www.facebook.com/pxlcld/pvVs Reviewers: lgalanis, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D63477 07 September 2016, 04:07:47 UTC
1ec75ee Joel Marcey 07 September 2016, 04:07:13 UTC Add redirects from old blog posts link to new format Summary: The new blog post links will be formatted differently coming over to gh-pages. But we can redirect from the old style over to the new style for existing blog posts. Test Plan: Visual https://www.facebook.com/pxlcld/pvWQ Reviewers: lgalanis, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D63513 07 September 2016, 04:07:13 UTC
607628d sdong 01 September 2016, 22:28:40 UTC Support ZSTD with finalized format Summary: ZSTD 1.0.0 is coming. We can finally add a support of ZSTD without worrying about compatibility. Still keep ZSTDNotFinal for compatibility reason. Test Plan: Run all tests. Run db_bench with ZSTD version with RocksDB built with ZSTD 1.0 and older. Reviewers: andrewkr, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: cyan, igor, IslamAbdelRahman, leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D63141 06 September 2016, 19:22:16 UTC
ce1be2c Injun Song 06 September 2016, 15:41:43 UTC Fix build error on Windows (AppVeyor) (#1315) Add 'cf_options' to source list and db_imple.cc fix casting 06 September 2016, 15:41:43 UTC
f7669b4 sdong 02 September 2016, 21:06:04 UTC Fix Windows Build Summary: Fix two Windows build problems. Test Plan: Build on Windows and run all Linux tests. Reviewers: IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D63189 03 September 2016, 00:10:28 UTC
22696b0 sdong 02 September 2016, 23:08:54 UTC Fix uninitlized CompactionJob::SubcompactionState::current_output_file_size Summary: The new variable introduced in 2149059f910149197d1a0f79ac08cf19465ea2d may be unitialized. Valgrind is failing because of it. Test Plan: Run valgrind tests Reviewers: yiwu, andrewkr, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D63201 03 September 2016, 00:06:20 UTC
c1865e0 Islam AbdelRahman 02 September 2016, 23:30:47 UTC Trigger more tests per diff Summary: We trigger tests for diffs we land, we should trigger as much as possible to make our dashboard as recent as possible Test Plan: none Reviewers: gunnarku, kradhakrishnan, andrewkr, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D63159 02 September 2016, 23:30:47 UTC
a88677d Yi Wu 02 September 2016, 21:16:31 UTC Remove ImmutableCFOptions from public API Summary: There's no reference to ImmutableCFOptions elsewhere in /include/rocksdb. ImmutableCFOptions was introduced in this commit (https://github.com/facebook/rocksdb/commit/5665e5e285c25c1674567f747df92c131037d2dc) but later its reference in /include/rocksdb/table.h is removed. Test Plan: make all check Reviewers: IslamAbdelRahman, sdong, yhchiang Reviewed By: yhchiang Subscribers: yhchiang, andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D63177 02 September 2016, 21:16:31 UTC
80c7559 Islam AbdelRahman 02 September 2016, 18:17:58 UTC Fix data race in AddFile() with multiple files + custom comparator bug Summary: When ingesting multiple files - We should use user comparator - Should not call `cfd->current()` outside of mutex Test Plan: unit tests Reviewers: sdong, lightmark Reviewed By: lightmark Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D63075 02 September 2016, 18:17:58 UTC
5051755 Islam AbdelRahman 02 September 2016, 17:58:08 UTC Fix db_bench memory use after free (detected by clang_analyze) Summary: Fix using `arg[i].thread` after deleting it Test Plan: run clang_analyze Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D63171 02 September 2016, 17:58:08 UTC
4fd08f4 John Alexander 02 September 2016, 02:57:55 UTC Ensure Correct Behavior of StatsLevel kExceptDetailedTimers and kExceptTimeForMutex (#1308) * Fix StatsLevel so that kExceptTimeForMutex leaves compression stats enabled and kExceptDetailedTimers disables mutex lock stats. Also change default stats level to kExceptDetailedTimers (disabling both compression and mutex timing). * Changed order of StatsLevel enum to simplify logic for determining what stats to record. 02 September 2016, 02:57:55 UTC
e14fbaa Joel Marcey 02 September 2016, 00:34:39 UTC Add FAQ based on the front page of the current rocksdb.org Summary: This takes the questions currently on the front of the Wordpress-based rocksdb.org and starts an official FAQ with them. Depends on: https://reviews.facebook.net/D63105 Test Plan: Visual Reviewers: IslamAbdelRahman, lgalanis Reviewed By: lgalanis Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D63123 02 September 2016, 00:34:39 UTC
3c22624 Joel Marcey 02 September 2016, 00:28:49 UTC Migrate the RocksDB Worpdress blog over to Jekyll Summary: Tried to: - preserve existing links - move existing images over (there were 2) - preserve codeblocks (modified where apprporiate) - etc. Also as agreed upon: - All blog posts are preserved. - Comments are not preserved. - Not turning on comments for future blog posts (use the FB developer group instead). - Like button at the end of the blog post. Depends on https://reviews.facebook.net/D63051 Test Plan: Visual Reviewers: IslamAbdelRahman, lgalanis Reviewed By: lgalanis Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D63105 02 September 2016, 00:28:49 UTC
ee0e220 Joel Marcey 02 September 2016, 00:26:38 UTC Transfer the current Getting Started contents to GitHub Pages Summary: This moves the current Getting Started that we have hosted over to our new GitHub pages infra. Next will be to move the blog over, along with some updates and maintenance of links, etc. in the new infra. Depends on: https://reviews.facebook.net/D62985 Test Plan: Visual Reviewers: IslamAbdelRahman, lgalanis Reviewed By: lgalanis Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D63051 02 September 2016, 00:26:38 UTC
5a0e9a4 Joel Marcey 02 September 2016, 00:25:08 UTC Initial Landing Page Summary: This is the proof of concept for the initial landing page for RocksDB when we move to GitHub pages. The images for each feature can change, but using these as placeholders for now. The styling may change a bit as well. Test Plan: Visual Reviewers: IslamAbdelRahman, lgalanis, sdong Reviewed By: lgalanis Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D62985 02 September 2016, 00:25:08 UTC
9447a85 Joel Marcey 01 September 2016, 22:24:19 UTC Remove the `doc` directory Summary: The content in this directory is old, has not been updated in some cases years, and superceded by the current Wordpress-based docs (soon to be changed to Jekyll) Test Plan: visual Reviewers: IslamAbdelRahman, lgalanis Reviewed By: lgalanis Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D62877 01 September 2016, 22:24:19 UTC
3214905 sdong 16 June 2016, 23:02:52 UTC Merge options source_compaction_factor, max_grandparent_overlap_bytes and expanded_compaction_factor into max_compaction_bytes Summary: To reduce number of options, merge source_compaction_factor, max_grandparent_overlap_bytes and expanded_compaction_factor into max_compaction_bytes. Test Plan: Add two new unit tests. Run all existing tests, including jtest. Reviewers: yhchiang, igor, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D59829 01 September 2016, 21:33:24 UTC
4590b53 Aaron Gao 01 September 2016, 20:50:39 UTC add stats to Cache::LookUp() Summary: basically for SimCache stats. I find most times it is hard to pass Statistics* to SimCache constructor. Test Plan: make all check Reviewers: andrewkr, sdong, yiwu Reviewed By: yiwu Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D62193 01 September 2016, 20:50:39 UTC
85bb308 Sherlock 01 September 2016, 20:47:21 UTC Expose Utility function StringToMap() (#1306) 01 September 2016, 20:47:21 UTC
8ce1b84 Islam AbdelRahman 31 August 2016, 22:10:12 UTC Fix Travis on Mac Summary: not sure why travis complain about this line, works fine on my mac Test Plan: run on my mac Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D63045 31 August 2016, 22:10:12 UTC
380e651 Kien-hung Li 31 August 2016, 22:03:10 UTC Fix Mac build failure (#1309) 31 August 2016, 22:03:10 UTC
1613fa9 Andrew Kryczka 31 August 2016, 21:02:09 UTC Thread-specific histogram statistics Summary: To reduce contention for atomics when HistogramStats are shared across threads, this diff makes them thread-specific so updates are faster. This comes at the expense of slower reads (much less frequent), which now require merging all histograms. In this diff, - Thread-specific HistogramImpl is created upon the thread's first measureTime() - Thread-specific HistogramImpl are merged and deleted upon thread termination or ThreadLocalPtr destruction, whichever comes first - getHistogramString() and histogramData() merge all histograms, both thread-specific and previously merged ones Test Plan: unit tests, ran db_bench and verified histograms look similar before: $ TEST_TMPDIR=/dev/shm/ perf record -g ./db_bench --benchmarks=readwhilewriting --statistics --num=1000000 --use_existing_db --threads=64 --cache_size=250000000 --compression_type=lz4 ... + 7.63% db_bench db_bench [.] rocksdb::HistogramStat::Add after: $ TEST_TMPDIR=/dev/shm/ perf record -g ./db_bench --benchmarks=readwhilewriting --statistics --num=1000000 --use_existing_db --threads=64 --cache_size=250000000 --compression_type=lz4 ... + 0.98% db_bench db_bench [.] rocksdb::HistogramStat::Add Reviewers: sdong, MarkCallaghan, kradhakrishnan, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D62649 31 August 2016, 21:02:09 UTC
6a14d55 Aaron Gao 31 August 2016, 19:07:09 UTC add prefix_seek_mode to db_iter_test Summary: add prefix_seek_mode to db_iter_test to enable data race test for iterator when prefix_extractor != nullptr Test Plan: make all check -j64 Reviewers: andrewkr, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D63027 31 August 2016, 19:07:09 UTC
de47e2b Yi Wu 31 August 2016, 15:56:34 UTC Fix ClockCache memory leak Summary: Fix ClockCache memory leak found by valgrind: # Add destructor to cleanup cached values. # Delete key with cache handle immediately after handle is recycled, and erase table entry immediately if duplicated cache entry is inserted. Test Plan: make DISABLE_JEMALLOC=1 valgrind_check Reviewers: IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D62973 31 August 2016, 15:56:34 UTC
f099af4 Islam AbdelRahman 31 August 2016, 07:10:49 UTC Fix travis Summary: Fix travis for mac by using gflags package instead of doing `brew install` Test Plan: https://travis-ci.org/facebook/rocksdb/builds/156358515 Reviewers: sdong, andrewkr, lightmark, yiwu Reviewed By: yiwu Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D62997 31 August 2016, 07:10:49 UTC
db74b1a Aaron Gao 31 August 2016, 05:19:42 UTC fix bug in merge_iterator when data race happens Summary: core dump when run `./db_stress --max_background_compactions=1 --max_write_buffer_number=3 --sync=0 --reopen=20 --write_buffer_size=33554432 --delpercent=5 --log2_keys_per_lock=10 --block_size=16384 --allow_concurrent_memtable_write=1 --test_batches_snapshots=0 --max_bytes_for_level_base=67108864 --progress_reports=0 --mmap_read=1 --kill_prefix_blacklist=WritableFileWriter::Append,WritableFileWriter::WriteBuffered --writepercent=35 --disable_data_sync=0 --readpercent=50 --subcompactions=3 --ops_per_thread=20000000 --memtablerep=skip_list --prefix_size=0 --target_file_size_multiplier=1 --column_families=1 --db=/dev/shm/rocksdb/rocksdb_crashtest_whitebox --threads=32 --disable_wal=0 --open_files=500000 --destroy_db_initially=0 --target_file_size_base=16777216 --nooverwritepercent=1 --iterpercent=10 --max_key=100000000 --prefixpercent=0 --use_clock_cache=false --kill_random_test=189 --cache_size=1048576 --verify_checksum=1` Actually the relevant flag is `--threads`, data race when --thread > 1 cause problem. It is possible that multiple threads read/write memtable simultaneously. After one thread calls Prev(), another thread may insert a new key just between the current key and the key next, which may cause the assert(current_ == CurrentForward()) failure when the first thread calls Next() again if in prefix seek mode Test Plan: rerun db_stress with >1 thread / make all check -j64 Reviewers: sdong, andrewkr, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D62979 31 August 2016, 05:19:42 UTC
b18f9c9 Aaron Gao 30 August 2016, 20:48:31 UTC add nullptr check to internal_prefix_transform Summary: patch for D62361 Test Plan: make all check Reviewers: sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D62883 30 August 2016, 20:48:31 UTC
4e395e8 Joel Marcey 30 August 2016, 15:05:43 UTC Update docs README.md Summary: Update the docs `README.md` to be more specific to RocksDB as opposed to the more generic information that is there now. Test Plan: visual Reviewers: lgalanis, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D62733 30 August 2016, 15:05:43 UTC
2482d5f Aaron Gao 30 August 2016, 03:55:39 UTC support Prev() in prefix seek mode Summary: As title, make sure Prev() works as expected with Next() when the current iter->key() in the range of the same prefix in prefix seek mode Test Plan: make all check -j64 (add prefix_test with PrefixSeekModePrev test case) Reviewers: andrewkr, sdong, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: yoshinorim, andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D61419 30 August 2016, 03:55:39 UTC
7541c7a Yi Wu 29 August 2016, 17:40:00 UTC Fix cache_test valgrind_check failure Summary: Refactor cache_test to get around gtest valgrind failure. Test Plan: make valgrind_check Reviewers: sdong, kradhakrishnan, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D62817 29 August 2016, 17:40:00 UTC
c8513cd Adam Retter 29 August 2016, 03:24:08 UTC Update the download location of Snappy (#1304) * Fix the download location of Snappy, no longer available from Google Code * Ensure that curl follows any redirect headers when downloading binaries 29 August 2016, 03:24:08 UTC
b49b92c Islam AbdelRahman 27 August 2016, 01:55:58 UTC Introduce Read amplification bitmap (read amp statistics) Summary: Add ReadOptions::read_amp_bytes_per_bit option which allow us to create a bitmap for every data block we read the bitmap will contain (block_size / read_amp_bytes_per_bit) bits. We will use this bitmap to mark which bytes have been used of the block so we can calculate the read amplification Test Plan: added new tests Reviewers: andrewkr, yhchiang, sdong Reviewed By: sdong Subscribers: yiwu, leveldb, march, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D58707 27 August 2016, 01:55:58 UTC
c700484 Aaron Gao 26 August 2016, 18:46:32 UTC store prefix_extractor_name in table Summary: Make sure prefix extractor name is stored in SST files and if DB is opened with a prefix extractor of a different name, prefix bloom is skipped when read the file. Also add unit tests for that. Test Plan: before change: ``` Note: Google Test filter = BlockBasedTableTest.SkipPrefixBloomFilter [==========] Running 1 test from 1 test case. [----------] Global test environment set-up. [----------] 1 test from BlockBasedTableTest [ RUN ] BlockBasedTableTest.SkipPrefixBloomFilter table/table_test.cc:1421: Failure Value of: db_iter->Valid() Actual: false Expected: true [ FAILED ] BlockBasedTableTest.SkipPrefixBloomFilter (1 ms) [----------] 1 test from BlockBasedTableTest (1 ms total) [----------] Global test environment tear-down [==========] 1 test from 1 test case ran. (1 ms total) [ PASSED ] 0 tests. [ FAILED ] 1 test, listed below: [ FAILED ] BlockBasedTableTest.SkipPrefixBloomFilter 1 FAILED TEST ``` after: ``` Note: Google Test filter = BlockBasedTableTest.SkipPrefixBloomFilter [==========] Running 1 test from 1 test case. [----------] Global test environment set-up. [----------] 1 test from BlockBasedTableTest [ RUN ] BlockBasedTableTest.SkipPrefixBloomFilter [ OK ] BlockBasedTableTest.SkipPrefixBloomFilter (0 ms) [----------] 1 test from BlockBasedTableTest (0 ms total) [----------] Global test environment tear-down [==========] 1 test from 1 test case ran. (0 ms total) [ PASSED ] 1 test. ``` Reviewers: sdong, andrewkr, yiwu, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D61215 26 August 2016, 18:46:32 UTC
4ad928e Aaron Gao 26 August 2016, 18:36:14 UTC add comment to SimCache to estimate actual capacity Summary: as title Test Plan: make all check Reviewers: yiwu Reviewed By: yiwu Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D62493 26 August 2016, 18:36:14 UTC
e9b2af8 Islam AbdelRahman 26 August 2016, 17:41:35 UTC Expose ThreadPool under include/rocksdb/threadpool.h Summary: This diff split ThreadPool to -ThreadPool (abstract interface exposed in include/rocksdb/threadpool.h) -ThreadPoolImpl (actual implementation in util/threadpool_imp.h) This allow us to expose ThreadPool to the user so we can use it as an option later Test Plan: existing unit tests Reviewers: andrewkr, yiwu, yhchiang, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D62085 26 August 2016, 17:41:35 UTC
23a0570 Justin T. Gibbs 26 August 2016, 17:14:40 UTC Document memtable flush behavior in CancelAllBackgroundWork() Summary: Update History.md to reflect recent change that ensures unpersisted data is flushed even if clients call CancelAllBackgroundWork() directly. Test Plan: Review rendering of markdown. Reviewers: sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D62703 26 August 2016, 17:14:40 UTC
dade61a sdong 25 August 2016, 21:43:12 UTC Mitigate regression bug of options.max_successive_merges hit during DB Recovery Summary: After 1b8a2e8fdd1db0dac3cb50228065f8e7e43095f0, DB Pointer is passed to WriteBatchInternal::InsertInto() while DB recovery. This can cause deadlock if options.max_successive_merges hits. In that case DB::Get() will be called. Get() will try to acquire the DB mutex, which is already held by the DB::Open(), causing a deadlock condition. This commit mitigates the problem by not passing the DB pointer unless 2PC is allowed. Test Plan: Add a new test and run it. Reviewers: IslamAbdelRahman, andrewkr, kradhakrishnan, horuff Reviewed By: kradhakrishnan Subscribers: leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D62625 26 August 2016, 00:30:34 UTC
cce702a Islam AbdelRahman 25 August 2016, 19:57:35 UTC [db_bench] Support single benchmark arguments (Repeat for X times, Warm up for X times), Support CombinedStats (AVG / MEDIAN) Summary: This diff allow us to run a single benchmark X times and warm it up for Y times. and see the AVG & MEDIAN throughput of these X runs for example ``` $ ./db_bench --benchmarks="fillseq,readseq[X5-W2]" Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags RocksDB: version 4.12 Date: Wed Aug 24 10:45:26 2016 CPU: 32 * Intel(R) Xeon(R) CPU E5-2660 0 @ 2.20GHz CPUCache: 20480 KB Keys: 16 bytes each Values: 100 bytes each (50 bytes after compression) Entries: 1000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 110.6 MB (estimated) FileSize: 62.9 MB (estimated) Write rate: 0 bytes/second Compression: Snappy Memtablerep: skip_list Perf Level: 1 WARNING: Assertions are enabled; benchmarks unnecessarily slow ------------------------------------------------ Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags DB path: [/tmp/rocksdbtest-8616/dbbench] fillseq : 4.695 micros/op 212971 ops/sec; 23.6 MB/s DB path: [/tmp/rocksdbtest-8616/dbbench] Warming up benchmark by running 2 times readseq : 0.214 micros/op 4677005 ops/sec; 517.4 MB/s readseq : 0.212 micros/op 4706834 ops/sec; 520.7 MB/s Running benchmark for 5 times readseq : 0.218 micros/op 4588187 ops/sec; 507.6 MB/s readseq : 0.208 micros/op 4816538 ops/sec; 532.8 MB/s readseq : 0.213 micros/op 4685376 ops/sec; 518.3 MB/s readseq : 0.214 micros/op 4676787 ops/sec; 517.4 MB/s readseq : 0.217 micros/op 4618532 ops/sec; 510.9 MB/s readseq [AVG 5 runs] : 4677084 ops/sec; 517.4 MB/sec readseq [MEDIAN 5 runs] : 4676787 ops/sec; 517.4 MB/sec ``` Test Plan: run db_bench Reviewers: sdong, andrewkr, yhchiang Reviewed By: yhchiang Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D62235 25 August 2016, 19:57:35 UTC
3586901 Islam AbdelRahman 25 August 2016, 19:53:26 UTC cat tests logs sorted by exit code Summary: Instead of doing a cat for all the log files, we first sort them and by exit code and cat the failing tests at the end. This will make it easier to debug failing tests, since we will just need to look at the end of the logs instead of searching in them Test Plan: run it locally Reviewers: sdong, yiwu, lightmark, kradhakrishnan, yhchiang, andrewkr Reviewed By: andrewkr Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D62211 25 August 2016, 19:53:26 UTC
b2ce595 Justin Gibbs 25 August 2016, 19:24:22 UTC Persist data during user initiated shutdown Summary: Move the manual memtable flush for databases containing data that has bypassed the WAL from DBImpl's destructor to CancleAllBackgroundWork(). CancelAllBackgroundWork() is a publicly exposed API which allows async operations performed by background threads to be disabled on a database. In effect, this places the database into a "shutdown" state in advance of calling the database object's destructor. No compactions or flushing of SST files can occur once a call to this API completes. When writes are issued to a database with WriteOptions::disableWAL set to true, DBImpl::has_unpersisted_data_ is set so that memtables can be flushed when the database object is destroyed. If CancelAllBackgroundWork() has been called prior to DBImpl's destructor, this flush operation is not possible and is skipped, causing unnecessary loss of data. Since CancelAllBackgroundWork() is already invoked by DBImpl's destructor in order to perform the thread join portion of its cleanup processing, moving the manual memtable flush to CancelAllBackgroundWork() ensures data is persisted regardless of client behavior. Test Plan: Write an amount of data that will not cause a memtable flush to a rocksdb database with all writes marked with WriteOptions::disableWAL. Properly "close" the database. Reopen database and verify that the data was persisted. Reviewers: IslamAbdelRahman, yiwu, yoshinorim, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D62277 25 August 2016, 19:24:22 UTC
4b3438d Islam AbdelRahman 25 August 2016, 18:02:39 UTC Fix parallel valgrind (valgrind_check) Summary: I just realized that when we run parallel valgrind we actually don't run the parallel tests under valgrind (we run the normally) This patch make sure that we run both parallel and non-parallel tests with valgrind Test Plan: DISABLE_JEMALLOC=1 make valgrind_check -j64 Reviewers: andrewkr, yiwu, lightmark, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D62469 25 August 2016, 18:02:39 UTC
a081f79 Andrew Kryczka 25 August 2016, 17:42:26 UTC Relax consistency for thread-local ticker stats Summary: see discussion in D62337 Test Plan: unit tests Reviewers: sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D62577 25 August 2016, 17:42:26 UTC
b10d65c Alex Robinson 25 August 2016, 17:40:38 UTC Update and slightly clarify instructions in build_detect_platform (#1301) 25 August 2016, 17:40:38 UTC
f85f99b Adam Retter 25 August 2016, 17:16:26 UTC Fix the Windows build of RocksDB Java. Similar to https://github.com/facebook/rocksdb/issues/1220 (#1284) 25 August 2016, 17:16:26 UTC
7b81095 Mike Kolupaev 25 August 2016, 11:39:26 UTC Fix a crash when compaction fails to open a file Summary: We've got a crash with this stack trace: Program terminated with signal SIGTRAP, Trace/breakpoint trap. #0 0x00007fc85f2f4009 in raise () from /usr/local/fbcode/gcc-4.9-glibc-2.20-fb/lib/libpthread.so.0 #1 0x00000000005c8f61 in facebook::logdevice::handle_sigsegv(int) () at logdevice/server/sigsegv.cpp:159 #2 0x00007fc85f2f4150 in <signal handler called> () at /usr/local/fbcode/gcc-4.9-glibc-2.20-fb/lib/libpthread.so.0 #3 0x00000000031ed80c in rocksdb::NewReadaheadRandomAccessFile() at util/file_reader_writer.cc:383 #4 0x00000000031ed80c in rocksdb::NewReadaheadRandomAccessFile() at util/file_reader_writer.cc:472 #5 0x00000000031558e7 in rocksdb::TableCache::GetTableReader() at db/table_cache.cc:99 #6 0x0000000003156329 in rocksdb::TableCache::NewIterator() at db/table_cache.cc:198 #7 0x0000000003166568 in rocksdb::VersionSet::MakeInputIterator() at db/version_set.cc:3345 #8 0x000000000324a94f in rocksdb::CompactionJob::ProcessKeyValueCompaction(rocksdb::CompactionJob::SubcompactionState*) () at db/compaction_job.cc:650 #9 0x000000000324c2f6 in rocksdb::CompactionJob::Run() () at db/compaction_job.cc:530 #10 0x00000000030f5ae5 in rocksdb::DBImpl::BackgroundCompaction() at db/db_impl.cc:3269 #11 0x0000000003108d36 in rocksdb::DBImpl::BackgroundCallCompaction(void*) () at db/db_impl.cc:2970 #12 0x00000000029a2a9a in facebook::logdevice::RocksDBEnv::callback(void*) () at logdevice/server/locallogstore/RocksDBEnv.cpp:26 #13 0x00000000029a2a9a in facebook::logdevice::RocksDBEnv::callback(void*) () at logdevice/server/locallogstore/RocksDBEnv.cpp:30 #14 0x00000000031e7521 in rocksdb::ThreadPool::BGThread() at util/threadpool.cc:230 #15 0x00000000031e7663 in rocksdb::BGThreadWrapper(void*) () at util/threadpool.cc:254 #16 0x00007fc85f2ea7f1 in start_thread () at /usr/local/fbcode/gcc-4.9-glibc-2.20-fb/lib/libpthread.so.0 #17 0x00007fc85e8fb46d in clone () at /usr/local/fbcode/gcc-4.9-glibc-2.20-fb/lib/libc.so.6 From looking at the code, probably what happened is this: - `TableCache::GetTableReader()` called `Env::NewRandomAccessFile()`, which dispatched to a `PosixEnv::NewRandomAccessFile()`, where probably an `open()` call failed, so the `NewRandomAccessFile()` left a nullptr in the resulting file, - `TableCache::GetTableReader()` called `NewReadaheadRandomAccessFile()` with that `nullptr` file, - it tried to call file's method and crashed. This diff is a trivial fix to this crash. Test Plan: `make -j check` Reviewers: sdong, andrewkr, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D62451 25 August 2016, 11:39:26 UTC
7c95868 Andrew Kryczka 24 August 2016, 22:42:31 UTC Thread-specific ticker statistics Summary: The global atomics we previously used for tickers had poor cache performance since they were typically updated from different threads, causing frequent invalidations. In this diff, - recordTick() updates a local ticker value specific to the thread in which it was called - When a thread exits, its local ticker value is added into merged_sum - getTickerCount() returns the sum of all threads' local ticker values and the merged_sum - setTickerCount() resets all threads' local ticker values and sets merged_sum to the value provided by the caller. In a next diff I will make a similar change for histogram stats. Test Plan: before: $ TEST_TMPDIR=/dev/shm/ perf record -g ./db_bench --benchmarks=readwhilewriting --statistics --num=1000000 --use_existing_db --threads=64 --cache_size=250000000 --compression_type=lz4 $ perf report -g --stdio | grep recordTick 7.59% db_bench db_bench [.] rocksdb::StatisticsImpl::recordTick ... after: $ TEST_TMPDIR=/dev/shm/ perf record -g ./db_bench --benchmarks=readwhilewriting --statistics --num=1000000 --use_existing_db --threads=64 --cache_size=250000000 --compression_type=lz4 $ perf report -g --stdio | grep recordTick 1.46% db_bench db_bench [.] rocksdb::StatisticsImpl::recordTick ... Reviewers: kradhakrishnan, MarkCallaghan, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: yiwu, andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D62337 24 August 2016, 22:42:31 UTC
ea9e075 Joel Marcey 24 August 2016, 22:35:38 UTC Add initial GitHub pages infra for RocksDB documentation move and update. (#1294) This is the initial commit with the templates necessary to have our RocksDB user documentation hosted on GitHub pages. Ensure you meet requirements here: https://help.github.com/articles/setting-up-your-github-pages-site-locally-with-jekyll/#requirements Then you can run this right now by doing the following: ``` % bundle install % bundle exec jekyll serve --config=_config.yml,_config_local_dev.yml ``` Then go to: http://127.0.0.1:4000/ Obviously, this is just the skeleton. Moving forward we will do these things in separate pull requests: - Replace logos with RocksDB logos - Update the color schemes - Add current information on rocksdb.org to markdown in this infra - Migrate current Wodpress blog to Jekyll and Disqus comments - Etc. 24 August 2016, 22:35:38 UTC
2a9c971 Islam AbdelRahman 24 August 2016, 22:32:01 UTC [Flaky Test] Disable DBPropertiesTest.GetProperty Summary: Disable flaky test Test Plan: run it Reviewers: yiwu, andrewkr, kradhakrishnan, yhchiang, lightmark, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D62487 24 August 2016, 22:32:01 UTC
d76ddf3 Yi Wu 24 August 2016, 21:19:05 UTC Disable ClockCache db_crashtest Summary: Tempororily disable clock cache in db_crashtest while we investigate data race issue with clock cache. Test Plan: python ./tools/db_crashtest.py blackbox Reviewers: sdong, lightmark, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D62481 24 August 2016, 21:19:05 UTC
cec2c64 Aaron Gao 24 August 2016, 01:20:41 UTC fix data race in NewIndexIterator() in block_based_table_reader.cc Summary: fixed data race described in https://github.com/facebook/rocksdb/issues/1267 and add regression test Test Plan: ./table_test --gtest_filter=BlockBasedTableTest.NewIndexIteratorLeak make all check -j64 core dump before fix. ok after fix. Reviewers: andrewkr, sdong Reviewed By: sdong Subscribers: igor, andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D62361 24 August 2016, 01:20:41 UTC
badbff6 Yi Wu 23 August 2016, 20:53:49 UTC Not insert into block cache if cache is full and not holding handle Summary: We used to allow insert into full block cache as long as `strict_capacity_limit=false`. This diff further restrict insert to full cache if caller don't intent to hold handle to the cache entry after insert. Hope this diff fix the assertion failure with db_stress: https://our.intern.facebook.com/intern/sandcastle/log/?instance_id=211853102&step_id=2475070014 db_stress: util/lru_cache.cc:278: virtual void rocksdb::LRUCacheShard::Release(rocksdb::Cache::Handle*): Assertion `lru_.next == &lru_' failed. The assertion at lru_cache.cc:278 can fail when an entry is inserted into full cache and stay in LRU list. Test Plan: make all check Reviewers: IslamAbdelRahman, lightmark, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D62325 23 August 2016, 20:53:49 UTC
4a16c32 Yi Wu 23 August 2016, 20:44:13 UTC Option to cache index/filter blocks with priority Summary: Add option to block based table to insert index/filter blocks to block cache with priority. Combined with LRUCache with high_pri_pool_ratio, we can reserved space for index/filter blocks, make them less likely to be evicted. Depends on D61977. Test Plan: See unit test. Reviewers: lightmark, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, march, leveldb Differential Revision: https://reviews.facebook.net/D62241 23 August 2016, 20:44:13 UTC
99c4af7 Yi Wu 23 August 2016, 20:40:47 UTC Make ClockCache available with TSAN build Summary: ..to unblock TSAN contbuild. Test Plan: COMPILE_WITH_TSAN=1 make db_stress -j64 ./db_stress --use_clock_cache=true Reviewers: sdong, lightmark, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D62397 23 August 2016, 20:40:47 UTC
f57bc1d Andrew Kryczka 23 August 2016, 20:34:56 UTC Fix lambda expression for clang/windows Summary: make the variables static so capturing is unnecessary since I couldn't find a portable way to capture variables in a lambda that's converted to a C-style pointer-to-function. Test Plan: https://ci.appveyor.com/project/Facebook/rocksdb/build/1.0.1658 Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D62403 23 August 2016, 20:34:56 UTC
5440675 Andrew Kryczka 23 August 2016, 17:20:00 UTC Fix lambda capture expression for windows Summary: there was an error when accessing kItersPerThread in the lambda: https://ci.appveyor.com/project/Facebook/rocksdb/build/1.0.1654 Test Plan: doitlive Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D62379 23 August 2016, 17:20:00 UTC
6584cec Andrew Kryczka 22 August 2016, 22:37:39 UTC Fold function for thread-local data Summary: This function allows the user to provide a custom function to fold all threads' local data. It will be used in my next diff for aggregating statistics stored in thread-local data. Note the test case uses atomics as thread-local values due to the synchronization requirement (documented in code). Test Plan: unit test Reviewers: yhchiang, sdong, kradhakrishnan Reviewed By: kradhakrishnan Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D62049 22 August 2016, 22:37:39 UTC
817eeb2 Adam Retter 22 August 2016, 18:02:31 UTC Add singleDelete to RocksJava (#1275) * Rename RocksDB#remove -> RocksDB#delete to match C++ API; Added deprecated versions of RocksDB#remove for backwards compatibility. * Add missing experimental feature RocksDB#singleDelete 22 August 2016, 18:02:31 UTC
ffdf6ee Adam Retter 22 August 2016, 18:01:42 UTC Add Status to RocksDBException so that meaningful function result Status from the C++ API isn't lost (#1273) 22 August 2016, 18:01:42 UTC
ecf9003 Andrew Kryczka 22 August 2016, 17:27:50 UTC Fix bug in printing values for block-based table Summary: value is not an InternalKey, we do not need to decode it Test Plan: setup: $ ldb put --create_if_missing=true k v $ ldb put --db=./tmp --create_if_missing k v $ ldb compact --db=./tmp before: $ sst_dump --command=raw --file=./tmp/000004.sst ... terminate called after throwing an instance of 'std::length_error' after: $ ./sst_dump --command=raw --file=./tmp/000004.sst $ cat tmp/000004_dump.txt ... ASCII k : v ... Reviewers: sdong, yhchiang, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D62301 22 August 2016, 17:27:50 UTC
72f8cc7 Yi Wu 19 August 2016, 23:43:31 UTC LRU cache mid-point insertion Summary: Add mid-point insertion functionality to LRU cache. Caller of `Cache::Insert()` can set an additional parameter to make a cache entry have higher priority. The LRU cache will reserve at most `capacity * high_pri_pool_pct` bytes for high-pri cache entries. If `high_pri_pool_pct` is zero, the cache degenerates to normal LRU cache. Context: If we are to put index and filter blocks into RocksDB block cache, index/filter block can be swap out too early. We want to add an option to RocksDB to reserve some capacity in block cache just for index/filter blocks, to mitigate the issue. In later diffs I'll update block based table reader to use the interface to cache index/filter blocks at high priority, and expose the option to `DBOptions` and make it dynamic changeable. Test Plan: unit test. Reviewers: IslamAbdelRahman, sdong, lightmark Reviewed By: lightmark Subscribers: andrewkr, dhruba, march, leveldb Differential Revision: https://reviews.facebook.net/D61977 19 August 2016, 23:43:31 UTC
6a17b07 Islam AbdelRahman 19 August 2016, 23:17:56 UTC Add TablePropertiesCollector support in SstFileWriter Summary: Update SstFileWriter to use user TablePropertiesCollectors that are passed in Options Test Plan: unittests Reviewers: sdong Reviewed By: sdong Subscribers: jkedgar, andrewkr, hermanlee4, dhruba, yoshinorim Differential Revision: https://reviews.facebook.net/D62253 19 August 2016, 23:17:56 UTC
78837f5 Wanning Jiang 19 August 2016, 22:10:31 UTC TableBuilder / TableReader support for range deletion Summary: 1. Range Deletion Tombstone structure 2. Modify Add() in table_builder to make it usable for adding range del tombstones 3. Expose NewTombstoneIterator() API in table_reader Test Plan: table_test.cc (now BlockBasedTableBuilder::Add() only accepts InternalKey. I make table_test only pass InternalKey to BlockBasedTableBuidler. Also test writing/reading range deletion tombstones in table_test ) Reviewers: sdong, IslamAbdelRahman, lightmark, andrewkr Reviewed By: andrewkr Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D61473 19 August 2016, 22:10:31 UTC
4cc37f5 Yi Wu 19 August 2016, 19:28:19 UTC Introduce ClockCache Summary: Clock-based cache implemenetation aim to have better concurreny than default LRU cache. See inline comments for implementation details. Test Plan: Update cache_test to run on both LRUCache and ClockCache. Adding some new tests to catch some of the bugs that I fixed while implementing the cache. Reviewers: kradhakrishnan, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D61647 19 August 2016, 19:28:19 UTC
ff17a2a Yi Wu 18 August 2016, 17:44:29 UTC Adding TBB as dependency. Summary: Splitting the makefile part of D55581. Test Plan: make all check -j32 ROCKSDB_FBCODE_BUILD_WITH_481=1 make all check -j32 ROCKSDB_NO_FBCODE=1 make all check -j32 export TBB_BASE=/mnt/gvfs/third-party2/tbb/afa54b33cfcf93f1d90a3160cdb894d6d63d5dca/4.0_update2/gcc-4.9-glibc-2.20/e9936bf; ROCKSDB_NO_FBCODE=1 CFLAGS="-I $TBB_BASE/include" LDFLAGS="-L $TBB_BASE/lib -Wl,-rpath=$TBB_BASE/lib" make all check -j32 Reviewers: IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: kradhakrishnan, yhchiang, IslamAbdelRahman, andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D56979 18 August 2016, 17:44:29 UTC
49d88be Jay 18 August 2016, 01:48:43 UTC c abi: allow compaction filter ignore snapshot (#1268) close #1262 18 August 2016, 01:48:43 UTC
0b63f51 Alexander Jipa 17 August 2016, 23:59:22 UTC fixes 1215: execute_process(COMMAND mkdir ${DIR}) fails to create a directory with cmake on Windows (#1219) CMake - Use a platform-neutral mkdir function 17 August 2016, 23:59:22 UTC
3981345 Dmitri Smirnov 17 August 2016, 07:42:35 UTC Small nits (#1280) * Create rate limiter using factory function in the test. * Convert function local statics in option helper to a C array that does not perform dynamic memory allocation. This is helpful when you try to memory isolate different DB instances. 17 August 2016, 07:42:35 UTC
2a2ebb6 Yi Wu 16 August 2016, 21:43:41 UTC Move LRUCache structs to lru_cache.h header Summary: ... so that I can include the header and create LRUCache specific tests for D61977 Test Plan: make check Reviewers: lightmark, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D62145 16 August 2016, 21:43:41 UTC
2fc2fd9 Anirban Rahut 16 August 2016, 15:21:43 UTC Single Delete Mismatch and Fallthrough statistics Summary: Added 2 statistics in compaction job statistics, to identify if single deletes are not meeting a matching key (fallthrough) or single deletes are meeting a merge, delete or another single delete (i.e. not the expected case of put). Test Plan: Tested the statistics using write_stress and compaction_job_stats_test Reviewers: sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D61749 16 August 2016, 15:21:43 UTC
3771e37 Andrew Kryczka 16 August 2016, 15:16:04 UTC WriteBatch support for range deletion Summary: Add API to WriteBatch to store range deletions in its buffer which are later added to memtable. In the WriteBatch buffer, a range deletion is encoded as "<optype><CF ID (optional)><begin key><end key>". With this diff, the range tombstones are stored inline with the data in the memtable. It's useful for now because the test cases rely on the data being accessible via memtable. My next step is to store range tombstones in a separate area in the memtable. Test Plan: unit tests Reviewers: IslamAbdelRahman, sdong, wanning Reviewed By: wanning Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D61401 16 August 2016, 15:16:04 UTC
back to top