Skip to main content

Browse the archive

https://github.com/facebook/rocksdb
17 June 2024, 23:04:45 UTC
sort by:
Revision Author Date Message Commit Date
c03f246 krad 20 July 2015, 19:37:54 UTC Relax assertions in unit DropWrites to be more permissible Summary: This unit test is blocking our release since it fails under certain compiler versions. The failure is due to a race in the unit test and not the core functionality. Test Plan: Run locally Reviewers: sdong CC: leveldb Task ID: #7760955 Blame Rev: (cherry picked from commit a75f23eb871b4981d9e784f763e08dcc3e8c0437) Conflicts: db/db_test.cc 21 July 2015, 00:11:02 UTC
9de6ee2 agiardullo 17 July 2015, 00:33:45 UTC Tag 3.12.1 17 July 2015, 00:36:26 UTC
d99227d sdong 16 July 2015, 02:58:28 UTC Fix data loss after DB recovery by not allowing flush/compaction to be scheduled until DB opened Summary: Previous run may leave some SST files with higher file numbers than manifest indicates. Compaction or flush may start to run while DB::Open() is still going on. SST file garbage collection may happen interleaving with compaction or flush, and overwrite files generated by compaction of flushes after they are generated. This might cause data loss. This possibility of interleaving is recently introduced. Fix it by not allowing compaction or flush to be scheduled before DB::Open() finishes. Test Plan: Add a unit test. This verification will have a chance to fail without the fix but doesn't fix without the fix. Reviewers: kradhakrishnan, anthony, yhchiang, IslamAbdelRahman, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D42399 16 July 2015, 21:57:50 UTC
28d3aa3 Yueh-Hsuan Chiang 07 July 2015, 19:39:36 UTC Update HISTORY.md for Listener Summary: Update HISTORY.md for Listener Test Plan: no code change Reviewers: igor, sdong, IslamAbdelRahman, anthony Reviewed By: anthony Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D41325 (cherry picked from commit 59b50dcef904b86dad7fe1c45a22ff59dbc2a865) 07 July 2015, 21:20:58 UTC
79c2075 Yueh-Hsuan Chiang 07 July 2015, 19:10:10 UTC fixed leaking log::Writers Summary: Fixes valgrind errors in column_family_test. Test Plan: `make check`, `make valgrind_check` Reviewers: igor, yhchiang Reviewed By: yhchiang Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D41181 (cherry picked from commit 4ce5be4255f654442986f0a9c0269e41cce7d362) 07 July 2015, 19:36:28 UTC
0640c28 Yueh-Hsuan Chiang 07 July 2015, 18:36:24 UTC Revert two diffs related to DBIter::FindPrevUserKey() Summary: This diff reverts the following two previous diffs related to DBIter::FindPrevUserKey(), which makes db_stress unstable. We should bake a better fix for this. * "Fix a comparison in DBIter::FindPrevUserKey()" ec70fea4c4025351190eba7a02bd09bb5f083790. * "Fixed endless loop in DBIter::FindPrevUserKey()" acee2b08a2d37154b8f9e2dc74b1966202c15ec5. Test Plan: db_stress Reviewers: anthony, igor, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D41301 (cherry picked from commit 685582a0b43cb19cb12bf7122657f9ea0f8c52c4) 07 July 2015, 19:36:11 UTC
7183094 Aaron Feldman 07 July 2015, 17:49:16 UTC Initialize threads later in constructor Summary: This addresses a test failure where an exception occured in the constructor's call to CreateDirIfMissing(). The existence of unjoined threads prevented this exception from propogating properly. See http://stackoverflow.com/questions/7381757/c-terminate-called-without-an-active-exception Test Plan: Re-run tests from task #7626266 Reviewers: sdong, anthony, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D41313 (cherry picked from commit e12b403991217c5472135a1e2f8faad1b915d011) 07 July 2015, 19:35:58 UTC
bc672ae Igor Canadi 06 July 2015, 18:14:08 UTC Fix compaction_job_test Summary: Two issues: * the input keys to the compaction don't include sequence number. * sequence number is set to max(seq_num), but it should be set to max(seq_num)+1, because the condition here is strictly-larger (i.e. we will only zero-out sequence number if the DB's sequence number is strictly greater than the key's sequence number): https://github.com/facebook/rocksdb/blob/master/db/compaction_job.cc#L830 Test Plan: make compaction_job_test && ./compaction_job_test Reviewers: sdong, lovro Reviewed By: lovro Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D41247 (cherry picked from commit 155ce60dafbc3841da73eedf0d9dc6c4fabd8404) 07 July 2015, 19:35:36 UTC
b565228 Yueh-Hsuan Chiang 02 July 2015, 23:10:31 UTC Fixed endless loop in DBIter::FindPrevUserKey() Summary: Fixed endless loop in DBIter::FindPrevUserKey() Test Plan: ./db_stress --test_batches_snapshots=1 --threads=32 --write_buffer_size=4194304 --destroy_db_initially=0 --reopen=20 --readpercent=45 --prefixpercent=5 --writepercent=35 --delpercent=5 --iterpercent=10 --db=/tmp/rocksdb_crashtest_KdCI5F --max_key=100000000 --mmap_read=0 --block_size=16384 --cache_size=1048576 --open_files=500000 --verify_checksum=1 --sync=0 --progress_reports=0 --disable_wal=0 --disable_data_sync=1 --target_file_size_base=2097152 --target_file_size_multiplier=2 --max_write_buffer_number=3 --max_background_compactions=20 --max_bytes_for_level_base=10485760 --filter_deletes=0 --memtablerep=prefix_hash --prefix_size=7 --ops_per_thread=200 --kill_random_test=97 Reviewers: tnovak, igor, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D41085 (cherry picked from commit acee2b08a2d37154b8f9e2dc74b1966202c15ec5) 07 July 2015, 19:33:29 UTC
dc3cdfe Mike Kolupaev 02 July 2015, 21:27:00 UTC [wal changes 1/3] fixed unbounded wal growth in some workloads Summary: This fixes the following scenario we've hit: - we reached max_total_wal_size, created a new wal and scheduled flushing all memtables corresponding to the old one, - before the last of these flushes started its column family was dropped; the last background flush call was a no-op; no one removed the old wal from alive_logs_, - hours have passed and no flushes happened even though lots of data was written; data is written to different column families, compactions are disabled; old column families are dropped before memtable grows big enough to trigger a flush; the old wal still sits in alive_logs_ preventing max_total_wal_size limit from kicking in, - a few more hours pass and we run out disk space because of one huge .log file. Test Plan: `make check`; backported the new test, checked that it fails without this diff Reviewers: igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D40893 (cherry picked from commit 218487d8dc45c4cb03dbb80bd4d7031b131b9f25) 07 July 2015, 19:33:12 UTC
8856b22 Aaron Feldman 02 July 2015, 19:27:35 UTC Fix unity build by removing anonymous namespace Summary: see title Test Plan: run 'make unity' Reviewers: igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D41079 (cherry picked from commit e70115e71b74e6cce9484fcf2ded7ecb2d77ece3) 07 July 2015, 19:32:28 UTC
4159f5b agiardullo 02 July 2015, 18:08:34 UTC Prepare 3.12 Summary: About to cut release Test Plan: none Reviewers: igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D41061 02 July 2015, 19:20:36 UTC
a69bc91 Aaron Feldman 02 July 2015, 18:35:51 UTC Multithreaded backup and restore in BackupEngineImpl Summary: Add a new field: BackupableDBOptions.max_background_copies. CreateNewBackup() and RestoreDBFromBackup() will use this number of threads to perform copies. If there is a backup rate limit, then max_background_copies must be 1. Update backupable_db_test.cc to test multi-threaded backup and restore. Update backupable_db_test.cc to test backups when the backup environment is not the same as the database environment. Test Plan: Run ./backupable_db_test Run valgrind ./backupable_db_test Run with TSAN and ASAN Reviewers: yhchiang, rven, anthony, sdong, igor Reviewed By: igor Subscribers: yhchiang, anthony, sdong, leveldb, dhruba Differential Revision: https://reviews.facebook.net/D40725 02 July 2015, 18:35:51 UTC
03d433e Yueh-Hsuan Chiang 02 July 2015, 06:22:03 UTC [RocksJava] Fixed test failures Summary: The option bottommost_level_compaction was introduced lately. This option breaks the Java API behavior. To prevent the library from doing so we set that option to a fixed value in Java. In future we are going to remove that portion and replace the hardcoded options using a more flexible way. Fixed bug introduced by WriteBatchWithIndex Patch Lately icanadi changed the behavior of WriteBatchWithIndex. See commit: 821cff114e57efa67711c1c1c105aa02831a0d23 This commit solves problems introduced by above mentioned commit. Test Plan: make rocksdbjava make jtest Reviewers: adamretter, ankgup87, yhchiang Reviewed By: yhchiang Subscribers: igor, dhruba Differential Revision: https://reviews.facebook.net/D40647 02 July 2015, 06:22:03 UTC
c00948d Yueh-Hsuan Chiang 01 July 2015, 23:03:56 UTC [RocksJava] Fix test failure of compactRangeToLevel Summary: Rewrite Java tests compactRangeToLevel and compactRangeToLevelColumnFamily to make them more deterministic and robust. Test Plan: make rocksdbjava make jtest Reviewers: anthony, fyrz, adamretter, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D40941 01 July 2015, 23:03:56 UTC
05e2831 sdong 30 June 2015, 21:15:24 UTC Allocate LevelFileIteratorState and LevelFileNumIterator from DB iterator's arena Summary: Try to allocate LevelFileIteratorState and LevelFileNumIterator from DB iterator's arena, instead of calling malloc and free. Test Plan: valgrind check Reviewers: rven, yhchiang, anthony, kradhakrishnan, igor Reviewed By: igor Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D40929 01 July 2015, 00:30:38 UTC
436ed90 Igor Canadi 30 June 2015, 20:30:54 UTC Add rpath option to production builds for 4.8.1 toolchain Summary: Copy change from D37533 to gcc 4.8.1 config Test Plan: make db_bench, `ldd db_bench`, try running it Reviewers: MarkCallaghan, anthony Reviewed By: anthony Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D40845 30 June 2015, 20:30:54 UTC
b0f1927 krad 29 June 2015, 23:04:24 UTC Increasing timeout for drop writes. Summary: We have a race in the way test works. We avoided the race by adding the wait to the counter. I thought 1s was eternity, but that is not true in some scenarios. Increasing the timeout to 10s and adding warnings. Also, adding nosleep to avoid the case where the wakeup thread is waiting behind the sleeping thread for scheduling. Test Plan: Run make check Reviewers: siying igorcanadi CC: leveldb@ Task ID: #7312624 Blame Rev: 30 June 2015, 18:11:56 UTC
ec70fea Tomislav Novak 26 June 2015, 20:18:27 UTC Fix a comparison in DBIter::FindPrevUserKey() Summary: When seek target is a merge key (`kTypeMerge`), `DBIter::FindNextUserEntry()` advances the underlying iterator _past_ the current key (`saved_key_`); see `MergeValuesNewToOld()`. However, `FindPrevUserKey()` assumes that `iter_` points to an entry with the same user key as `saved_key_`. As a result, `it->Seek(key) && it->Prev()` can cause the iterator to be positioned at the _next_, instead of the previous, entry (new test, written by @lovro, reproduces the bug). This diff changes `FindPrevUserKey()` to also skip keys that are _greater_ than `saved_key_`. Test Plan: db_test Reviewers: igor, sdong Reviewed By: sdong Subscribers: leveldb, dhruba, lovro Differential Revision: https://reviews.facebook.net/D40791 30 June 2015, 00:04:03 UTC
501591c Yueh-Hsuan Chiang 29 June 2015, 21:39:01 UTC Make column_family_test runnable in ROCKSDB_LITE Summary: Make column_family_test runnable in ROCKSDB_LITE. Test Plan: column_family_test Reviewers: sdong, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D40251 29 June 2015, 21:39:01 UTC
91cb82f krad 29 June 2015, 18:47:24 UTC Merge branch 'master' of github.com:facebook/rocksdb 29 June 2015, 18:47:24 UTC
09f5a4b Igor Canadi 29 June 2015, 18:43:25 UTC set -e in fb_compile_mongo.sh Summary: Based on @anthony's feedback, we want to fail early if our static linking fails. Test Plan: none Reviewers: anthony Reviewed By: anthony Subscribers: dhruba, anthony, leveldb Differential Revision: https://reviews.facebook.net/D40839 29 June 2015, 18:43:25 UTC
6199cba krad 25 June 2015, 20:32:32 UTC Fix race in unit test. Summary: Avoid falling victim to race condition. Test Plan: Run the unit test Reviewers: sdong igor CC: leveldb@ Task ID: #7312624 Blame Rev: 29 June 2015, 18:40:21 UTC
0a019d7 Igor Canadi 26 June 2015, 18:48:09 UTC Use malloc_usable_size() for accounting block cache size Summary: Currently, when we insert something into block cache, we say that the block cache capacity decreased by the size of the block. However, size of the block might be less than the actual memory used by this object. For example, 4.5KB block will actually use 8KB of memory. So even if we configure block cache to 10GB, our actually memory usage of block cache will be 20GB! This problem showed up a lot in testing and just recently also showed up in MongoRocks production where we were using 30GB more memory than expected. This diff will fix the problem. Instead of counting the block size, we will count memory used by the block. That way, a block cache configured to be 10GB will actually use only 10GB of memory. I'm using non-portable function and I couldn't find info on portability on Google. However, it seems to work on Linux, which will cover majority of our use-cases. Test Plan: 1. fill up mongo instance with 80GB of data 2. restart mongo with block cache size configured to 10GB 3. do a table scan in mongo 4. memory usage before the diff: 12GB. memory usage after the diff: 10.5GB Reviewers: sdong, MarkCallaghan, rven, yhchiang Reviewed By: yhchiang Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D40635 26 June 2015, 18:48:09 UTC
4cbc4e6 Igor Canadi 26 June 2015, 18:35:46 UTC Call merge operators with empty values Summary: It's not really nice to call user's API with garbage data in new_value. This diff makes sure that new_value is empty before calling the merge operator. Test Plan: Added assert to Merge operator in merge_test Reviewers: sdong, yhchiang Reviewed By: yhchiang Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D40773 26 June 2015, 18:35:46 UTC
619167e Igor Canadi 26 June 2015, 17:29:24 UTC Fix mac compile Summary: as title Test Plan: make check Reviewers: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D40785 26 June 2015, 17:29:24 UTC
472e64d Igor Canadi 26 June 2015, 17:24:08 UTC Improve fb_compile_mongo.sh Summary: If we create a new temp directory for each build, scons will recompile everything because we have different parameters. Instead, let's set up a constant path to our static lib. That way we won't have to recompile. Test Plan: Run fb_compile_mongo.sh twice -- second time it didn't recompile everything Reviewers: MarkCallaghan, anthony Reviewed By: anthony Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D40707 26 June 2015, 17:24:08 UTC
c9cd404 Venkatesh Radhakrishnan 25 June 2015, 21:43:25 UTC Make flush check for shutdown Summary: Fixes task 7156865 where a compaction causes a hang in flush memtable if CancelAllBackgroundWork was called prior to it. Stack trace is in : https://phabricator.fb.com/P19848829 We end up waiting for a flush which will never happen because there are no background threads. Test Plan: PreShutdownFlush Reviewers: sdong, igor Reviewed By: sdong, igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D40617 25 June 2015, 21:43:25 UTC
4fb09c6 Poornima Chozhiyath Raman 25 June 2015, 16:44:30 UTC Updating SeekToLast with upper bound Summary: #7124486: RocksDB's Iterator.SeekToLast should seek to the last key before iterate_upper_bound if presents Test Plan: ./db_iter_test run successfully with the new testcase Reviewers: rven, yhchiang, igor, anthony, kradhakrishnan, sdong Reviewed By: sdong Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D40425 25 June 2015, 16:44:30 UTC
dadc429 Igor Canadi 24 June 2015, 22:09:55 UTC Reproducible MongoRocks compile with FB toolchain Summary: Added a script that will compile MongoRocks with the same flags as RocksDB binary. On FB infra, we can now do: cd ~/rocksdb; make static_lib cd ~/mongo; ~/rocksdb/build_tools/fb_compile_mongo.sh No need to upgrade the g++ on the devbox (like Aaron and I did) or maintain a separate script to compile (like Mark did) fb_compile_mongo.sh gets the settings from fbcode_config.sh, so it also makes it easier to upgrade the environment one day. Test Plan: Compiled mongod with new script. Also, ldd output looks good: https://phabricator.fb.com/P19891602 Reviewers: AaronFeldman, MarkCallaghan, anthony Reviewed By: anthony Subscribers: anthony, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D40659 24 June 2015, 22:09:55 UTC
62a8fd1 Yueh-Hsuan Chiang 24 June 2015, 22:01:43 UTC Make stringappend_test runnable in ROCKSDB_LITE Summary: Make stringappend_test runnable in ROCKSDB_LITE Test Plan: stringappend_test Reviewers: sdong, rven, anthony, kradhakrishnan, IslamAbdelRahman, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D40593 24 June 2015, 22:01:43 UTC
48da7a9 Yueh-Hsuan Chiang 24 June 2015, 22:00:51 UTC Improve the comment for BYTES_READ in statistics. Summary: BYTES_READ only count the number of logical bytes read from the DB::Get() function. It neither includes all logical bytes read nor indicates IO read bytes. This patch improves the comment for BYTES_READ. Test Plan: Only change comment. Reviewers: sdong, rven, anthony, kradhakrishnan, IslamAbdelRahman, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D40599 24 June 2015, 22:00:51 UTC
72cab88 Yueh-Hsuan Chiang 24 June 2015, 08:44:21 UTC Block redis_test in ROCKSDB_LITE Summary: Block redis_test in ROCKSDB_LITE as utilities not supported in ROCKSDB_LITE. Test Plan: redis_test Reviewers: sdong, igor, rven, anthony, kradhakrishnan, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D40587 24 June 2015, 08:44:21 UTC
dec2c9f Yueh-Hsuan Chiang 24 June 2015, 08:38:53 UTC Make table_properties_collector_test runnable in ROCKSDB_LITE Summary: Make table_properties_collector_test runnable in ROCKSDB_LITE Test Plan: table_properties_collector_test Reviewers: sdong, rven, anthony, kradhakrishnan, IslamAbdelRahman, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D40581 24 June 2015, 08:38:53 UTC
0b1ffe2 Yueh-Hsuan Chiang 23 June 2015, 23:32:59 UTC Remove -Wl,--no-as-needed flag when making shared_lib in OSX and IOS Summary: Remove -Wl,--no-as-needed flag when making shared_lib in OSX and IOS as those environment doe not have compile option --no-as-needed ld: unknown option: --no-as-needed clang: error: linker command failed with exit code 1 (use -v to see invocation) Test Plan: make shared_lib Reviewers: meyering, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D40353 23 June 2015, 23:32:59 UTC
674b118 Islam AbdelRahman 23 June 2015, 20:32:40 UTC Bottommost level compaction option Summary: Replace force_bottommost_level_compaction in CompactRangeOption with an option that allow the user to (always skip, always compact, compact if compaction filter is present) the bottommost level for level based compaction. Test Plan: make check Reviewers: sdong, yhchiang, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D40527 23 June 2015, 20:32:40 UTC
782a159 Giuseppe Ottaviano 23 June 2015, 17:25:45 UTC Implement a table-level row cache Summary: Implementation of a table-level row cache. It only caches point queries done through the `DB::Get` interface, queries done through the `Iterator` interface will completely skip the cache. Supports snapshots and merge operations. Test Plan: Ran `make valgrind_check commit-prereq` Reviewers: igor, philipp, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D39849 23 June 2015, 17:25:45 UTC
de85e4c krad 15 June 2015, 19:03:13 UTC Introduce WAL recovery consistency levels Summary: The "one size fits all" approach with WAL recovery will only introduce inconvenience for our varied clients as we go forward. The current recovery is a bit heuristic. We introduce the following levels of consistency while replaying the WAL. 1. RecoverAfterRestart (kTolerateCorruptedTailRecords) This mocks the current recovery mode. 2. RecoverAfterCleanShutdown (kAbsoluteConsistency) This is ideal for unit test and cases where the store is shutdown cleanly. We tolerate no corruption or incomplete writes. 3. RecoverPointInTime (kPointInTimeRecovery) This is ideal when using devices with controller cache or file systems which can loose data on restart. We recover upto the point were is no corruption or incomplete write. 4. RecoverAfterDisaster (kSkipAnyCorruptRecord) This is ideal mode to recover data. We tolerate corruption and incomplete writes, and we hop over those sections that we cannot make sense of salvaging as many records as possible. Test Plan: (1) Run added unit test to cover all levels. (2) Run make check. Reviewers: leveldb, sdong, igor Subscribers: yoshinorim, dhruba Differential Revision: https://reviews.facebook.net/D38487 22 June 2015, 22:28:12 UTC
530534f Islam AbdelRahman 22 June 2015, 22:20:30 UTC Fix trivial move merge Summary: Fixing bad merge Test Plan: make -j64 check (this is not enough to verify the fix) Reviewers: igor, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D40521 22 June 2015, 22:20:30 UTC
7015fd8 krad 19 June 2015, 18:25:33 UTC Add read_nanos to IOStatsContext. Summary: MyRocks need a mechanism to track read outliers. We need to expose this stat. Test Plan: None Reviewers: sdong CC: leveldb Task ID: #7152512 Blame Rev: 22 June 2015, 18:09:35 UTC
7160f5d Aaron Feldman 22 June 2015, 16:31:52 UTC Fix broken gflags link Summary: Fix broken gflags link Test Plan: Follow the link Reviewers: igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D40503 22 June 2015, 16:31:52 UTC
e1d3c7d Venkatesh Radhakrishnan 20 June 2015, 03:21:23 UTC Fixing valgrind error in checkpoint_test Summary: Fixed a valgrind issue in checkpoint_test Test Plan: valgrind on checkpoint_test Reviewers: igor, anthony Reviewed By: anthony Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D40455 20 June 2015, 03:21:23 UTC
3bdec09 Michael Callahan 20 June 2015, 00:41:49 UTC Remove ldb_tests.py from make check until it is working again. Summary: Recent checkin added ldb_test.py to the make check target but the test fails. Remove it again for now and make task. Test Plan: No more ldb_tests.py running Reviewers: igor, anthony Reviewed By: anthony Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D40449 20 June 2015, 00:41:49 UTC
15325bf Michael Callahan 19 June 2015, 23:24:36 UTC First version of rocksdb_dump and rocksdb_undump. Summary: Hack up rocksdb_dump and rocksdb_undump utilities to get this task rolling/promote discussion. Test Plan: Dump/undump databases recursively to see if nothing is lost. Reviewers: sdong, yhchiang, rven, anthony, kradhakrishnan, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D37269 19 June 2015, 23:24:36 UTC
04251e1 Venkatesh Radhakrishnan 19 June 2015, 23:08:31 UTC Add wal files to Checkpoint for multiple column families. Summary: When there are multiple column families, the flush in GetLiveFiles is not atomic, so that there are entries in the wal files which are needed to get a consisten RocksDB. We now add the log files to the checkpoint. Test Plan: CheckpointCF - This test forces more data to be written to the other column families after the flush of the first column family but before the second. Reviewers: igor, yhchiang, IslamAbdelRahman, anthony, kradhakrishnan, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D40323 19 June 2015, 23:08:31 UTC
18cc501 Aaron Feldman 19 June 2015, 16:43:08 UTC Fix memory leaks in PinnedUsageTest Summary: See title Test Plan: Run valgrind ./cache_test Reviewers: igor Reviewed By: igor Subscribers: anthony, dhruba Differential Revision: https://reviews.facebook.net/D40419 19 June 2015, 16:43:08 UTC
bf03f59 Igor Canadi 19 June 2015, 01:46:26 UTC Disable CompressLevelCompaction() if Zlib is not supported Summary: CompressLevelCompaction() depends on Zlib. We should skip it when zlib is not present. Test Plan: `make check` without zlib Reviewers: yhchiang Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D40401 19 June 2015, 01:46:26 UTC
df719d4 Yueh-Hsuan Chiang 18 June 2015, 22:58:00 UTC Make autovector_test runnable in ROCKSDB_LITE Summary: Make autovector_test runnable in ROCKSDB_LITE Test Plan: autovector_test Reviewers: sdong, rven, anthony, kradhakrishnan, IslamAbdelRahman, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D40245 18 June 2015, 22:58:00 UTC
4d6d476 Yueh-Hsuan Chiang 18 June 2015, 22:57:19 UTC Block geodb_test in ROCKSDB_LITE Summary: Block geodb_test in ROCKSDB_LITE as geodb is not supported in ROCKSDB_LITE Test Plan: geodb_test Reviewers: sdong, rven, anthony, kradhakrishnan, IslamAbdelRahman, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D40335 18 June 2015, 22:57:19 UTC
71b438c Yueh-Hsuan Chiang 18 June 2015, 22:54:52 UTC Remove unused target --- compactor_test Summary: Remove compactor_test, which depends on a directory not exist in our code base. make compactor_test GEN util/build_version.cc GEN util/build_version.cc make: *** No rule to make target `utilities/compaction/compactor_test.o', needed by `compactor_test'. Stop. Test Plan: verify the output message of make compactor_test Reviewers: rven, anthony, kradhakrishnan, igor, IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D40341 18 June 2015, 22:54:52 UTC
eade498 Yueh-Hsuan Chiang 18 June 2015, 22:54:05 UTC Block utilities/write_batch_with_index in ROCKSDB_LITE Summary: Block utilities/write_batch_with_index in ROCKSDB_LITE as we don't include anly utilities in ROCKSDB_LITE Test Plan: write_batch_with_index_test Reviewers: rven, anthony, kradhakrishnan, IslamAbdelRahman, igor, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D40347 18 June 2015, 22:54:05 UTC
760e9a9 Igor Canadi 18 June 2015, 21:55:05 UTC Fail DB::Open() when the requested compression is not available Summary: Currently RocksDB silently ignores this issue and doesn't compress the data. Based on discussion, we agree that this is pretty bad because it can cause confusion for our users. This patch fails DB::Open() if we don't support the compression that is specified in the options. Test Plan: make check with LZ4 not present. If Snappy is not present all tests will just fail because Snappy is our default library. We should make Snappy the requirement, since without it our default DB::Open() fails. Reviewers: sdong, MarkCallaghan, rven, yhchiang Reviewed By: yhchiang Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D39687 18 June 2015, 21:55:05 UTC
69bb210 Aaron Feldman 18 June 2015, 20:56:31 UTC Add Cache.GetPinnedUsageUsage() Summary: Add the funcion Cache.GetPinnedUsage() to return the memory size of entries that are in use by the system (that is, all the entries not in the LRU list). Test Plan: Run ./cache_test and examine PinnedUsageTest. Reviewers: tnovak, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D40305 18 June 2015, 20:56:31 UTC
4eabbdb Islam AbdelRahman 18 June 2015, 18:03:31 UTC Skip bottommost level compaction if possible Summary: This is https://reviews.facebook.net/D39999 but after introducing an option to force compaction the bottom most level Changes in this patch - Introduce force_bottommost_level_compaction to CompactRangeOptions that force compacting bottommost level during compaction - Skip bottommost level compaction if we dont have a compaction filter and force_bottommost_level_compaction options is not set Although tests pass on my machine but I suspect that there maybe some tests that I am not aware of that should use force_bottommost_level_compaction to pass in a deterministic way Test Plan: make check adding new tests Reviewers: igor, sdong, yhchiang Reviewed By: yhchiang Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D40059 18 June 2015, 18:03:31 UTC
4b8bb62 Igor Canadi 18 June 2015, 17:15:54 UTC Don't dump DBOptions for each column family Summary: Currently we dump DBOptions for each column family options we dump. This leads to duplicate lines in our LOG file. This diff fixes that. Test Plan: Check out the LOG Reviewers: sdong, rven, yhchiang Reviewed By: yhchiang Subscribers: IslamAbdelRahman, yoshinorim, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D39729 18 June 2015, 17:15:54 UTC
176f0be Poornima Chozhiyath Raman 18 June 2015, 17:12:55 UTC Merge branch 'master' of github.com:facebook/rocksdb D40233: Replace %llu with format macros in ParsedInternalKey::DebugString()) 18 June 2015, 17:12:55 UTC
bb1c74c Yueh-Hsuan Chiang 18 June 2015, 06:40:34 UTC Fixed a bug of CompactionStats in multi-level universal compaction case Summary: Universal compaction can involves in multiple levels. However, the current implementation of bytes_readn and bytes_readnp1 (and some other stats with postfix `n` and `np1`) assumes compaction can only have two levels. This patch fixes this bug and redefines bytes_readn and bytes_readnp1: * bytes_readnp1: the number of bytes read in the compaction output level. * bytes_readn: the total number of bytes read minus bytes_readnp1 Test Plan: Add a test in compaction_job_stats_test Reviewers: igor, sdong, rven, anthony, kradhakrishnan, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D40239 18 June 2015, 06:40:34 UTC
a66b815 Poornima Chozhiyath Raman 18 June 2015, 03:44:52 UTC Merge branch 'master' of github.com:facebook/rocksdb D40233: Replace %llu with format macros in ParsedInternalKey::DebugString()) 18 June 2015, 03:44:52 UTC
f06be62 Poornima Chozhiyath Raman 17 June 2015, 05:26:34 UTC Replace %llu with format macros in ParsedInternalKey::DebugString()) Test Plan: successfully compiled the code Reviewers: sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D40233 18 June 2015, 03:44:26 UTC
2dc3910 Igor Canadi 17 June 2015, 23:44:52 UTC Add --benchmark_write_rate_limit option to db_bench Summary: So far, we benchmarked RocksDB by writing as fast as possible. With this change, we're able to limit our write throughput, which should help us better understand how RocksDB performes under varying write workloads. Specifically, I'm currently interested in the shape of the graph that has write throughput on one axis and write rate on another. This should help us with designing our stall system, as we have started to do with D36351. Test Plan: $ ./db_bench --benchmarks=fillrandom --benchmark_write_rate_limit=1000000 fillrandom : 118.523 micros/op 8437 ops/sec; 0.9 MB/s $ ./db_bench --benchmarks=fillrandom --benchmark_write_rate_limit=2000000 fillrandom : 59.136 micros/op 16910 ops/sec; 1.9 MB/s Reviewers: MarkCallaghan, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D39759 17 June 2015, 23:44:52 UTC
12e030a Islam AbdelRahman 17 June 2015, 21:36:14 UTC Use CompactRangeOptions for CompactRange Summary: This diff update DB::CompactRange to use RangeCompactionOptions instead of using multiple parameters Old CompactRange is still available but deprecated Test Plan: make all check make rocksdbjava USE_CLANG=1 make all OPT=-DROCKSDB_LITE make release Reviewers: sdong, yhchiang, igor Reviewed By: igor Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D40209 17 June 2015, 21:36:14 UTC
c89369f Igor Canadi 17 June 2015, 21:09:12 UTC Move dockerbuild.sh to build_tools/ Summary: That's where we keep build tools :) Test Plan: none Reviewers: sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D39741 17 June 2015, 21:09:12 UTC
4716ab4 Igor Canadi 17 June 2015, 20:16:20 UTC Merge pull request #638 from HolodovAlexander/master C api: human-readable statistics 17 June 2015, 20:16:20 UTC
25d6005 Igor Canadi 17 June 2015, 19:37:59 UTC Clean up InstallSuperVersion Summary: We go to great lengths to make sure MaybeScheduleFlushOrCompaction() is called outside of write thread. But anyway, it's still called in the mutex, so it's not that much cheaper. This diff removes the "optimization" and cleans up the code a bit. Test Plan: make check Reviewers: rven, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D40113 17 June 2015, 19:37:59 UTC
1369f01 Yueh-Hsuan Chiang 17 June 2015, 18:21:18 UTC Only initialize the ThreadStatusData when necessary. Summary: Before this patch, any function call to ThreadStatusUtil might automatically initialize and register the thread status data. However, if it is the user-thread making this call, the allocated thread-status-data will never be released as such threads are not managed by rocksdb. In this patch, I remove the automatic-initialization part. Thread-status data is only initialized and uninitialized in Env during the thread creation and destruction. Test Plan: db_test thread_list_test listener_test Reviewers: igor, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D40017 17 June 2015, 18:21:18 UTC
1a08d0b Yueh-Hsuan Chiang 17 June 2015, 17:54:51 UTC Block c_test in ROCKSDB_LITE Summary: Block c_test in ROCKSDB_LITE as it's not supported in ROCKSDB_LITE. Test Plan: c_test Reviewers: sdong, rven, anthony, kradhakrishnan, IslamAbdelRahman, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D40257 17 June 2015, 17:54:51 UTC
40f562e sdong 13 June 2015, 01:04:30 UTC Allow GetApproximateSize() to include mem table size if it is skip list memtable Summary: Add an option in GetApproximateSize() so that the result will include estimated sizes in mem tables. To implement it, implement an estimated count from the beginning to a key in skip list. The approach is to count to find the entry, how many Next() is issued from each level, and sum them with a weight that is <branching factor> ^ <level>. Test Plan: Add a test case Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D40119 17 June 2015, 01:13:23 UTC
d59d90b Igor Canadi 12 June 2015, 21:31:53 UTC db_bench periodically writes QPS to CSV file Summary: This is part of an effort to better understand and optimize RocksDB stalls under high load. I added a feature to db_bench to periodically write QPS to CSV files. That way we can nicely see how our QPS changes in time (especially when DB is stalled) and can do a better job of evaluating our stall system (i.e. we want the QPS to be as constant as possible, as opposed to having bunch of stalls) Cool part of CSV files is that we can easily graph them -- there are a bunch of tools available. Test Plan: Ran ./db_bench --report_interval_seconds=10 --benchmarks=fillrandom --num=10000000 and observed this in report.csv: secs_elapsed,interval_qps 10,2725860 20,1980480 30,1863456 40,1454359 50,1460389 Reviewers: sdong, MarkCallaghan, rven, yhchiang Reviewed By: yhchiang Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D40047 12 June 2015, 21:31:53 UTC
46296cc sdong 12 June 2015, 20:54:29 UTC Cygwin build not to use -fPIC Summary: Cygwin doesn't support -fPIC. Remove it. Not sure whether we can build shared library in Cygwin but at least it can build without warning. Test Plan: Build under Cygwin Reviewers: yhchiang, rven, kradhakrishnan, igor Reviewed By: igor Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D40077 12 June 2015, 20:59:59 UTC
bee8d03 Yueh-Hsuan Chiang 12 June 2015, 17:45:02 UTC Removed two unused macros in iostats_context Summary: Removed two unused macros in iostats_context Test Plan: make all check Reviewers: sdong, rven, IslamAbdelRahman, kradhakrishnan, igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D40005 12 June 2015, 17:45:02 UTC
5fec963 Yueh-Hsuan Chiang 12 June 2015, 17:44:07 UTC Fixed false alarm of size comparison in compaction_job_stats_test Summary: Fixed false alarm of size comparison in compaction_job_stats_test Test Plan: compaction_job_stats_test Reviewers: igor Reviewed By: igor Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D39921 12 June 2015, 17:44:07 UTC
cccd219 Islam AbdelRahman 12 June 2015, 17:43:33 UTC Revert skip bottommost compaction Summary: Reverting this diff https://reviews.facebook.net/D39999 Will add an option to force bottom most level compaction and then re submit it Test Plan: make check Reviewers: igor, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D40041 12 June 2015, 17:43:33 UTC
20f2b54 Islam AbdelRahman 12 June 2015, 16:56:08 UTC Skip bottom most level compaction if no compaction filter Summary: If we don't have a compaction filter then we can skip compacting the bottom most level Test Plan: make check added unit tests Reviewers: yhchiang, sdong, igor Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D39999 12 June 2015, 16:56:08 UTC
7842920 sdong 15 May 2015, 22:52:51 UTC Slow down writes by bytes written Summary: We slow down data into the database to the rate of options.delayed_write_rate (a new option) with this patch. The thread synchronization approach I take is to still synchronize write controller by DB mutex and GetDelay() is inside DB mutex. Try to minimize the frequency of getting time in GetDelay(). I verified it through db_bench and it seems to work hard_rate_limit is deprecated. options.delayed_write_rate is still not dynamically changeable. Need to work on it as a follow-up. Test Plan: Add new unit tests in db_test Reviewers: yhchiang, rven, kradhakrishnan, anthony, MarkCallaghan, igor Reviewed By: igor Subscribers: ikabiljo, leveldb, dhruba Differential Revision: https://reviews.facebook.net/D36351 12 June 2015, 03:42:18 UTC
a84df65 Igor Canadi 11 June 2015, 22:42:16 UTC Don't let two L0->L1 compactions run in parallel Summary: With experimental feature SuggestCompactRange() we don't restrict running two L0->L1 compactions in parallel. This diff fixes this. Test Plan: added a unit test to reproduce the failure. fixed the unit test Reviewers: yhchiang, rven, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D39981 11 June 2015, 22:42:16 UTC
d6ce0f7 Islam AbdelRahman 11 June 2015, 22:22:22 UTC Add largest sequence to FlushJobInfo Summary: Adding largest sequence number to FlushJobInfo and passing flushed file metadata to NotifyOnFlushCompleted which include alot of other values that we may want to expose in FlushJobInfo Test Plan: make check Reviewers: igor, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D39927 11 June 2015, 22:22:22 UTC
ab455ce Islam AbdelRahman 11 June 2015, 21:32:10 UTC fix clang build 11 June 2015, 21:32:10 UTC
3eddd1a Yueh-Hsuan Chiang 11 June 2015, 21:18:02 UTC Add Env::GetThreadID(), which returns the ID of the current thread. Summary: Add Env::GetThreadID(), which returns the ID of the current thread. In addition, make GetThreadList() and InfoLog use same unique ID for the same thread. Test Plan: db_test listener_test Reviewers: igor, rven, IslamAbdelRahman, kradhakrishnan, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D39735 11 June 2015, 21:18:02 UTC
73faa3d Islam AbdelRahman 11 June 2015, 21:15:52 UTC Handling edge cases for ReFitLevel Summary: Right now the level we pass to ReFitLevel is the maximum level with files (before compaction), there are multiple cases where this maximum level have changed after compaction - all files where in L0 (now maximum level is L1) - using kCompactionStyleUniversal (now maximum level in the last level) - level_compaction_dynamic_level_bytes ?? We can handle each of these cases individually, but I felt it's safer to calculate max_level_with_files again if we want to do a ReFitLevel Test Plan: adding some tests make -j64 check Reviewers: igor, sdong Reviewed By: sdong Subscribers: ott, dhruba Differential Revision: https://reviews.facebook.net/D39663 11 June 2015, 21:15:52 UTC
bffaf0a Igor Canadi 10 June 2015, 20:21:18 UTC Merge pull request #631 from mkhq/patch-1 Updated OS X instructions, replace homebrew/dupes with homebrew/versions 10 June 2015, 20:21:18 UTC
821cff1 Igor Canadi 10 June 2015, 19:57:38 UTC Re-generate WriteEntry on WBWIIterator::Entry() Summary: [This is the resubmit of D39813. Tests were failing, so I reverted the diff. I found the bug and I'm now resubmitting] If we don't do this, any calls to Entry() after WBWI mutation will result in undefined behavior. We need to re-fetch the offset from the skip list and regenerate the new pointer (because string's base pointer can change while mutating). Test Plan: COMPILE_WITH_ASAN=1 make write_batch_with_index_test && ./write_batch_with_index_test Reviewers: sdong Reviewed By: sdong Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D39897 10 June 2015, 19:57:38 UTC
8b7be18 Mikael Högqvist 10 June 2015, 19:52:42 UTC Updated OS X instructions, replace homebrew/dupes with homebrew/versions 10 June 2015, 19:52:42 UTC
d03f110 Igor Canadi 10 June 2015, 19:04:21 UTC Link all libraries when building shared libraries Summary: The motivation is here: http://stackoverflow.com/questions/30706348/rocksdb-make-error Apparently there is a bug (?) in g++ where we don't link rt library properly when building RocksDB's shared library. It looks like this works when we add an option -Wl, --no-as-needed. Test Plan: make shared_lib Reviewers: meyering, sdong, yhchiang Reviewed By: yhchiang Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D39681 10 June 2015, 19:04:21 UTC
75222d1 Igor Canadi 10 June 2015, 18:05:09 UTC Revert "Fix compile" This reverts commit 51440f83ec82b4b3fa54c1963f58d07a8b2c2810. Revert "Re-generate WriteEntry on WBWIIterator::Entry()" This reverts commit 4949ef08db89bdc63028679d1cc11312094e860e. 10 June 2015, 18:05:27 UTC
47f1e72 Igor Canadi 10 June 2015, 17:50:28 UTC Merge pull request #630 from rdallman/c-wb-logdata C: add WriteBatch.PutLogData support 10 June 2015, 17:50:28 UTC
51440f8 Igor Canadi 10 June 2015, 17:42:15 UTC Fix compile Summary: Ooops, sorry about this. Test Plan: compiles Reviewers: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D39885 10 June 2015, 17:42:15 UTC
4949ef0 Igor Canadi 10 June 2015, 17:35:19 UTC Re-generate WriteEntry on WBWIIterator::Entry() Summary: If we don't do this, any calls to Entry() after WBWI mutation will result in undefined behavior. We need to re-fetch the offset from the skip list and regenerate the new pointer (because string's base pointer can change while mutating). Test Plan: COMPILE_WITH_ASAN=1 make write_batch_with_index_test && ./write_batch_with_index_test Reviewers: sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D39813 10 June 2015, 17:35:19 UTC
735df66 Reed Allman 10 June 2015, 07:12:33 UTC C: add WriteBatch.PutLogData support 10 June 2015, 07:12:33 UTC
e409d3d sdong 08 June 2015, 18:43:55 UTC Make "make all" work for CYGWIN Summary: Some test and benchmark codes don't build for CYGWIN. Fix it. Test Plan: Build "make all" with TARGET_OS=Cygwin on cygwin and make sure it passes. Reviewers: rven, yhchiang, anthony, igor, kradhakrishnan Reviewed By: igor, kradhakrishnan Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D39711 09 June 2015, 23:36:07 UTC
62c3a95 Igor Canadi 09 June 2015, 20:10:31 UTC Add test for iteration+mutation of WBWI Summary: We should support use-cases that mutate WBWI while they're iterating it. This diff adds a unit test to check this behavior. Test Plan: this is a test Reviewers: sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D39501 09 June 2015, 20:10:31 UTC
d9b3338 Igor Canadi 09 June 2015, 19:55:58 UTC Add Yahoo's blog post about Sherpa to USERS.md Summary: As title Test Plan: ran unit tests Reviewers: sdong, yhchiang, rven Reviewed By: rven Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D39765 09 June 2015, 19:55:58 UTC
75d7075 sdong 09 June 2015, 06:14:13 UTC Print info message about files need compaction for debuging purpose Summary: When there are files marked for compaction after compactions, print extra messages to help debugging. Example: 2015/06/08-23:12:55.212855 7ff5013ff700 [default] [JOB 121] Generated table #75: 54 keys, 4807 bytes (need compaction) 2015/06/08-23:12:55.556194 7ff5013ff700 (Original Log Time 2015/06/08-23:12:55.556160) [default] compacted to: base level 1 max bytes base 10240 files[0 1 9 32 12 0 0 0] max score 0.96 (2 files need compaction), MB/sec: 0.0 rd, 0.1 wr, level 2, files in(1, 3) out(5) MB in(0.0, 0.0) out(0.0), read-write-amplify(11.3) write-amplify(5.7) OK, records in: 40, records dropped: 0 Test Plan: Run test and see LOG files. valgrind test DBTest.TablePropertiesNeedCompactTest Reviewers: rven, yhchiang, kradhakrishnan, IslamAbdelRahman, igor Reviewed By: igor Subscribers: yoshinorim, maykov, leveldb, dhruba Differential Revision: https://reviews.facebook.net/D39771 09 June 2015, 18:23:29 UTC
406a568 Venkatesh Radhakrishnan 09 June 2015, 17:39:49 UTC Fix hang when closing a DB after doing loads with WAL disabled. Summary: There is a hang during DB close in the following scenario: a) a load with WAL disabled was done, b) CancelAllBackgroundWork was called, c) DB Close was called This was because in that we will wait for a flush but we cannot do a background flush because we have called CancelAllBackgroundWork which marks the DB as shutting downn. Test Plan: Added DBTest FlushOnDestroy Reviewers: sdong Reviewed By: sdong Subscribers: yoshinorim, hermanlee4, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D39747 09 June 2015, 17:39:49 UTC
d8c8f08 sdong 08 June 2015, 23:57:44 UTC GetSnapshot() and ReleaseSnapshot() to move new and free out of DB mutex Summary: We currently issue malloc and free inside DB mutex in GetSnapshot() and ReleaseSnapshot(). Move them out. Test Plan: Go through all tests make valgrind_check Reviewers: yhchiang, rven, IslamAbdelRahman, anthony, igor Reviewed By: igor Subscribers: maykov, hermanlee4, MarkCallaghan, yoshinorim, leveldb, dhruba Differential Revision: https://reviews.facebook.net/D39753 09 June 2015, 04:57:02 UTC
643bbbf Islam AbdelRahman 08 June 2015, 23:34:26 UTC Use nullptr for default compaction_filter_factory Summary: Replacing the default value for compaction_filter_factory and compaction_filter_factory_v2 to be nullptr instead of DefaultCompactionFilterFactory / DefaultCompactionFilterFactoryV2 The reason for this is to be able to determine easily if we have compaction filter factory or not without depending on RTTI Test Plan: make check Reviewers: yoshinorim, ott, igor, sdong Reviewed By: sdong Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D39693 08 June 2015, 23:34:26 UTC
21f2492 sdong 05 June 2015, 21:29:22 UTC Fix CYGWin release build Summary: Change from one std::to_string() to ToString() for Cygwin build Test Plan: Build it under cygwin Reviewers: rven, anthony, IslamAbdelRahman, igor, kradhakrishnan Reviewed By: igor, kradhakrishnan Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D39657 08 June 2015, 18:35:22 UTC
f02ce0c Igor Canadi 08 June 2015, 18:28:40 UTC Fix ASAN errors in c_test Summary: key_sizes claims that 3rd key is of length 8, but it's really only 3. This diff makes it length 8. Test Plan: asan c_test works again. Reviewers: sdong, yhchiang Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D39699 08 June 2015, 18:28:40 UTC
133130a Igor Canadi 08 June 2015, 17:14:44 UTC Merge pull request #625 from rdallman/c-slice-parts-support C: add support for WriteBatch SliceParts params 08 June 2015, 17:14:44 UTC
de4d172 Igor Canadi 08 June 2015, 17:13:54 UTC Merge pull request #622 from rdallman/c-multiget C: add MultiGet support 08 June 2015, 17:13:54 UTC
6df589b sdong 04 June 2015, 19:03:40 UTC Add TablePropertiesCollector::NeedCompact() to suggest DB to further compact output files Summary: It is experimental. Allow users to return from a call back function TablePropertiesCollector::NeedCompact(), based on the data in the file. It can be used to allow users to suggest DB to clear up delete tombstones faster. Test Plan: Add a unit test. Reviewers: igor, yhchiang, kradhakrishnan, rven Reviewed By: rven Subscribers: yoshinorim, MarkCallaghan, maykov, leveldb, dhruba Differential Revision: https://reviews.facebook.net/D39585 06 June 2015, 03:18:21 UTC
back to top