Skip to main content

Browse the archive

swh:1:snp:5115096b921df712aeb2a08114fede57fb3331fb
sort by:
Revision Author Date Message Commit Date
9fe60d5 kailiu 28 January 2014, 21:06:09 UTC Add history log and revise script Summary: * Add a change log for rocksdb releases. * Remove the hacky parts of make_new_version.sh, which are either no longer useful or will be done in our dedicated 3rd-party release tool. Test Plan: N/A Reviewers: igor, haobo, sdong, dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D15543 29 January 2014, 04:24:41 UTC
9a126ba Lei Jin 29 January 2014, 00:03:55 UTC only corrupt private file checksum in backupable_db_test Summary: if it happens (randomly) to corrupt shared file in the test, then the checksum will be inconsistent between meta files from different backup. BackupEngine will then detect this issue and fail. But in reality, this does not happen since the checksum is checked on every backup. So here, only corrupt checksum of private file to let BackupEngine to construct properly (but fail during restore). Test Plan: run test with valgrind Reviewers: igor Reviewed By: igor CC: leveldb Differential Revision: https://reviews.facebook.net/D15531 29 January 2014, 00:03:55 UTC
5d2c628 Igor Canadi 29 January 2014, 00:02:51 UTC Only get the manifest file size if there is no error Summary: I came across this while working on column families. CorruptionTest::RecoverWriteError threw a SIGSEG because the descriptor_log_->file() was nullptr. I'm not sure why it doesn't happen in master, but better safe than sorry. @kailiu, can we get this in release, too? Test Plan: make check Reviewers: kailiu, dhruba, haobo Reviewed By: haobo CC: leveldb, kailiu Differential Revision: https://reviews.facebook.net/D15513 29 January 2014, 00:02:51 UTC
e5ec738 Igor Canadi 29 January 2014, 00:01:53 UTC Better interface to create BackupEngine Summary: I think it looks nicer. In RocksDB we have both styles, but I think that static method is the more common version. Test Plan: backupable_db_test Reviewers: ljin, benj, swk Reviewed By: ljin CC: leveldb Differential Revision: https://reviews.facebook.net/D15519 29 January 2014, 00:01:53 UTC
ec2fa4a Igor Canadi 28 January 2014, 19:26:07 UTC Export BackupEngine Summary: Lots of clients have problems with using StackableDB interface. It's nice to have BackupableDB as a layer on top of DB, but not necessary. This diff exports BackupEngine, which can be used to create backups without forcing clients to use StackableDB interface. Test Plan: backupable_db_test Reviewers: dhruba, ljin, swk Reviewed By: ljin CC: leveldb, benj Differential Revision: https://reviews.facebook.net/D15477 28 January 2014, 19:26:07 UTC
9dc2941 Lei Jin 28 January 2014, 17:43:36 UTC add checksum for backup files Summary: Keep checksum of each backuped file in meta file. When it restores these files, compute their checksum on the fly and compare against what is in the meta file. Fail the restore process if checksum mismatch. Test Plan: unit test Reviewers: haobo, igor, sdong, kailiu Reviewed By: igor CC: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D15381 28 January 2014, 17:43:36 UTC
90f29cc Mark Callaghan 22 January 2014, 22:37:20 UTC Update monitoring to include average time per compaction and stall Summary: The new columns are msComp and msStall that provide average time per compaction and stall for that level in milliseconds. Level Files Size(MB) Score Time(sec) Read(MB) Write(MB) Rn(MB) Rnp1(MB) Wnew(MB) RW-Amplify Read(MB/s) Write(MB/s) Rn Rnp1 Wnp1 NewW Count msComp msStall Ln-stall Stall-cnt ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 0 8 15 1.5 2 0 30 0 0 30 0.0 0.0 15.5 0 0 0 0 16 112 0.2 1.3 7568 1 8 16 1.6 1 26 26 15 11 16 3.5 17.6 18.1 8 6 13 7 3 362 0.0 0.0 0 2 1 2 0.0 0 0 2 0 0 2 0.0 0.0 18.4 0 0 0 0 1 50 0.0 0.0 0 Task ID: # Blame Rev: Test Plan: run db_bench Revert Plan: Database Impact: Memcache Impact: Other Notes: EImportant: - begin *PUBLIC* platform impact section - Bugzilla: # - end platform impact - Reviewers: haobo Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D15345 27 January 2014, 23:06:04 UTC
3d33da7 Schalk-Willem Kruger 27 January 2014, 22:49:10 UTC Fix UnmarkEOF for partial blocks Summary: Blocks in the transaction log are a fixed size, but the last block in the transaction log file is usually a partial block. When a new record is added after the reader hit the end of the file, a new physical record will be appended to the last block. ReadPhysicalRecord can only read full blocks and assumes that the file position indicator is aligned to the start of a block. If the reader is forced to read further by simply clearing the EOF flag, ReadPhysicalRecord will read a full block starting from somewhere in the middle of a real block, causing it to lose alignment and to have a partial physical record at the end of the read buffer. This will result in length mismatches and checksum failures. When the log file is tailed for replication this will cause the log iterator to become invalid, necessitating the creation of a new iterator which will have to read the log file from scratch. This diff fixes this issue by reading the remaining portion of the last block we read from. This is done when the reader is forced to read further (UnmarkEOF is called). Test Plan: - Added unit tests - Stress test (with replication). Check dbdir/LOG file for corruptions. - Test on test tier Reviewers: emayanke, haobo, dhruba Reviewed By: haobo CC: vamsi, sheki, dhruba, kailiu, igor Differential Revision: https://reviews.facebook.net/D15249 27 January 2014, 22:49:10 UTC
832158e Igor Canadi 27 January 2014, 19:02:21 UTC Fsync directory after we create a new file Summary: @dhruba, I'm not sure where we need to sync the directory. I implemented the function in Env() and added the dir sync just after we close the newly created file in the builder. Should I also add FsyncDir() to new files that get created by a compaction? Test Plan: Confirmed that FsyncDir is returning Status::OK() Reviewers: dhruba, haobo Reviewed By: dhruba CC: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D14751 27 January 2014, 19:02:21 UTC
6c2ca1d Igor Canadi 27 January 2014, 17:59:00 UTC Move NeedsCompaction() from VersionSet to Version Summary: There is no reason to have functions NeedCompaction(), MaxCompactionScore() and MaxCompactionScoreLevel() in VersionSet, since they don't access any data in VersionSet. Test Plan: make check Reviewers: kailiu, haobo, sdong Reviewed By: kailiu CC: leveldb Differential Revision: https://reviews.facebook.net/D15333 27 January 2014, 17:59:00 UTC
e55b3c0 Igor Canadi 27 January 2014, 01:40:43 UTC Fixing ref-counting memtables 27 January 2014, 02:03:38 UTC
983fafa Igor Canadi 25 January 2014, 21:50:30 UTC Fix memory leak 25 January 2014, 21:57:11 UTC
04afa32 Igor Canadi 25 January 2014, 02:30:00 UTC Fix reduce levels ReduceNumberOfLevels had segmentation fault in WriteSnapshot() since we didn't change the number of levels in VersionSet (we consider them immutable from now on). This fixes the problem. 25 January 2014, 02:32:38 UTC
8477255 Siying Dong 25 January 2014, 00:15:05 UTC Moving Some includes from options.h to forward declaration Summary: By removing some includes form options.h and reply on forward declaration, we can more easily reason the dependencies. Test Plan: make all check Reviewers: kailiu, haobo, igor, dhruba Reviewed By: kailiu CC: leveldb Differential Revision: https://reviews.facebook.net/D15411 25 January 2014, 01:16:22 UTC
f653fdc Igor Canadi 24 January 2014, 23:51:01 UTC Fixing iterator cleanup for Tailing iterator Immutable tailing iterator doesn't set CleanupState::mem, so we don't have to unref it. 24 January 2014, 23:51:06 UTC
b13bdfa Igor Canadi 24 January 2014, 22:57:52 UTC Add a call DisownData() to Cache, which should speed up shutdown Summary: On a shutdown, freeing memory takes a long time. If we're shutting down, we don't really care about memory leaks. I added a call to Cache that will avoid freeing all objects in cache. Test Plan: I created a script to test the speedup and demonstrate how to use the call: https://phabricator.fb.com/P3864368 Clean shutdown took 7.2 seconds, while fast and dirty one took 6.3 seconds. Unfortunately, the speedup is not that big, but should be bigger with bigger block_cache. I have set up the capacity to 80GB, but the script filled up only ~7GB. Reviewers: dhruba, haobo, MarkCallaghan, xjin Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D15069 24 January 2014, 22:57:52 UTC
677fee2 Igor Canadi 24 January 2014, 22:57:04 UTC Make VersionSet::ReduceNumberOfLevels() static Summary: A lot of our code implicitly assumes number_levels to be static. ReduceNumberOfLevels() breaks that assumption. For example, after calling ReduceNumberOfLevels(), DBImpl::NumberLevels() will be different from VersionSet::NumberLevels(). This is dangerous. Thankfully, it's not in public headers and is only used from LDB cmd tool. LDB tool is only using it statically, i.e. it never calls it with running DB instance. With this diff, we make it explicitly static. This way, we can assume number_levels to be immutable and not break assumption that lot of our code is relying upon. LDB tool can still use the method. Also, I removed the method from a separate file since it breaks filename completition. version_se<TAB> now completes to "version_set." instead of "version_set" (without the dot). I don't see a big reason that the function should be in a different file. Test Plan: reduce_levels_test Reviewers: dhruba, haobo, kailiu, sdong Reviewed By: kailiu CC: leveldb Differential Revision: https://reviews.facebook.net/D15303 24 January 2014, 22:57:04 UTC
c583157 Igor Canadi 24 January 2014, 22:52:08 UTC MemTableListVersion Summary: MemTableListVersion is to MemTableList what Version is to VersionSet. I took almost the same ideas to develop MemTableListVersion. The reason is to have copying std::list done in background, while flushing, rather than in foreground (MultiGet() and NewIterator()) under a mutex! Also, whenever we copied MemTableList, we copied also some MemTableList metadata (flush_requested_, commit_in_progress_, etc.), which was wasteful. This diff avoids std::list copy under a mutex in both MultiGet() and NewIterator(). I created a small database with some number of immutable memtables, and creating 100.000 iterators in a single-thread (!) decreased from {188739, 215703, 198028} to {154352, 164035, 159817}. A lot of the savings come from code under a mutex, so we should see much higher savings with multiple threads. Creating new iterator is very important to LogDevice team. I also think this diff will make SuperVersion obsolete for performance reasons. I will try it in the next diff. SuperVersion gave us huge savings on Get() code path, but I think that most of the savings came from copying MemTableList under a mutex. If we had MemTableListVersion, we would never need to copy the entire object (like we still do in NewIterator() and MultiGet()) Test Plan: `make check` works. I will also do `make valgrind_check` before commit Reviewers: dhruba, haobo, kailiu, sdong, emayanke, tnovak Reviewed By: kailiu CC: leveldb Differential Revision: https://reviews.facebook.net/D15255 24 January 2014, 22:52:08 UTC
f131d4c kailiu 24 January 2014, 19:56:01 UTC Add a make target for shared library Summary: Previous we made `make release` also compile shared library. However it takes a long time to complete. To make our development process more efficient. I added a new make target shared_lib. User can of course run `make <library_name>` for direct compilation. However the <library_name> changed under certain condition. Thus we need `make shared_lib` to get rid of the memorization from users' side. Test Plan: make shared_lib Reviewers: igor, sdong, haobo, dhruba Reviewed By: igor CC: leveldb Differential Revision: https://reviews.facebook.net/D15309 24 January 2014, 19:56:01 UTC
e832e72 Igor Canadi 24 January 2014, 19:50:38 UTC Revert "Moving to glibc-fb" This reverts commit d24961b65edeb155ebfc9f31c33e5aae380a4fec. For some reason, glibc2.17-fb breaks gflags. Reverting for now 24 January 2014, 19:50:38 UTC
66dc033 kailiu 24 January 2014, 18:57:15 UTC Temporarily disable caching index/filter blocks Summary: Mixing index/filter blocks with data blocks resulted in some known issues. To make sure in next release our users won't be affected, we added a new option in BlockBasedTableFactory::TableOption to conceal this functionality for now. This patch also introduced a BlockBasedTableReader::OpenOptions, which avoids the "infinite" growth of parameters in BlockBasedTableReader::Open(). Test Plan: make check Reviewers: haobo, sdong, igor, dhruba Reviewed By: igor CC: leveldb, tnovak Differential Revision: https://reviews.facebook.net/D15327 24 January 2014, 18:57:15 UTC
d24961b Igor Canadi 24 January 2014, 18:24:08 UTC Moving to glibc-fb Summary: It looks like we might have some trouble when building the new release with 4.8, since fbcode is using glibc2.17-fb by default and we are using glibc2.17. It was reported by Benjamin Renard in our internal group. This diff moves our fbcode build to use glibc2.17-fb by default. I got some linker errors when compiling, complaining that `google::SetUsageMessage()` was undefined. After deleting all offending lines, the compile was successful and everything works. Test Plan: Compiled Ran ./db_bench ./db_stress ./db_repl_stress Reviewers: kailiu Reviewed By: kailiu CC: leveldb Differential Revision: https://reviews.facebook.net/D15405 24 January 2014, 18:24:08 UTC
4605e20 Siying Dong 24 January 2014, 00:27:34 UTC If User setting of compaction multipliers overflow, use default value 1 instead Summary: Currently, compaction multipliers can overflow and cause unexpected behaviors. In this patch, we detect those overflows and use multiplier 1 for them. Test Plan: make all check Reviewers: dhruba, haobo, igor, kailiu Reviewed By: kailiu CC: leveldb Differential Revision: https://reviews.facebook.net/D15321 24 January 2014, 18:14:23 UTC
aba2acb Lei Jin 22 January 2014, 20:46:24 UTC CompactRange() to return status Summary: as title Test Plan: make all check What else tests shall I cover? Reviewers: igor, haobo CC: Differential Revision: https://reviews.facebook.net/D15339 24 January 2014, 00:41:46 UTC
81c9cc9 Tomislav Novak 17 January 2014, 05:56:26 UTC Tailing iterator Summary: This diff implements a special type of iterator that doesn't create a snapshot (can be used to read newly inserted data) and is optimized for doing sequential reads. TailingIterator uses current superversion number to determine whether to invalidate its internal iterators. If the version hasn't changed, it can often avoid doing expensive seeks over immutable structures (sst files and immutable memtables). Test Plan: * new unit tests * running LD with this patch Reviewers: igor, dhruba, haobo, sdong, kailiu Reviewed By: sdong CC: leveldb, lovro, march Differential Revision: https://reviews.facebook.net/D15285 24 January 2014, 00:26:08 UTC
4e91f27 Igor Canadi 24 January 2014, 00:11:55 UTC Fix performance regression in statistics Summary: For some reason, D15099 caused a big performance regression: https://fburl.com/16059000 After digging a bit, I figured out that the reason was that std::atomic_uint_fast64_t was allocated in an array. When I switched from an array to vector, the QPS returned to the previous level. I'm not sure why this is happening, but this diff seems to fix the performance regression. Test Plan: I ran the regression script, observed the performance going back to normal Reviewers: tnovak, kailiu, haobo Reviewed By: kailiu CC: leveldb Differential Revision: https://reviews.facebook.net/D15375 24 January 2014, 00:11:55 UTC
d045846 kailiu 23 January 2014, 23:04:12 UTC Add google-style checker to "arc lint" Summary: After we reached a consensus on code format, which follows exactly Google's coding style, a natural follow-up is to have a style checker that can handle stuffs beyond format. Google already has a powerful style checker "cpplint.py" and, luckily, phabricator already provides the built-in linter for it! Next time with "arc lint" most style inconsistency will be detected (but will not be fixed). Also I copied cpplint.py to linters directory, which is mostly because we may need the flexibility to make some modifications on it for our own need. Test Plan: ran arc lint table/block_based_table_builder.cc to see the amazing results. Reviewers: haobo, sdong, igor, dhruba Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D15369 23 January 2014, 23:04:12 UTC
fb01755 Igor Canadi 22 January 2014, 18:55:16 UTC Unfriending classes Summary: In this diff I made some effort to reduce usage of friending. To do that, I had to expose Compaction::inputs_ through a method inputs(). Not sure if this is a good idea, there is a trade-off. I think it's less confusing than having lots of friends. I also thought about other friendship relationships, but they are too much tangled at this point. Once you friend two classes, it's very hard to unfriend them :) Test Plan: make check Reviewers: haobo, kailiu, sdong, dhruba Reviewed By: kailiu CC: leveldb Differential Revision: https://reviews.facebook.net/D15267 22 January 2014, 18:55:16 UTC
6fe9b57 Igor Canadi 22 January 2014, 18:45:26 UTC Refactor Recover() code Summary: This diff does two things: * Rethinks how we call Recover() with read_only option. Before, we call it with pointer to memtable where we'd like to apply those changes to. This memtable is set in db_impl_readonly.cc and it's actually DBImpl::mem_. Why don't we just apply updates to mem_ right away? It seems more intuitive. * Changes when we apply updates to manifest. Before, the process is to recover all the logs, flush it to sst files and then do one giant commit that atomically adds all recovered sst files and sets the next log number. This works good enough, but causes some small troubles for my column family approach, since I can't have one VersionEdit apply to more than single column family[1]. The change here is to commit the files recovered from logs right away. Here is the state of the world before the change: 1. Recover log 5, add new sst files to edit 2. Recover log 7, add new sst files to edit 3. Recover log 8, add new sst files to edit 4. Commit all added sst files to manifest and mark log files 5, 7 and 8 as recoverd (via SetLogNumber(9) function) After the change, we'll do: 1. Recover log 5, commit the new sst files and set log 5 as recovered 2. Recover log 7, commit the new sst files and set log 7 as recovered 3. Recover log 8, commit the new sst files and set log 8 as recovered The added (small) benefit is that if we fail after (2), the new recovery will only have to recover log 8. In previous case, we'll have to restart the recovery from the beginning. The bigger benefit will be to enable easier integration of multiple column families in Recovery code path. [1] I'm happy to dicuss this decison, but I believe this is the cleanest way to go. It also makes backward compatibility much easier. We don't have a requirement of adding multiple column families atomically. Test Plan: make check Reviewers: dhruba, haobo, kailiu, sdong Reviewed By: kailiu CC: leveldb Differential Revision: https://reviews.facebook.net/D15237 22 January 2014, 18:45:26 UTC
4036d58 kailiu 22 January 2014, 01:51:36 UTC Fix a Statistics-related unit test faulure Summary: In my MacOS, the member variables are populated with random numbers after initialization. This diff fixes it by fill these arrays with 0. Test Plan: make && ./table_test Reviewers: igor CC: leveldb Differential Revision: https://reviews.facebook.net/D15315 22 January 2014, 02:02:55 UTC
4e8321b Mark Callaghan 18 January 2014, 00:38:54 UTC Boost access before mutex is unlocked Summary: This moves the use of versions_ to before the mutex is unlocked to avoid a possible race. Task ID: # Blame Rev: Test Plan: make check Revert Plan: Database Impact: Memcache Impact: Other Notes: EImportant: - begin *PUBLIC* platform impact section - Bugzilla: # - end platform impact - Reviewers: haobo, dhruba Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D15279 18 January 2014, 05:32:23 UTC
83681bf Igor Canadi 17 January 2014, 20:46:06 UTC Statistics code cleanup Summary: I'm separating code-cleanup part of https://reviews.facebook.net/D14517. This will make D14517 easier to understand and this diff easier to review. Test Plan: make check Reviewers: haobo, kailiu, sdong, dhruba, tnovak Reviewed By: tnovak CC: leveldb Differential Revision: https://reviews.facebook.net/D15099 17 January 2014, 20:46:06 UTC
0f4a75b Igor Canadi 17 January 2014, 20:02:03 UTC Fix SIGSEGV in compaction picker Summary: The SIGSEGV was introduced by https://reviews.facebook.net/D15171 I also fixed ExpandWhileOverlapping() which returned the failure by setting it's own stack variable to nullptr (!). This bug is present in 2.6 release, so I guess ExpandWhileOverlapping never fails :) Test Plan: `make check`. Also MarkCallaghan confirmed it fixed the SIGSEGV he reported. Reviewers: MarkCallaghan, kailiu, sdong, dhruba, haobo Reviewed By: kailiu CC: leveldb Differential Revision: https://reviews.facebook.net/D15261 17 January 2014, 20:02:03 UTC
439e36d Mark Callaghan 17 January 2014, 02:44:23 UTC Fix SlowdownAmount Summary: This had a few bugs. 1) bottom and top were reversed. top is for the max value but the callers were passing the max value to bottom. The result is that the max sleep is used when n >= bottom. 2) one of the callers passed values with type double and these values are frequently between 1.0 and 2.0 so rounding will do some bad things 3) sometimes the function returned 0 when there should be a stall With this change and one other diff (out for review soon) there are slightly fewer stalls on one workload. With the fix. Stalls(secs): 160.166 level0_slowdown, 0.000 level0_numfiles, 0.000 memtable_compaction, 58.495 leveln_slowdown Stalls(count): 910261 level0_slowdown, 0 level0_numfiles, 0 memtable_compaction, 54526 leveln_slowdown Without the fix. Stalls(secs): 172.227 level0_slowdown, 0.000 level0_numfiles, 0.000 memtable_compaction, 56.538 leveln_slowdown Stalls(count): 160831 level0_slowdown, 0 level0_numfiles, 0 memtable_compaction, 52845 leveln_slowdown Task ID: # Blame Rev: Test Plan: run db_bench for --benchmarks=overwrite with IO-bound database Revert Plan: Database Impact: Memcache Impact: Other Notes: EImportant: - begin *PUBLIC* platform impact section - Bugzilla: # - end platform impact - Reviewers: haobo Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D15243 17 January 2014, 18:15:30 UTC
e19bad9 Kai Liu 16 January 2014, 22:26:51 UTC Fix some "make format" issue Summary: * make sure when some pre-check fails, the script won't halt immediately. * change fburl to google's short url. * Fix a bug in this script: now it checks the uncommitted code only. Test Plan: Ran the script under differnet environments. Reviewers: igor Reviewed By: igor CC: leveldb Differential Revision: https://reviews.facebook.net/D15231 16 January 2014, 22:26:51 UTC
6d6fb70 Igor Canadi 16 January 2014, 22:06:53 UTC Remove compaction pointers Summary: The only thing we do with compaction pointers is set them to some values, we never actually read them. I don't know what we used them for, but it doesn't look like we use them anymore. Test Plan: make check Reviewers: dhruba, haobo, kailiu, sdong Reviewed By: kailiu CC: leveldb Differential Revision: https://reviews.facebook.net/D15225 16 January 2014, 22:06:53 UTC
c699c84 Igor Canadi 16 January 2014, 21:03:52 UTC CompactionPicker Summary: This is a big one. This diff moves all the code related to picking compactions from VersionSet to new class CompactionPicker. Column families' compactions will be completely separate processes, so we need to have multiple CompactionPickers. To make this easier to review, most of the code change is just copy/paste. There is also a small change not to use VersionSet::current_, but rather to take `Version* version` as a parameter. Most of the other code is exactly the same. In future diffs, I will also make some improvements to CompactionPickers. I think the most important part will be encapsulating it better. Currently Version, VersionSet, Compaction and CompactionPicker are all friend classes, which makes it harder to change the implementation. This diff depends on D15171, D15183, D15189 and D15201 Test Plan: `make check` Reviewers: kailiu, sdong, dhruba, haobo Reviewed By: kailiu CC: leveldb Differential Revision: https://reviews.facebook.net/D15207 16 January 2014, 21:03:52 UTC
eae1804 kailiu 16 January 2014, 02:17:58 UTC Remove the unnecessary use of shared_ptr Summary: shared_ptr is slower than unique_ptr (which literally comes with no performance cost compare with raw pointers). In memtable and memtable rep, we use shared_ptr when we'd actually should use unique_ptr. According to igor's previous work, we are likely to make quite some performance gain from this diff. Test Plan: make check Reviewers: dhruba, igor, sdong, haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D15213 16 January 2014, 02:22:01 UTC
787f11b Igor Canadi 16 January 2014, 00:23:36 UTC Move more functions from VersionSet to Version Summary: This moves functions: * VersionSet::Finalize() -> Version::UpdateCompactionStats() * VersionSet::UpdateFilesBySize() -> Version::UpdateFilesBySize() The diff depends on D15189, D15183 and D15171 Test Plan: make check Reviewers: kailiu, sdong, haobo, dhruba Reviewed By: sdong CC: leveldb Differential Revision: https://reviews.facebook.net/D15201 16 January 2014, 00:23:36 UTC
615d1ea Igor Canadi 16 January 2014, 00:22:34 UTC Moving Compaction class to separate header file Summary: I'm sure we'll all agree that version_set.cc needs simplifying. This diff moves Compaction class to a separate file. The diff depends on D15171 and D15183 Test Plan: make check Reviewers: dhruba, haobo, kailiu, sdong Reviewed By: kailiu CC: leveldb Differential Revision: https://reviews.facebook.net/D15189 16 January 2014, 00:22:34 UTC
2f4eda7 Igor Canadi 16 January 2014, 00:18:04 UTC Move functions from VersionSet to Version Summary: There were some functions in VersionSet that had no reason to be there instead of Version. Moving them to Version will make column families implementation easier. The functions moved are: * NumLevelBytes * LevelSummary * LevelFileSummary * MaxNextLevelOverlappingBytes * AddLiveFiles (previously AddLiveFilesCurrentVersion()) * NeedSlowdownForNumLevel0Files The diff continues on (and depends on) D15171 Test Plan: make check Reviewers: dhruba, haobo, kailiu, sdong, emayanke Reviewed By: sdong CC: leveldb Differential Revision: https://reviews.facebook.net/D15183 16 January 2014, 00:18:04 UTC
65a8a52 Igor Canadi 16 January 2014, 00:15:43 UTC Decrease reliance on VersionSet::NumberLevels() Summary: With column families VersionSet will not have a constant number of levels (each CF can have different options), so we'll need to eliminate call to VersionSet::NumberLevels() This diff decreases number of callsites, but we're not there yet. It associates number of levels with Version (each version is associated with single CF) instead of VersionSet. I have also slightly changed how VersionSet keeps track of manifest size. This diff also modifies constructor of Compaction such that it takes input_version and automatically Ref()s it. Before this was done outside of constructor. In next diffs I will continue to decrease number of callsites of VersionSet::NumberLevels() and also references to current_ Test Plan: make check Reviewers: haobo, dhruba, kailiu, sdong Reviewed By: sdong Differential Revision: https://reviews.facebook.net/D15171 16 January 2014, 00:15:43 UTC
c8f1622 kailiu 15 January 2014, 02:03:56 UTC Fix the return type of WriteBatch::Data(). Summary: Quick fix for https://reviews.facebook.net/D15123 Test Plan: Make check Reviewers: sdong, vkrest CC: leveldb Differential Revision: https://reviews.facebook.net/D15165 15 January 2014, 04:24:48 UTC
9b51af5 Siying Dong 12 December 2013, 18:54:03 UTC [RocksDB Performance Branch] DBImpl.NewInternalIterator() to reduce works inside mutex Summary: To reduce mutex contention caused by DBImpl.NewInternalIterator(), in this function, move all the iteration creation works out of mutex, only leaving object ref and get. Test Plan: make all check will run db_stress for a while too to make sure no problem. Reviewers: haobo, dhruba, kailiu Reviewed By: haobo CC: igor, leveldb Differential Revision: https://reviews.facebook.net/D14589 Conflicts: db/db_impl.cc 15 January 2014, 01:41:44 UTC
d9cd7a0 Igor Canadi 15 January 2014, 00:19:09 UTC Fix CompactRange to apply filter to every key Summary: When doing CompactRange(), we should first flush the memtable and then calculate max_level_with_files. Also, we want to compact all the levels that have files, including level `max_level_with_files`. This patch fixed the unit test. Test Plan: Added a failing unit test and a fix, so it's not failing anymore. Reviewers: dhruba, haobo, sdong Reviewed By: haobo CC: leveldb, xjin Differential Revision: https://reviews.facebook.net/D14421 15 January 2014, 00:19:09 UTC
1ed2404 Igor Canadi 14 January 2014, 23:54:11 UTC Wrong number of levels is Invalid argument now, not corruption 14 January 2014, 23:54:11 UTC
6291020 Igor Canadi 14 January 2014, 23:41:30 UTC Fix test 14 January 2014, 23:41:30 UTC
7f3e417 Igor Canadi 14 January 2014, 23:32:37 UTC Fix memtable construction in tests 14 January 2014, 23:36:12 UTC
055e6df Igor Canadi 14 January 2014, 23:27:09 UTC VersionEdit not to take NumLevels() Summary: I will submit a sequence of diffs that are preparing master branch for column families. There are a lot of implicit assumptions in the code that are making column family implementation hard. If I make the change only in column family branch, it will make merging back to master impossible. Most of the diffs will be simple code refactorings, so I hope we can have fast turnaround time. Feel free to grab me in person to discuss any of them. This diff removes number of level check from VersionEdit. It is used only when VersionEdit is read, not written, but has to be set when it is written. I believe it is a right thing to make VersionEdit dumb and check consistency on the caller side. This will also make it much easier to implement Column Families, since different column families can have different number of levels. Test Plan: make check Reviewers: dhruba, haobo, sdong, kailiu Reviewed By: kailiu CC: leveldb Differential Revision: https://reviews.facebook.net/D15159 14 January 2014, 23:27:09 UTC
7d9f21c Igor Canadi 14 January 2014, 22:49:31 UTC BuildBatchGroup -- memcpy outside of lock Summary: When building batch group, don't actually build a new batch since it requires heavy-weight mem copy and malloc. Only store references to the batches and build the batch group without lock held. Test Plan: `make check` I am also planning to run performance tests. The workload that will benefit from this change is readwhilewriting. I will post the results once I have them. Reviewers: dhruba, haobo, kailiu Reviewed By: haobo CC: leveldb, xjin Differential Revision: https://reviews.facebook.net/D15063 14 January 2014, 22:49:31 UTC
481c77e kailiu 14 January 2014, 21:54:33 UTC Move the compilation of the shared libraries to "make release" Compiling the shared libraries took a long time. Thus to speed up the development speed, it still makes sense to be separated from regular compilation. 14 January 2014, 21:54:33 UTC
78ee225 Naman Gupta 14 January 2014, 20:41:29 UTC Merge branch 'master' of github.com:facebook/rocksdb into sanitizedOptions 14 January 2014, 20:41:29 UTC
d702d80 Kai Liu 14 January 2014, 08:39:42 UTC A script that automatically reformat affected lines Summary: Added a script that reformat only the affected lines in a given diff. I planned to make that file as pre-commit hook but looks it's a little bit more difficult than I thought. Since I don't want to spend too much time on this task right now, I eventually added a "make command" to achieve this with a few additional key strokes. Also make the clang-format solely inherited from Google's style -- there are still debates on some of the style issues, but we can address them later once we reach a consensus. Test Plan: Did some ugly format change and ran "make format", all affected lines are formatted as expected. Reviewers: igor, sdong, haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D15147 14 January 2014, 20:21:24 UTC
1d9bac4 Naman Gupta 16 November 2013, 01:17:13 UTC Use sanitized options while opening db Summary: We use SanitizeOptions() to set appropriate values for some options, based on other options. So we should use the sanitized options by default. Luckily it hasn't caused a bug yet, but can result in a bug in the fugture. Test Plan: make check Reviewers: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D14103 14 January 2014, 19:46:24 UTC
fbbf0d1 Siying Dong 14 January 2014, 19:04:27 UTC Pre-calculate whether to slow down for too many level 0 files Summary: Currently in DBImpl::MakeRoomForWrite(), we do "versions_->NumLevelFiles(0) >= options_.level0_slowdown_writes_trigger" to check whether the writer thread needs to slow down. However, versions_->NumLevelFiles(0) is slightly more expensive than we expected. By caching the result of the comparison when installing a new version, we can avoid this function call every time. Test Plan: make all check Manually trigger this behavior by applying universal compaction style and make sure inserts are made slow after there are certain number of files. Reviewers: haobo, kailiu, igor Reviewed By: kailiu CC: nkg-, leveldb Differential Revision: https://reviews.facebook.net/D15141 14 January 2014, 19:23:02 UTC
51dd219 Siying Dong 14 January 2014, 18:42:36 UTC DB::Put() to estimate write batch data size needed and pre-allocate buffer Summary: In one of CPU profiles, we see some CPU costs of string::reserve() inside Batch.Put(). This patch should be able to reduce some of the costs by allocating sufficient buffer before hand. Since it is a trivial percentage of CPU costs, I didn't find a way to show the improvement in one of the benchmarks. I'll deploy it to same application and do the same CPU profiling to make sure those CPU costs are reduced. Test Plan: make all check Reviewers: haobo, kailiu, igor Reviewed By: haobo CC: leveldb, nkg- Differential Revision: https://reviews.facebook.net/D15135 14 January 2014, 18:53:16 UTC
ac2fe72 kailiu 14 January 2014, 06:09:41 UTC Compile dynamic library by default Summary: Per request, some users need to use dynamic rocksdb library instead of static one. However currently the dynamic libraries have to be manually compiled by default, which is inconvenient. I made dymamic libraries to be compiled by default. Test Plan: make clean; make; make clean; Reviewers: haobo, sdong, dhruba, igor Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D15117 14 January 2014, 08:28:10 UTC
c4548d5 Siying Dong 13 January 2014, 23:01:34 UTC WriteBatch to provide a way for user to query data size directly and only return constant reference of data in Data() Summary: WriteBatch::Data() now is easily to be misuse by users. Also, there is no cheap way for user of WriteBatch to know the data size accumulated. This patch fix the problem by: (1) return a constant reference to Data() so it's obvious to caller what it means. (2) add a function to return data size directly Test Plan: make all check Reviewers: haobo, igor, kailiu Reviewed By: kailiu CC: zshao, leveldb Differential Revision: https://reviews.facebook.net/D15123 14 January 2014, 00:52:14 UTC
dd6ecdf Igor Canadi 11 January 2014, 17:25:42 UTC Use ASSERT_EQ() instead of assert() in merge_test 11 January 2014, 17:25:47 UTC
a09ee10 Schalk-Willem Kruger 11 January 2014, 01:33:56 UTC Improve RocksDB "get" performance by computing merge result in memtable Summary: Added an option (max_successive_merges) that can be used to specify the maximum number of successive merge operations on a key in the memtable. This can be used to improve performance of the "get" operation. If many successive merge operations are performed on a key, the performance of "get" operations on the key deteriorates, as the value has to be computed for each "get" operation by applying all the successive merge operations. FB Task ID: #3428853 Test Plan: make all check db_bench --benchmarks=readrandommergerandom counter_stress_test Reviewers: haobo, vamsi, dhruba, sdong Reviewed By: haobo CC: zshao Differential Revision: https://reviews.facebook.net/D14991 11 January 2014, 01:33:56 UTC
62197d2 Igor Canadi 10 January 2014, 21:40:46 UTC Merge pull request #62 from matope/fix-BackupableDBTest-NoDoubleCopy-test-fail Fix share_table_files condition in BackupEngine constructor. 10 January 2014, 21:40:46 UTC
f8642da ono_matope 10 January 2014, 20:12:07 UTC Fix share_table_files condition in BackupEngine constructor. That makes BackupableDBTest.NoDoubleCopy test error. 10 January 2014, 20:12:07 UTC
9996e2d Kai Liu 10 January 2014, 18:28:50 UTC Merge pull request #61 from Yancey1989/master fix compile warning 10 January 2014, 18:28:50 UTC
afdd2d1 Yancey 10 January 2014, 09:56:35 UTC fix compile warning 10 January 2014, 09:56:35 UTC
cb37ddf Igor Canadi 09 January 2014, 20:24:28 UTC Feature requests for BackupableDB Summary: This diff introduces some features that were requested by two internal customers: * Ability for backups not to share table files, because we can't guarantee that equal filename means equal content accross replicas * Ability for two threads to call EnableFileDeletions() and DisableFileDeletions() * Ability to stop backup from another thread and not slow down the DB close * Copy the files to the temporary folder first and then atomically rename Test Plan: Added some tests to backupable_db_test Reviewers: dhruba, sanketh, muthu, sdong, haobo Reviewed By: haobo CC: leveldb, sanketh, muthu Differential Revision: https://reviews.facebook.net/D14769 09 January 2014, 20:24:28 UTC
d040667 Igor Canadi 09 January 2014, 01:44:58 UTC readwhilewriting benchmark Summary: Added readwhilewriting benchmark to our regression tests. Changed block cache shards from 16 to 64, as Mark found that cache mutex contention is a big bottleneck. Test Plan: Ran it. Reviewers: dhruba, haobo, MarkCallaghan, xjin Reviewed By: MarkCallaghan CC: leveldb Differential Revision: https://reviews.facebook.net/D15075 09 January 2014, 01:44:58 UTC
5575316 Siying Dong 07 January 2014, 22:33:15 UTC StopWatch not to get time if it is created for statistics and it is disabled Summary: Currently, even if statistics is not enabled, StopWatch only for the stats still gets the time of the day, which is wasteful. This patch adds a new option to StopWatch to disable this get in this case. Test Plan: make all check Reviewers: dhruba, haobo, igor CC: leveldb Differential Revision: https://reviews.facebook.net/D14703 09 January 2014, 00:05:36 UTC
12b6d2b kailiu 08 January 2014, 23:06:07 UTC Separate the aligned and unaligned memory allocation Summary: Use two vectors for different types of memory allocation. Test Plan: run all unit tests. Reviewers: haobo, sdong Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D15027 08 January 2014, 23:11:42 UTC
50994bf Mark Callaghan 19 December 2013, 18:02:53 UTC Don't always compress L0 files written by memtable flush Summary: Code was always compressing L0 files written by a memtable flush when compression was enabled. Now this is done when min_level_to_compress=0 for leveled compaction and when universal_compaction_size_percent=-1 for universal compaction. Task ID: #3416472 Blame Rev: Test Plan: ran db_bench with compression options Revert Plan: Database Impact: Memcache Impact: Other Notes: EImportant: - begin *PUBLIC* platform impact section - Bugzilla: # - end platform impact - Reviewers: dhruba, igor, sdong Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D14757 08 January 2014, 05:50:26 UTC
a45b7d8 Igor Canadi 08 January 2014, 00:33:03 UTC Merge pull request #59 from mlin/more-c-bindings C API: add rocksdb_env_set_high_priority_background_threads 08 January 2014, 00:33:03 UTC
9f690ec Tomislav Novak 21 December 2013, 23:10:39 UTC Fix a deadlock in CompactRange() Summary: The way DBImpl::TEST_CompactRange() throttles down the number of bg compactions can cause it to deadlock when CompactRange() is called concurrently from multiple threads. Imagine a following scenario with only two threads (max_background_compactions is 10 and bg_compaction_scheduled_ is initially 0): 1. Thread #1 increments bg_compaction_scheduled_ (to LargeNumber), sets bg_compaction_scheduled_ to 9 (newvalue), schedules the compaction (bg_compaction_scheduled_ is now 10) and waits for it to complete. 2. Thread #2 calls TEST_CompactRange(), increments bg_compaction_scheduled_ (now LargeNumber + 10) and waits on a cv for bg_compaction_scheduled_ to drop to LargeNumber. 3. BG thread completes the first manual compaction, decrements bg_compaction_scheduled_ and wakes up all threads waiting on bg_cv_. Thread #1 runs, increments bg_compaction_scheduled_ by LargeNumber again (now 2*LargeNumber + 9). Since that's more than LargeNumber + newvalue, thread #2 also goes to sleep (waiting on bg_cv_), without resetting bg_compaction_scheduled_. This diff attempts to address the problem by introducing a new counter bg_manual_only_ (when positive, MaybeScheduleFlushOrCompaction() will only schedule manual compactions). Test Plan: I could pretty much consistently reproduce the deadlock with a program that calls CompactRange(nullptr, nullptr) immediately after Write() from multiple threads. This no longer happens with this patch. Tests (make check) pass. Reviewers: dhruba, igor, sdong, haobo Reviewed By: igor CC: leveldb Differential Revision: https://reviews.facebook.net/D14799 07 January 2014, 18:37:34 UTC
c370f55 kailiu 06 January 2014, 19:53:19 UTC Revert change in 8f6e319. 06 January 2014, 19:53:19 UTC
be271c3 Kai Liu 06 January 2014, 19:50:24 UTC Merge pull request #56 from sepeth/refactor-detect-version Refactor build_tools/build_detect_version 06 January 2014, 19:50:24 UTC
7e70ff6 kailiu 06 January 2014, 19:11:19 UTC Fix issue #57 06 January 2014, 19:11:19 UTC
d800dc5 Doğan Çeçen 05 January 2014, 20:52:19 UTC Refactor build_tools/build_detect_version 06 January 2014, 06:44:43 UTC
8f6e319 Kai Liu 05 January 2014, 07:47:44 UTC Add a hack to build_detect_platform so it works in all types of fb-servers 05 January 2014, 07:47:44 UTC
463086b Kai Liu 02 January 2014, 21:26:38 UTC Add clang-format rules Summary: The rule file is forked from that in Facebook's repo. I'll add format file for now and team members can tune the rules later. In this patch, I made only two changes in order to be consistent with existing coding style `SpacesBeforeTrailingComments: 2` `ColumnLimit: 80` Test Plan: N/A Reviewers: igor CC: leveldb Differential Revision: https://reviews.facebook.net/D15015 02 January 2014, 22:32:15 UTC
4695059 Kai Liu 31 December 2013, 00:40:08 UTC Automate the preparation step for a new release Summary: Added a script that prepares the repo for facebook's new rocksdb release, which will automatically do some necessary work to make sure this repo is ready for 3rdparty release. Test Plan: Run this script and observed: * new version was created (both in local and remote repo) as a git tag. * build_version.cc was updated * build_detect_platform was changed so that it won't create any new change. Reviewers: haobo, dhruba, sdong, igor CC: leveldb Differential Revision: https://reviews.facebook.net/D15003 02 January 2014, 19:35:33 UTC
9281a82 kailiu 02 January 2014, 18:29:48 UTC Hotfix the bug in table cache's GetSliceForFileNumber Forgot to fix this problem in master branch. Already fixed it in performance branch. 02 January 2014, 18:30:42 UTC
b60c14f Igor Canadi 02 January 2014, 11:33:42 UTC Support multi-threaded DisableFileDeletions() and EnableFileDeletions() Summary: We don't want two threads to clash if they concurrently call DisableFileDeletions() and EnableFileDeletions(). I'm adding a counter that will enable file deletions only after all DisableFileDeletions() calls have been negated with EnableFileDeletions(). However, we also don't want to break the old behavior, so I added a parameter force to EnableFileDeletions(). If force is true, we will still enable file deletions after every call to EnableFileDeletions(), which is what is happening now. Test Plan: make check Reviewers: dhruba, haobo, sanketh Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D14781 02 January 2014, 11:33:42 UTC
345fb94 Igor Canadi 02 January 2014, 11:30:29 UTC moving autovector_test after db_test 02 January 2014, 11:30:29 UTC
52ea1be Igor Canadi 02 January 2014, 10:00:04 UTC Add -DROCKSDB_FALLOCATE_PRESENT to fbcode build 02 January 2014, 10:00:04 UTC
2b3aab3 Igor Canadi 02 January 2014, 08:27:18 UTC Merge pull request #48 from dyu/master fix build bug 02 January 2014, 08:27:18 UTC
4b1d049 Mike Lin 31 December 2013, 23:14:18 UTC C API: add rocksdb_env_set_high_priority_background_threads 31 December 2013, 23:14:18 UTC
fe030bd Kai Liu 31 December 2013, 00:16:24 UTC update the latest version in README.fb to 2.7 31 December 2013, 00:16:24 UTC
5a20744 Kai Liu 31 December 2013, 00:14:55 UTC Simplify build_tools/build_detect_version 31 December 2013, 00:14:55 UTC
1795397 Kai Liu 30 December 2013, 22:53:56 UTC Update README.fb Update the latest version number. 30 December 2013, 22:53:56 UTC
e842b99 dyu 30 December 2013, 13:34:45 UTC docs for shared library builds 30 December 2013, 13:34:45 UTC
a6b476a dyu 30 December 2013, 13:33:52 UTC tweak build bug fix 30 December 2013, 13:33:52 UTC
9d4dc0d dyu 27 December 2013, 07:19:31 UTC fix build bug from recent commit:https://github.com/facebook/rocksdb/commit/43c386b72ee834c88a1a22500ce1fc36a8208277 27 December 2013, 07:19:31 UTC
a094f3b Siying Dong 27 December 2013, 00:25:45 UTC TableCache.FindTable() to avoid the mem copy of file number Summary: I'm not sure what's the purpose of encoding file number to a new buffer for looking up the table cache. It seems to be unnecessary to me. With this patch, we point the lookup key to the address of the int64 of the file number. Test Plan: make all check Reviewers: dhruba, haobo, igor, kailiu Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D14811 27 December 2013, 00:57:07 UTC
18df47b Siying Dong 26 December 2013, 21:49:04 UTC Avoid malloc in NotFound key status if no message is given. Summary: In some places we have NotFound status created with empty message, but it doesn't avoid a malloc. With this patch, the malloc is avoided for that case. The motivation of it is that I found in db_bench readrandom test when all keys are not existing, about 4% of the total running time is spent on malloc of Status, plus a similar amount of CPU spent on free of them, which is not necessary. Test Plan: make all check Reviewers: dhruba, haobo, igor Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D14691 27 December 2013, 00:23:10 UTC
b40c052 Kai Liu 26 December 2013, 23:56:20 UTC Fix all the comparison issue in fb dev servers 27 December 2013, 00:13:49 UTC
113a08c kailiu 26 December 2013, 23:47:07 UTC Fix [-Werror=sign-compare] in autovector_test 26 December 2013, 23:47:07 UTC
079a21b kailiu 26 December 2013, 23:12:30 UTC Fix the unused variable warning message in mac os 26 December 2013, 23:12:30 UTC
c01676e kailiu 12 December 2013, 23:32:56 UTC Implement autovector Summary: A vector that leverages pre-allocated stack-based array to achieve better performance for array with small amount of items. Test Plan: Added tests for both correctness and performance Here is the performance benchmark between vector and autovector Please note that in the test "Creation and Insertion Test", the test case were designed with the motivation described below: * no element inserted: internal array of std::vector may not really get initialize. * one element inserted: internal array of std::vector must have initialized. * kSize elements inserted. This shows the most time we'll spend if we keep everything in stack. * 2 * kSize elements inserted. The internal vector of autovector must have been initialized. Note: kSize is the capacity of autovector ===================================================== Creation and Insertion Test ===================================================== created 100000 vectors: each was inserted with 0 elements total time elapsed: 128000 (ns) created 100000 autovectors: each was inserted with 0 elements total time elapsed: 3641000 (ns) created 100000 VectorWithReserveSizes: each was inserted with 0 elements total time elapsed: 9896000 (ns) ----------------------------------- created 100000 vectors: each was inserted with 1 elements total time elapsed: 11089000 (ns) created 100000 autovectors: each was inserted with 1 elements total time elapsed: 5008000 (ns) created 100000 VectorWithReserveSizes: each was inserted with 1 elements total time elapsed: 24271000 (ns) ----------------------------------- created 100000 vectors: each was inserted with 4 elements total time elapsed: 39369000 (ns) created 100000 autovectors: each was inserted with 4 elements total time elapsed: 10121000 (ns) created 100000 VectorWithReserveSizes: each was inserted with 4 elements total time elapsed: 28473000 (ns) ----------------------------------- created 100000 vectors: each was inserted with 8 elements total time elapsed: 75013000 (ns) created 100000 autovectors: each was inserted with 8 elements total time elapsed: 18237000 (ns) created 100000 VectorWithReserveSizes: each was inserted with 8 elements total time elapsed: 42464000 (ns) ----------------------------------- created 100000 vectors: each was inserted with 16 elements total time elapsed: 102319000 (ns) created 100000 autovectors: each was inserted with 16 elements total time elapsed: 76724000 (ns) created 100000 VectorWithReserveSizes: each was inserted with 16 elements total time elapsed: 68285000 (ns) ----------------------------------- ===================================================== Sequence Access Test ===================================================== performed 100000 sequence access against vector size: 4 total time elapsed: 198000 (ns) performed 100000 sequence access against autovector size: 4 total time elapsed: 306000 (ns) ----------------------------------- performed 100000 sequence access against vector size: 8 total time elapsed: 565000 (ns) performed 100000 sequence access against autovector size: 8 total time elapsed: 512000 (ns) ----------------------------------- performed 100000 sequence access against vector size: 16 total time elapsed: 1076000 (ns) performed 100000 sequence access against autovector size: 16 total time elapsed: 1070000 (ns) ----------------------------------- Reviewers: dhruba, haobo, sdong, chip Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D14655 26 December 2013, 23:03:47 UTC
5643ae1 Kai Liu 26 December 2013, 22:20:22 UTC Merge pull request #32 from jamesgolick/master Only try to use fallocate if it's actually present on the system. 26 December 2013, 22:20:22 UTC
71ddb11 Dhruba Borthakur 23 December 2013, 20:19:18 UTC Add a pointer to the engineering design discussion forum. Summary: Add a pointer to the engineering design discussion forum. Test Plan: Reviewers: CC: Task ID: # Blame Rev: 23 December 2013, 20:19:18 UTC
b26dc95 Igor Canadi 20 December 2013, 18:01:12 UTC Initialize sequence number in BatchResult - issue #39 20 December 2013, 18:01:12 UTC
1fdb3f7 Igor Canadi 20 December 2013, 17:57:58 UTC [RocksDB] Optimize locking for Get Summary: Instead of locking and saving a DB state, we can cache a DB state and update it only when it changes. This change reduces lock contention and speeds up read operations on the DB. Performance improvements are substantial, although there is some cost in no-read workloads. I ran the regression tests on my devserver and here are the numbers: overwrite 56345 -> 63001 fillseq 193730 -> 185296 readrandom 771301 -> 1219803 (58% improvement!) readrandom_smallblockcache 677609 -> 862850 readrandom_memtable_sst 710440 -> 1109223 readrandom_fillunique_random 221589 -> 247869 memtablefillrandom 105286 -> 92643 memtablereadrandom 763033 -> 1288862 Test Plan: make asan_check I am also running db_stress Reviewers: dhruba, haobo, sdong, kailiu Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D14679 20 December 2013, 17:57:58 UTC
back to top