| 1f7850c | Dhruba Borthakur | 17 September 2012, 17:51:45 UTC | Build with gcc-4.7.1-glibc-2.14.1. Summary: Test Plan: Reviewers: CC: Task ID: # Blame Rev: | 17 September 2012, 17:56:26 UTC |
| b526342 | heyongqiang | 17 September 2012, 01:49:01 UTC | use 20d3328ac30f633840ce819ad03019f415267a86 as builder Summary: Summary: as subject Test Plan: build Reviewers: dhruba Reviewed By: dhruba Differential Revision: https://reviews.facebook.net/D5475 | 17 September 2012, 17:53:52 UTC |
| ba55d77 | Dhruba Borthakur | 15 September 2012, 00:11:35 UTC | Ability to take a file-lvel snapshot from leveldb. Summary: A set of apis that allows an application to backup data from the leveldb database based on a set of files. Test Plan: unint test attached. more coming soon. Reviewers: heyongqiang Reviewed By: heyongqiang Differential Revision: https://reviews.facebook.net/D5439 | 17 September 2012, 16:14:50 UTC |
| b85cdca | heyongqiang | 15 September 2012, 03:57:15 UTC | add a global var leveldb::useMmapRead to enable mmap Summary: Summary: as subject. this can be used for benchmarking. If we want it for some cases, we can do more changes to make this part of the option. Test Plan: db_test Reviewers: dhruba CC: MarkCallaghan Differential Revision: https://reviews.facebook.net/D5451 | 17 September 2012, 05:07:35 UTC |
| dcbd6be | heyongqiang | 17 September 2012, 02:33:03 UTC | remove boost Summary: as subject Test Plan: build Reviewers: dhruba Differential Revision: https://reviews.facebook.net/D5469 | 17 September 2012, 02:33:43 UTC |
| 33323f2 | Mark Callaghan | 14 September 2012, 19:35:02 UTC | Remove use of mmap for random reads Summary: Reads via mmap on concurrent workloads are much slower than pread. For example on a 24-core server with storage that can do 100k IOPS or more I can get no more than 10k IOPS with mmap reads and 32+ threads. Test Plan: db_bench benchmarks Reviewers: dhruba, heyongqiang Reviewed By: heyongqiang Differential Revision: https://reviews.facebook.net/D5433 | 14 September 2012, 23:43:50 UTC |
| fa29f82 | Mark Callaghan | 14 September 2012, 19:18:36 UTC | scan a long for FLAGS_cache_size to fix a compiler warning Summary: FLAGS_cache_size is a long, no need to scan %lld into a size_t for it (which generates a compiler warning) Test Plan: run db_bench Reviewers: dhruba, heyongqiang Reviewed By: heyongqiang CC: heyongqiang Differential Revision: https://reviews.facebook.net/D5427 | 14 September 2012, 19:45:42 UTC |
| 8371139 | Mark Callaghan | 14 September 2012, 18:21:54 UTC | Add --compression_type=X option with valid values: snappy (default) none bzip2 zlib Summary: This adds an option to db_bench to specify the compression algorithm to use for LevelDB Test Plan: ran db_bench Reviewers: dhruba Reviewed By: dhruba Differential Revision: https://reviews.facebook.net/D5421 | 14 September 2012, 19:28:21 UTC |
| 93f4952 | Dhruba Borthakur | 13 September 2012, 18:54:53 UTC | Ability to switch off filesystem read-aheads Summary: Ability to switch off filesystem read-aheads. This change is backward-compatible: the default setting is to allow file system read-aheads. Test Plan: run benchmarks Reviewers: heyongqiang, adsharma Reviewed By: heyongqiang Differential Revision: https://reviews.facebook.net/D5391 | 13 September 2012, 19:09:56 UTC |
| 4028ae7 | Dhruba Borthakur | 13 September 2012, 17:10:51 UTC | Do not cache readahead-pages in the OS cache. Summary: When posix_fadvise(offset, offset) is usedm it frees up only those pages in that specified range. But the filesystem could have done some read-aheads and those get cached in the OS cache. Do not cache readahead-pages in the OS cache. Test Plan: run db_bench benchmark. Reviewers: vamsi, heyongqiang Reviewed By: heyongqiang Differential Revision: https://reviews.facebook.net/D5379 | 13 September 2012, 17:56:02 UTC |
| 7ecc5d4 | Dhruba Borthakur | 13 September 2012, 16:23:18 UTC | Enable db_bench to specify block size. Summary: Enable db_bench to specify block size. Test Plan: compile and run Reviewers: heyongqiang Reviewed By: heyongqiang Differential Revision: https://reviews.facebook.net/D5373 | 13 September 2012, 17:22:43 UTC |
| 407727b | Dhruba Borthakur | 12 September 2012, 16:54:22 UTC | Fix compiler warnings. Use uint64_t instead of uint. Summary: Fix compiler warnings. Use uint64_t instead of uint. Test Plan: build using -Wall Reviewers: heyongqiang Reviewed By: heyongqiang Differential Revision: https://reviews.facebook.net/D5355 | 12 September 2012, 21:42:36 UTC |
| 0f43aa4 | heyongqiang | 06 September 2012, 00:44:13 UTC | put log in a seperate dir Summary: added a new option db_log_dir, which points the log dir. Inside that dir, in order to make log names unique, the log file name is prefixed with the leveldb data dir absolute path. Test Plan: db_test Reviewers: dhruba Reviewed By: dhruba Differential Revision: https://reviews.facebook.net/D5205 | 07 September 2012, 00:52:08 UTC |
| afb5f22 | heyongqiang | 04 September 2012, 23:49:09 UTC | build scribe with thrift lib Summary: Summary: as subject Test Plan: test build Reviewers: dhruba Reviewed By: dhruba Differential Revision: https://reviews.facebook.net/D5145 | 07 September 2012, 00:41:53 UTC |
| 536ca69 | Dhruba Borthakur | 06 September 2012, 06:45:42 UTC | The ReadnRandomWriteRandom was always looping FLAGS_num of times. Summary: If none of reads or writes are specified by user, then pick the FLAGS_NUM as the number of iterations in the ReadRandomWriteRandom test. If either reads or writes are defined, then use their maximum. Test Plan: run benchmark Reviewers: heyongqiang Reviewed By: heyongqiang Differential Revision: https://reviews.facebook.net/D5217 | 06 September 2012, 16:13:24 UTC |
| 354a9ea | Dhruba Borthakur | 05 September 2012, 07:02:16 UTC | Compile leveldb with gcc 4.7.1 Test Plan: run unit tests Reviewers: heyongqiang Reviewed By: heyongqiang Differential Revision: https://reviews.facebook.net/D5163 | 05 September 2012, 07:11:35 UTC |
| 7112c93 | Dhruba Borthakur | 04 September 2012, 23:31:48 UTC | Do not use scribe for release builds. Summary: Do not use scribe for release builds. Test Plan: build fbcode Reviewers: heyongqiang Reviewed By: heyongqiang Differential Revision: https://reviews.facebook.net/D5139 | 04 September 2012, 23:33:49 UTC |
| 94208a7 | Dhruba Borthakur | 31 August 2012, 06:06:17 UTC | Benchmark with both reads and writes at the same time. Summary: This patch enables the db_bench benchmark to issue both random reads and random writes at the same time. This options can be trigged via ./db_bench --benchmarks=readrandomwriterandom The default percetage of reads is 90. One can change the percentage of reads by specifying the --readwritepercent. ./db_bench --benchmarks=readrandomwriterandom=50 This is a feature request from Jeffro asking for leveldb performance with a 90:10 read:write ratio. Test Plan: run on test machine. Reviewers: heyongqiang Reviewed By: heyongqiang Differential Revision: https://reviews.facebook.net/D5067 | 04 September 2012, 19:06:26 UTC |
| 8bab056 | Dhruba Borthakur | 29 August 2012, 22:29:30 UTC | Release 1.5.0.fb. Summary: Test Plan: Reviewers: CC: Task ID: # Blame Rev: | 29 August 2012, 22:29:30 UTC |
| f0b1654 | Dhruba Borthakur | 29 August 2012, 22:21:56 UTC | Add libhdfs.a to the build process. Fix compilcation error for hdfs build. Summary: Test Plan: Reviewers: CC: Task ID: # Blame Rev: | 29 August 2012, 22:21:56 UTC |
| fe93631 | Dhruba Borthakur | 29 August 2012, 19:29:43 UTC | Clean up compiler warnings generated by -Wall option. Summary: Clean up compiler warnings generated by -Wall option. make clean all OPT=-Wall This is a pre-requisite before making a new release. Test Plan: compile and run unit tests Reviewers: heyongqiang Reviewed By: heyongqiang Differential Revision: https://reviews.facebook.net/D5019 | 29 August 2012, 21:24:51 UTC |
| e5fe80e | Dhruba Borthakur | 29 August 2012, 16:47:53 UTC | The sharding of the block cache is limited to 2*20 pieces. Summary: The numbers of shards that the block cache is divided into is configurable. However, if the user specifies that he/she wants the block cache to be divided into more than 2**20 pieces, then the system will rey to allocate a huge array of that size) that could fail. It is better to limit the sharding of the block cache to an upper bound. The default sharding is 16 shards (i.e. 2**4) and the maximum is now 2 million shards (i.e. 2**20). Also, fixed a bug with the LRUCache where the numShardBits should be a private member of the LRUCache object rather than a static variable. Test Plan: run db_bench with --cache_numshardbits=64. Task ID: # Blame Rev: Reviewers: heyongqiang Reviewed By: heyongqiang Differential Revision: https://reviews.facebook.net/D5013 | 29 August 2012, 19:17:59 UTC |
| a4f9b8b | heyongqiang | 27 August 2012, 06:45:35 UTC | merge 1.5 Summary: as subject Test Plan: db_test table_test Reviewers: dhruba | 28 August 2012, 18:43:33 UTC |
| 6fee5a7 | heyongqiang | 22 August 2012, 23:57:51 UTC | Do not spin in a tight loop attempting compactions if there is a compaction error Summary: as subject. ported the change from google code leveldb 1.5 Test Plan: run db_test Reviewers: dhruba Differential Revision: https://reviews.facebook.net/D4839 | 28 August 2012, 18:43:33 UTC |
| 935fdd0 | heyongqiang | 28 August 2012, 18:34:57 UTC | fix filename_test Summary: as subject Test Plan: run filename_test Reviewers: dhruba Differential Revision: https://reviews.facebook.net/D4965 | 28 August 2012, 18:42:42 UTC |
| 690bf88 | heyongqiang | 28 August 2012, 18:29:30 UTC | in db_stats_logger.cc, hold mutex_ while accessing versions_ Summary: as subject Test Plan:db_test Reviewers: dhruba | 28 August 2012, 18:29:30 UTC |
| d3759ca | heyongqiang | 27 August 2012, 07:50:26 UTC | fix db_test error with scribe logger turned on Summary: as subject Test Plan: db_test Reviewers: dhruba Reviewed By: dhruba Differential Revision: https://reviews.facebook.net/D4929 | 28 August 2012, 18:22:58 UTC |
| fc20273 | Dhruba Borthakur | 27 August 2012, 19:10:26 UTC | Introduce a new method Env->Fsync() that issues fsync (instead of fdatasync). Summary: Introduce a new method Env->Fsync() that issues fsync (instead of fdatasync). This is needed for data durability when running on ext3 filesystems. Added options to the benchmark db_bench to generate performance numbers with either fsync or fdatasync enabled. Cleaned up Makefile to build leveldb_shell only when building the thrift leveldb server. Test Plan: build and run benchmark Reviewers: heyongqiang Reviewed By: heyongqiang Differential Revision: https://reviews.facebook.net/D4911 | 28 August 2012, 04:24:17 UTC |
| e675351 | heyongqiang | 24 August 2012, 23:28:06 UTC | number to read is not resepected Summary: as subject Test Plan: sst_dump --command=scan --file= Reviewers: dhruba Differential Revision: https://reviews.facebook.net/D4887 | 25 August 2012, 01:29:40 UTC |
| 7cd6440 | heyongqiang | 24 August 2012, 22:22:47 UTC | Merge branch 'master' of https://github.com/facebook/leveldb | 24 August 2012, 22:22:47 UTC |
| 1de83cc | heyongqiang | 24 August 2012, 18:28:59 UTC | add more logs Summary: as subject add a tool to read sst file as subject. ./sst_reader --command=check --file= ./sst_reader --command=scan --file= Test Plan: db_test run this command Reviewers: dhruba Reviewed By: dhruba Differential Revision: https://reviews.facebook.net/D4881 | 24 August 2012, 22:20:49 UTC |
| 1c99b0a | heyongqiang | 23 August 2012, 02:15:06 UTC | add more logs Summary: as subject Test Plan:db_test Reviewers: dhruba | 24 August 2012, 22:18:43 UTC |
| 5d96f29 | Dhruba Borthakur | 21 August 2012, 07:33:21 UTC | Record the version of the source repository that was used to build the leveldb library. Summary: Record the version of the source that we are compiling. We keep a record of the git revision in util/version.cc. This source file is then built as a regular source file as part of the compilation process. One can run "strings executable_filename | grep _build_" to find the version of the source that we used to build the executable file. Test Plan: none Differential Revision: https://reviews.facebook.net/D4785 | 24 August 2012, 22:18:43 UTC |
| 6421927 | heyongqiang | 24 August 2012, 18:28:59 UTC | sst_dump tool Summary: as subject add a tool to read sst file as subject. ./sst_reader --command=check --file= ./sst_reader --command=scan --file= Test Plan: db_test run this command Reviewers: dhruba Differential Revision: https://reviews.facebook.net/D4881 | 24 August 2012, 22:18:43 UTC |
| d41316b | Dhruba Borthakur | 18 August 2012, 07:26:50 UTC | Prevent concurrent multiple opens of leveldb database. Summary: The fcntl call cannot detect lock conflicts when invoked multiple times from the same thread. Use a static lockedFile Set to record the paths that are locked. A lockfile request checks to see if htis filename already exists in lockedFiles, if so, then it triggers an error. Otherwise, it inserts the filename in the lockedFiles Set. A unlock file request verifies that the filename is in the lockedFiles set and removes it from lockedFiles set. Test Plan: unit test attached Reviewers: heyongqiang Reviewed By: heyongqiang Differential Revision: https://reviews.facebook.net/D4755 | 24 August 2012, 22:18:01 UTC |
| f3ee545 | Dhruba Borthakur | 17 August 2012, 17:48:40 UTC | Utility to dump manifest contents. Summary: ./manifest_dump --file=/tmp/dbbench/MANIFEST-000002 Output looks like manifest_file_number 30 next_file_number 31 last_sequence 388082 log_number 28 prev_log_number 0 --- level 0 --- --- level 1 --- --- level 2 --- 5:3244155['0000000000000000' @ 1 : 1 .. '0000000000028220' @ 28221 : 1] 7:3244177['0000000000028221' @ 28222 : 1 .. '0000000000056441' @ 56442 : 1] 9:3244156['0000000000056442' @ 56443 : 1 .. '0000000000084662' @ 84663 : 1] 11:3244178['0000000000084663' @ 84664 : 1 .. '0000000000112883' @ 112884 : 1] 13:3244158['0000000000112884' @ 112885 : 1 .. '0000000000141104' @ 141105 : 1] 15:3244176['0000000000141105' @ 141106 : 1 .. '0000000000169325' @ 169326 : 1] 17:3244156['0000000000169326' @ 169327 : 1 .. '0000000000197546' @ 197547 : 1] 19:3244178['0000000000197547' @ 197548 : 1 .. '0000000000225767' @ 225768 : 1] 21:3244155['0000000000225768' @ 225769 : 1 .. '0000000000253988' @ 253989 : 1] 23:3244179['0000000000253989' @ 253990 : 1 .. '0000000000282209' @ 282210 : 1] 25:3244157['0000000000282210' @ 282211 : 1 .. '0000000000310430' @ 310431 : 1] 27:3244176['0000000000310431' @ 310432 : 1 .. '0000000000338651' @ 338652 : 1] 29:3244156['0000000000338652' @ 338653 : 1 .. '0000000000366872' @ 366873 : 1] --- level 3 --- --- level 4 --- --- level 5 --- --- level 6 --- Test Plan: run on test directory created by dbbench Reviewers: heyongqiang Reviewed By: heyongqiang CC: hustliubo Differential Revision: https://reviews.facebook.net/D4743 | 24 August 2012, 22:17:09 UTC |
| e3bf8f0 | heyongqiang | 24 August 2012, 20:09:48 UTC | Merge branch 'master' of https://github.com/facebook/leveldb | 24 August 2012, 20:09:48 UTC |
| 89f0c25 | bol | 23 August 2012, 16:50:43 UTC | Merge remote-tracking branch 'origin/master' into task1181723 Conflicts: Makefile | 24 August 2012, 10:34:55 UTC |
| 2b443ba | Dhruba Borthakur | 14 August 2012, 21:52:48 UTC | Titile: a command line shell to read/write data from a leveldb thrift server Summary: implemented a commond line shell to talk with leveldb thrift server, which is based on state pattern and can be easily extended. Test Plan: build and run Reviewers: dhruba, zshao, heyongqiang Differential Revision: https://reviews.facebook.net/D4713 | 23 August 2012, 16:41:05 UTC |
| e5a7c8e | Dhruba Borthakur | 22 August 2012, 18:43:53 UTC | Log the open-options to the LOG. Summary: Log the open-options to the LOG. Use options_ instead of options because SanitizeOptions could modify the max_file_open limit. Test Plan: num db_bench Reviewers: heyongqiang Reviewed By: heyongqiang Differential Revision: https://reviews.facebook.net/D4833 | 22 August 2012, 19:22:12 UTC |
| af6fa30 | heyongqiang | 22 August 2012, 00:33:46 UTC | regression for trigger compaction logic Summary: as subject Test Plan: manually run db_bench confirmed Reviewers: dhruba Reviewed By: dhruba Differential Revision: https://reviews.facebook.net/D4809 | 22 August 2012, 18:41:33 UTC |
| 21082fa | heyongqiang | 22 August 2012, 00:33:46 UTC | regression for trigger compaction logic Summary: as subject Test Plan: manually run db_bench confirmed Reviewers: dhruba Differential Revision: https://reviews.facebook.net/D4809 | 22 August 2012, 01:11:21 UTC |
| a098207 | Dhruba Borthakur | 22 August 2012, 00:02:21 UTC | Fixed unit test c_test by initializing logger=NULL. Summary: Fixed unit test c_test by initializing logger=NULL. Removed "atomic" from last_log_ts so that unit tests do not require C11 compiler. Anyway, last_log_ts is mostly used for logging, so it is ok if it is loosely accurate. Test Plan: run c_test Reviewers: heyongqiang Reviewed By: heyongqiang Differential Revision: https://reviews.facebook.net/D4803 | 22 August 2012, 00:10:29 UTC |
| 407f020 | Dhruba Borthakur | 21 August 2012, 22:43:52 UTC | Unit test corruption_test do not compile. Summary: Unit test corruption_test do not compile. Test Plan: run unit tests Reviewers: heyongqiang Reviewed By: heyongqiang Differential Revision: https://reviews.facebook.net/D4797 | 21 August 2012, 22:46:16 UTC |
| f4e7feb | Dhruba Borthakur | 21 August 2012, 07:33:21 UTC | Record the version of the source repository that was used to build the leveldb library. Summary: Record the version of the source that we are compiling. We keep a record of the git revision in util/version.cc. This source file is then built as a regular source file as part of the compilation process. One can run "strings executable_filename | grep _build_" to find the version of the source that we used to build the executable file. Test Plan: none Differential Revision: https://reviews.facebook.net/D4785 | 21 August 2012, 21:47:15 UTC |
| 6ba1f17 | heyongqiang | 14 August 2012, 22:20:36 UTC | adding a scribe logger in leveldb to log leveldb deploy stats Summary: as subject. A new log is written to scribe via thrift client when a new db is opened and when there is a compaction. a new option var scribe_log_db_stats is added. Test Plan: manually checked using command "ptail -time 0 leveldb_deploy_stats" Reviewers: dhruba Differential Revision: https://reviews.facebook.net/D4659 | 21 August 2012, 18:43:22 UTC |
| e56b2c5 | Dhruba Borthakur | 18 August 2012, 07:26:50 UTC | Prevent concurrent multiple opens of leveldb database. Summary: The fcntl call cannot detect lock conflicts when invoked multiple times from the same thread. Use a static lockedFile Set to record the paths that are locked. A lockfile request checks to see if htis filename already exists in lockedFiles, if so, then it triggers an error. Otherwise, it inserts the filename in the lockedFiles Set. A unlock file request verifies that the filename is in the lockedFiles set and removes it from lockedFiles set. Test Plan: unit test attached Reviewers: heyongqiang Reviewed By: heyongqiang Differential Revision: https://reviews.facebook.net/D4755 | 21 August 2012, 06:55:04 UTC |
| deb1a1f | heyongqiang | 19 August 2012, 23:05:02 UTC | add disable wal to db_bench Summary: as subject. ./db_bench --benchmarks=fillrandom --num=1000000 --disable_data_sync=1 --write_buffer_size=50000000 --target_file_size_base=100000000 --disable_wal=1 LevelDB: version 1.4 Date: Sun Aug 19 16:01:59 2012 CPU: 8 * Intel(R) Xeon(R) CPU L5630 @ 2.13GHz CPUCache: 12288 KB Keys: 16 bytes each Values: 100 bytes each (50 bytes after compression) Entries: 1000000 RawSize: 110.6 MB (estimated) FileSize: 62.9 MB (estimated) ------------------------------------------------ fillrandom : 4.591 micros/op 217797 ops/sec; 24.1 MB/s ./db_bench --benchmarks=fillrandom --num=1000000 --disable_data_sync=1 --write_buffer_size=50000000 --target_file_size_base=100000000 LevelDB: version 1.4 Date: Sun Aug 19 16:02:54 2012 CPU: 8 * Intel(R) Xeon(R) CPU L5630 @ 2.13GHz CPUCache: 12288 KB Keys: 16 bytes each Values: 100 bytes each (50 bytes after compression) Entries: 1000000 RawSize: 110.6 MB (estimated) FileSize: 62.9 MB (estimated) ------------------------------------------------ fillrandom : 3.696 micros/op 270530 ops/sec; 29.9 MB/s Test Plan: db_bench Reviewers: dhruba Differential Revision: https://reviews.facebook.net/D4767 | 20 August 2012, 05:37:51 UTC |
| 2aa514e | Dhruba Borthakur | 17 August 2012, 17:48:40 UTC | Utility to dump manifest contents. Summary: ./manifest_dump --file=/tmp/dbbench/MANIFEST-000002 Output looks like manifest_file_number 30 next_file_number 31 last_sequence 388082 log_number 28 prev_log_number 0 --- level 0 --- --- level 1 --- --- level 2 --- 5:3244155['0000000000000000' @ 1 : 1 .. '0000000000028220' @ 28221 : 1] 7:3244177['0000000000028221' @ 28222 : 1 .. '0000000000056441' @ 56442 : 1] 9:3244156['0000000000056442' @ 56443 : 1 .. '0000000000084662' @ 84663 : 1] 11:3244178['0000000000084663' @ 84664 : 1 .. '0000000000112883' @ 112884 : 1] 13:3244158['0000000000112884' @ 112885 : 1 .. '0000000000141104' @ 141105 : 1] 15:3244176['0000000000141105' @ 141106 : 1 .. '0000000000169325' @ 169326 : 1] 17:3244156['0000000000169326' @ 169327 : 1 .. '0000000000197546' @ 197547 : 1] 19:3244178['0000000000197547' @ 197548 : 1 .. '0000000000225767' @ 225768 : 1] 21:3244155['0000000000225768' @ 225769 : 1 .. '0000000000253988' @ 253989 : 1] 23:3244179['0000000000253989' @ 253990 : 1 .. '0000000000282209' @ 282210 : 1] 25:3244157['0000000000282210' @ 282211 : 1 .. '0000000000310430' @ 310431 : 1] 27:3244176['0000000000310431' @ 310432 : 1 .. '0000000000338651' @ 338652 : 1] 29:3244156['0000000000338652' @ 338653 : 1 .. '0000000000366872' @ 366873 : 1] --- level 3 --- --- level 4 --- --- level 5 --- --- level 6 --- Test Plan: run on test directory created by dbbench Reviewers: heyongqiang Reviewed By: heyongqiang CC: hustliubo Differential Revision: https://reviews.facebook.net/D4743 | 18 August 2012, 05:36:59 UTC |
| 680e571 | heyongqiang | 18 August 2012, 01:10:09 UTC | add compaction log Summary: Summary: add compaction summary to log log looks like: 2012/08/17-18:18:32.557334 7fdcaa2bb700 Compaction summary: Base level 0, input file:[11 9 7 ],[] Test Plan: tested via db_test Reviewers: dhruba Differential Revision: https://reviews.facebook.net/D4749 | 18 August 2012, 02:29:39 UTC |
| 29c99fc | heyongqiang | 17 August 2012, 23:23:15 UTC | Merge branch 'master' of https://github.com/facebook/leveldb | 17 August 2012, 23:23:15 UTC |
| 20ee76b | heyongqiang | 17 August 2012, 23:06:05 UTC | use ts as suffix for LOG.old files Summary: as subject and only maintain 10 log files. Test Plan: new test in db_test Reviewers: dhruba Differential Revision: https://reviews.facebook.net/D4731 | 17 August 2012, 23:22:04 UTC |
| b56ff5e | Dhruba Borthakur | 16 August 2012, 18:03:07 UTC | Expose new configration variables via the thrift api Summary: Expose new configration variables via the thrift api. when compiing for fbcode, always build thrift server Test Plan: none. Reviewers: heyongqiang Reviewed By: heyongqiang Differential Revision: https://reviews.facebook.net/D4689 | 17 August 2012, 17:29:12 UTC |
| 7c0b5ec | Dhruba Borthakur | 16 August 2012, 05:52:39 UTC | leveldb thrift server uses an incorrect root directory Summary: The offsets used to speficy hostname and dbdir from the command line were not handled correctly. Test Plan: none. Reviewers: heyongqiang Reviewed By: heyongqiang Differential Revision: https://reviews.facebook.net/D4683 | 16 August 2012, 18:10:08 UTC |
| f16e393 | heyongqiang | 15 August 2012, 22:44:23 UTC | add more options to db_ben Summary: as subject Test Plan: run db_bench with new options Reviewers: dhruba Differential Revision: https://reviews.facebook.net/D4677 | 16 August 2012, 00:42:33 UTC |
| fcb2ea4 | heyongqiang | 15 August 2012, 23:38:08 UTC | disable data sync options needs to be checked when doing level-0 dump Summary: Summary: as subject Test Plan: use db_bench Reviewers: dhruba Differential Revision: https://reviews.facebook.net/D4671 | 15 August 2012, 23:39:02 UTC |
| fe6119b | Dhruba Borthakur | 14 August 2012, 21:52:48 UTC | "make all check" fails Summary: If I run "make all check" it fails to build the leveldb-thrift-server tests. This is because the vanilla build is not supposed to build thrift and/or hdfs extensions. Remove the server test from auto-running. The thrift/hdfs build instructions are more elaborate and I would like to avoid it for embedded database builds. Test Plan: build and run Reviewers: heyongqiang Reviewed By: heyongqiang Differential Revision: https://reviews.facebook.net/D4641 | 14 August 2012, 22:55:38 UTC |
| c3096af | Dhruba Borthakur | 03 August 2012, 22:20:58 UTC | Introduce a new option disableDataSync for opening the database. If this is set to true, then the data written to newly created data files are not sycned to disk, instead depend on the OS to flush dirty data to stable storage. This option is good for bulk Test Plan: manual tests Task ID: # Blame Rev: Differential Revision: https://reviews.facebook.net/D4515 | 03 August 2012, 22:23:53 UTC |
| c44be54 | Dhruba Borthakur | 03 August 2012, 07:58:26 UTC | Make leveldb server not exit immediately. Summary: Test Plan: Reviewers: CC: Task ID: # Blame Rev: | 03 August 2012, 08:00:35 UTC |
| 2d04ab5 | Dhruba Borthakur | 03 August 2012, 07:33:47 UTC | Latest release is 1.4.1.fb Summary: Test Plan: Reviewers: CC: Task ID: # Blame Rev: | 03 August 2012, 07:33:47 UTC |
| 88c515b | Dhruba Borthakur | 03 August 2012, 07:27:48 UTC | Implement taoAssocRangeGet(). Summary: Test Plan: Reviewers: CC: Task ID: # Blame Rev: | 03 August 2012, 07:27:48 UTC |
| 0c98fdc | Dhruba Borthakur | 02 August 2012, 21:09:13 UTC | Encode id1Type, id2Type and data version the assoc payload. Summary: Test Plan: Reviewers: CC: Task ID: # Blame Rev: | 02 August 2012, 21:09:13 UTC |
| a3a8f83 | Dhruba Borthakur | 02 August 2012, 00:28:35 UTC | Fix "gcc -Wall" warnings for assoc-thrift code. Summary: Test Plan: Reviewers: CC: Task ID: # Blame Rev: | 02 August 2012, 00:28:35 UTC |
| d82cfe1 | Dhruba Borthakur | 01 August 2012, 23:55:39 UTC | Checkin pre-compiled snappy libraries for leveldb. Summary: Test Plan: Reviewers: CC: Task ID: # Blame Rev: | 01 August 2012, 23:55:39 UTC |
| b40ad06 | Dhruba Borthakur | 01 August 2012, 23:17:43 UTC | Implement the FB-Assoc API via thrift. Summary: Test Plan: Reviewers: CC: Task ID: # Blame Rev: | 01 August 2012, 23:18:24 UTC |
| 4abf94e | Dhruba Borthakur | 17 July 2012, 07:44:35 UTC | Check in compiled binaries for snappy. Summary: Test Plan: Reviewers: CC: Task ID: # Blame Rev: | 17 July 2012, 07:44:35 UTC |
| 71fa013 | Dhruba Borthakur | 17 July 2012, 07:28:12 UTC | Modify symbolic links for snappy builds. Summary: Test Plan: Reviewers: CC: Task ID: # Blame Rev: | 17 July 2012, 07:32:38 UTC |
| 4274e6c | Dhruba Borthakur | 17 July 2012, 07:29:06 UTC | Added README for snappy builds Summary: Test Plan: Reviewers: CC: Task ID: # Blame Rev: | 17 July 2012, 07:30:12 UTC |
| 21f2fe4 | Dhruba Borthakur | 17 July 2012, 07:18:23 UTC | Build snappy for leveldb. Summary: Test Plan: Reviewers: CC: Task ID: # Blame Rev: | 17 July 2012, 07:18:23 UTC |
| d550320 | Dhruba Borthakur | 11 July 2012, 21:08:46 UTC | Share a single cache for all the DBs served by this server. Summary: Test Plan: Reviewers: CC: Task ID: # Blame Rev: | 11 July 2012, 21:08:46 UTC |
| bfaa1bd | Dhruba Borthakur | 10 July 2012, 17:19:49 UTC | Cleanup thrift header files. Move thrift api from thrift/leveldb.thrift to thrift/if/leveldb.thrift Summary: Test Plan: Reviewers: CC: Task ID: # Blame Rev: | 10 July 2012, 17:19:49 UTC |
| d11b637 | Dhruba Borthakur | 10 July 2012, 00:31:12 UTC | bits_per_key is already configurable. It defines how many bloom bits will be used for every key in the database. My change in this patch is to make the Hash code that is used for blooms to be confgurable. In fact, one can specify a modified HashCode that inspects only parts of the Key to generate the Hash (used by booms). Test Plan: none Differential Revision: https://reviews.facebook.net/D4059 | 10 July 2012, 06:06:07 UTC |
| 80c6638 | Dhruba Borthakur | 03 July 2012, 05:45:59 UTC | Create leveldb server via Thrift. Summary: First draft. Unit tests pass. Test Plan: unit tests attached Reviewers: heyongqiang Reviewed By: heyongqiang Differential Revision: https://reviews.facebook.net/D3969 | 07 July 2012, 16:42:39 UTC |
| 22ee777 | heyongqiang | 06 July 2012, 18:42:09 UTC | add flush interface to DB Summary: as subject. The flush will flush everything in the db. Test Plan: new test in db_test.cc Reviewers: dhruba Reviewed By: dhruba Differential Revision: https://reviews.facebook.net/D4029 | 06 July 2012, 19:11:19 UTC |
| a347d4a | heyongqiang | 05 July 2012, 20:39:28 UTC | add disable WAL option Summary: add disable WAL option Test Plan: new testcase in db_test.cc Reviewers: dhruba Reviewed By: dhruba Differential Revision: https://reviews.facebook.net/D4011 | 05 July 2012, 22:06:56 UTC |
| 7600228 | heyongqiang | 03 July 2012, 00:30:32 UTC | fix compile warning Summary: as subject Test Plan: compile Reviewers: dhruba Reviewed By: dhruba Differential Revision: https://reviews.facebook.net/D3957 | 03 July 2012, 00:37:45 UTC |
| daa816c | heyongqiang | 29 June 2012, 02:26:43 UTC | add bzip2 compression Summary: add bzip2 compression Test Plan: testcases in table_test Reviewers: dhruba Reviewed By: dhruba Differential Revision: https://reviews.facebook.net/D3909 | 29 June 2012, 17:27:28 UTC |
| 054a565 | heyongqiang | 28 June 2012, 06:41:33 UTC | add zlib compression Summary: add zlib compression Test Plan: Will add more testcases Reviewers: dhruba Reviewed By: dhruba Differential Revision: https://reviews.facebook.net/D3873 | 28 June 2012, 23:28:57 UTC |
| 4e4b681 | heyongqiang | 23 June 2012, 02:30:03 UTC | Make some variables configurable for each db instance Summary: Make configurable 'targetFileSize', 'targetFileSizeMultiplier', 'maxBytesForLevelBase', 'maxBytesForLevelMultiplier', 'expandedCompactionFactor', 'maxGrandParentOverlapFactor' Test Plan: N/A Reviewers: dhruba Reviewed By: dhruba Differential Revision: https://reviews.facebook.net/D3801 | 27 June 2012, 21:36:31 UTC |
| 2067d22 | Dhruba Borthakur | 20 June 2012, 22:15:25 UTC | Support for java access to leveldb via leveldbjni. Test Plan: unit tests. Differential Revision: https://reviews.facebook.net/D3735 | 20 June 2012, 22:17:03 UTC |
| 636e3ec | Dhruba Borthakur | 19 June 2012, 08:47:15 UTC | Release version 1.4.0.fb. Summary: Test Plan: Reviewers: CC: Task ID: # Blame Rev: | 19 June 2012, 08:47:15 UTC |
| e9bc777 | Dhruba Borthakur | 19 June 2012, 07:56:07 UTC | While creating the leveldb shared library, use the shared library for jemalloc. Summary: Test Plan: Reviewers: CC: Task ID: # Blame Rev: | 19 June 2012, 07:56:07 UTC |
| 7e6c9b9 | Dhruba Borthakur | 15 June 2012, 20:48:59 UTC | A version of snappy that uses jemalloc and gcc-4.6.2, It is here for ease of use by leveldb code. Test Plan: Run benchmark Differential Revision: https://reviews.facebook.net/D3687 | 15 June 2012, 20:54:30 UTC |
| a35e574 | Dhruba Borthakur | 08 June 2012, 08:11:14 UTC | Make Leveldb save data into HDFS files. You have to set USE_HDFS in your environment variable to compile leveldb with HDFS support. Test Plan: Run benchmark. Differential Revision: https://reviews.facebook.net/D3549 | 14 June 2012, 07:29:01 UTC |
| 338939e | Dhruba Borthakur | 31 May 2012, 00:04:35 UTC | Print log message when we are throttling writes. Summary: Added option --writes=xxx to specify the number of keys that we want to overwrite in the benchmark. Task ID: # Blame Rev: Test Plan: Revert Plan: Reviewers: adsharma CC: sc Differential Revision: https://reviews.facebook.net/D3465 | 01 June 2012, 21:03:37 UTC |
| f50ece6 | Dhruba Borthakur | 30 May 2012, 06:18:16 UTC | Fix table-cache size bug, gather table-cache statistics and prevent readahead done by fs. Summary: Summary: The db_bench test was not using the specified value for the max-file-open. Fixed. The fs readhead is switched off. Gather statistics about the table cache and print it out at the end of the tets run. Test Plan: Revert Plan: Reviewers: adsharma, sc Reviewed By: adsharma Differential Revision: https://reviews.facebook.net/D3441 | 30 May 2012, 23:42:45 UTC |
| 8f293b6 | Dhruba Borthakur | 24 May 2012, 22:54:02 UTC | Support --bufferedio=[0,1] from db_bench. If bufferedio = 0, then the read code path clears the OS page cache after the IO is completed. The default remains as bufferedio=1 Summary: Task ID: # Blame Rev: Test Plan: Revert Plan: Differential Revision: https://reviews.facebook.net/D3429 | 29 May 2012, 20:29:44 UTC |
| 33a3c6f | Dhruba Borthakur | 22 May 2012, 19:18:23 UTC | Ability to make the benchmark issue a large number of IOs. This is helpful to populate many gigabytes of data for benchmarking at scale. Summary: Task ID: # Blame Rev: Test Plan: Revert Plan: Differential Revision: https://reviews.facebook.net/D3333 | 22 May 2012, 19:20:09 UTC |
| 3b86a51 | Dhruba Borthakur | 19 May 2012, 07:05:48 UTC | Ability to switch on checksum verification from benchmark. Summary: Task ID: # Blame Rev: Test Plan: Revert Plan: Differential Revision: https://reviews.facebook.net/D3309 | 19 May 2012, 07:13:50 UTC |
| a2a0e35 | Dhruba Borthakur | 17 May 2012, 00:22:33 UTC | Add support to specify the number of shards for the Block cache. By default, the block cache is sharded into 16 parts. Summary: Task ID: # Blame Rev: Test Plan: Revert Plan: Differential Revision: https://reviews.facebook.net/D3273 | 17 May 2012, 00:23:49 UTC |
| 95af128 | Arun Sharma | 04 May 2012, 01:10:17 UTC | SSE4 optimization Summary: This speeds up CRC computation significantly on hardware that supports it. Enabled via -msse4. Note: the binary won't be usable on older CPUs that don't support the instruction. Test Plan: crc32c_test Reviewers: dhruba Reviewed By: dhruba Differential Revision: https://reviews.facebook.net/D3201 | 15 May 2012, 17:10:01 UTC |
| 8d41351 | Dhruba Borthakur | 15 May 2012, 01:17:06 UTC | Ability to switch to gcc 4.1.6 and also use jemalloc. Thanks for Chip for large amounts of help. Summary: Task ID: # Blame Rev: Test Plan: Revert Plan: Reviewers: chip Reviewed By: chip CC: sc, adsharma Differential Revision: https://reviews.facebook.net/D3225 | 15 May 2012, 05:15:09 UTC |
| 921a484 | Arun Sharma | 04 May 2012, 00:26:14 UTC | Optimize for lp64 Summary: Some code reorganization in-preparation for replacing with a hardware instruction. * Use u64 for some of the key types * Use an ALIGN macro so code is easier to read Test Plan: crc32c_test Reviewers: dhruba Reviewed By: dhruba Differential Revision: https://reviews.facebook.net/D3135 | 14 May 2012, 22:40:11 UTC |
| 37d0dcb | Dhruba Borthakur | 11 May 2012, 18:32:37 UTC | Use the elapsed time (instead of the per-thread time) to compute ops/sec. Summary: Task ID: # Blame Rev: Test Plan: Revert Plan: Differential Revision: https://reviews.facebook.net/D3147 | 11 May 2012, 19:43:31 UTC |
| 90b2924 | Arun Sharma | 04 May 2012, 21:40:36 UTC | skiplist: optimize for sequential insert pattern Summary: skiplist doesn't cache the location of the last insert and becomes CPU bound when the input data has sequential keys. Notes on thread safety: ::Insert() already requires external synchronization. So this change is not making it any worse. Test Plan: skiplist_test Reviewers: dhruba Reviewed By: dhruba Differential Revision: https://reviews.facebook.net/D3129 | 11 May 2012, 16:57:40 UTC |
| cc6c325 | Dhruba Borthakur | 10 May 2012, 06:32:20 UTC | Support arcdiff. Summary: Task ID: # Blame Rev: Test Plan: Revert Plan: Differential Revision: https://reviews.facebook.net/D3105 | 10 May 2012, 06:35:05 UTC |
| 85584d4 | Sanjay Ghemawat | 17 April 2012, 15:36:46 UTC | Added bloom filter support. In particular, we add a new FilterPolicy class. An instance of this class can be supplied in Options when opening a database. If supplied, the instance is used to generate summaries of keys (e.g., a bloom filter) which are placed in sstables. These summaries are consulted by DB::Get() so we can avoid reading sstable blocks that are guaranteed to not contain the key we are looking for. This change provides one implementation of FilterPolicy based on bloom filters. Other changes: - Updated version number to 1.4. - Some build tweaks. - C binding for CompactRange. - A few more benchmarks: deleteseq, deleterandom, readmissing, seekrandom. - Minor .gitignore update. | 17 April 2012, 15:36:46 UTC |
| bc1ee4d | Sanjay Ghemawat | 30 March 2012, 20:15:49 UTC | build shared libraries; updated version to 1.3; add Status accessors | 30 March 2012, 20:15:49 UTC |
| a1ad4d1 | Sanjay Ghemawat | 21 March 2012, 17:28:03 UTC | Build fixes and cleanups: (1) Separate out C++ and CC flags (fixes c_test compilation) (2) Move snappy/perftools detection to script (3) Fix db_bench_sqlite3 and db_bench_tree_db build rules | 21 March 2012, 17:28:03 UTC |
| 9013f13 | Sanjay Ghemawat | 15 March 2012, 16:14:00 UTC | use mmap on 64-bit machines to speed-up reads; small build fixes | 15 March 2012, 16:14:00 UTC |
