| a098207 | Dhruba Borthakur | 22 August 2012, 00:02:21 UTC | Fixed unit test c_test by initializing logger=NULL. Summary: Fixed unit test c_test by initializing logger=NULL. Removed "atomic" from last_log_ts so that unit tests do not require C11 compiler. Anyway, last_log_ts is mostly used for logging, so it is ok if it is loosely accurate. Test Plan: run c_test Reviewers: heyongqiang Reviewed By: heyongqiang Differential Revision: https://reviews.facebook.net/D4803 | 22 August 2012, 00:10:29 UTC |
| 407f020 | Dhruba Borthakur | 21 August 2012, 22:43:52 UTC | Unit test corruption_test do not compile. Summary: Unit test corruption_test do not compile. Test Plan: run unit tests Reviewers: heyongqiang Reviewed By: heyongqiang Differential Revision: https://reviews.facebook.net/D4797 | 21 August 2012, 22:46:16 UTC |
| f4e7feb | Dhruba Borthakur | 21 August 2012, 07:33:21 UTC | Record the version of the source repository that was used to build the leveldb library. Summary: Record the version of the source that we are compiling. We keep a record of the git revision in util/version.cc. This source file is then built as a regular source file as part of the compilation process. One can run "strings executable_filename | grep _build_" to find the version of the source that we used to build the executable file. Test Plan: none Differential Revision: https://reviews.facebook.net/D4785 | 21 August 2012, 21:47:15 UTC |
| 6ba1f17 | heyongqiang | 14 August 2012, 22:20:36 UTC | adding a scribe logger in leveldb to log leveldb deploy stats Summary: as subject. A new log is written to scribe via thrift client when a new db is opened and when there is a compaction. a new option var scribe_log_db_stats is added. Test Plan: manually checked using command "ptail -time 0 leveldb_deploy_stats" Reviewers: dhruba Differential Revision: https://reviews.facebook.net/D4659 | 21 August 2012, 18:43:22 UTC |
| e56b2c5 | Dhruba Borthakur | 18 August 2012, 07:26:50 UTC | Prevent concurrent multiple opens of leveldb database. Summary: The fcntl call cannot detect lock conflicts when invoked multiple times from the same thread. Use a static lockedFile Set to record the paths that are locked. A lockfile request checks to see if htis filename already exists in lockedFiles, if so, then it triggers an error. Otherwise, it inserts the filename in the lockedFiles Set. A unlock file request verifies that the filename is in the lockedFiles set and removes it from lockedFiles set. Test Plan: unit test attached Reviewers: heyongqiang Reviewed By: heyongqiang Differential Revision: https://reviews.facebook.net/D4755 | 21 August 2012, 06:55:04 UTC |
| deb1a1f | heyongqiang | 19 August 2012, 23:05:02 UTC | add disable wal to db_bench Summary: as subject. ./db_bench --benchmarks=fillrandom --num=1000000 --disable_data_sync=1 --write_buffer_size=50000000 --target_file_size_base=100000000 --disable_wal=1 LevelDB: version 1.4 Date: Sun Aug 19 16:01:59 2012 CPU: 8 * Intel(R) Xeon(R) CPU L5630 @ 2.13GHz CPUCache: 12288 KB Keys: 16 bytes each Values: 100 bytes each (50 bytes after compression) Entries: 1000000 RawSize: 110.6 MB (estimated) FileSize: 62.9 MB (estimated) ------------------------------------------------ fillrandom : 4.591 micros/op 217797 ops/sec; 24.1 MB/s ./db_bench --benchmarks=fillrandom --num=1000000 --disable_data_sync=1 --write_buffer_size=50000000 --target_file_size_base=100000000 LevelDB: version 1.4 Date: Sun Aug 19 16:02:54 2012 CPU: 8 * Intel(R) Xeon(R) CPU L5630 @ 2.13GHz CPUCache: 12288 KB Keys: 16 bytes each Values: 100 bytes each (50 bytes after compression) Entries: 1000000 RawSize: 110.6 MB (estimated) FileSize: 62.9 MB (estimated) ------------------------------------------------ fillrandom : 3.696 micros/op 270530 ops/sec; 29.9 MB/s Test Plan: db_bench Reviewers: dhruba Differential Revision: https://reviews.facebook.net/D4767 | 20 August 2012, 05:37:51 UTC |
| 2aa514e | Dhruba Borthakur | 17 August 2012, 17:48:40 UTC | Utility to dump manifest contents. Summary: ./manifest_dump --file=/tmp/dbbench/MANIFEST-000002 Output looks like manifest_file_number 30 next_file_number 31 last_sequence 388082 log_number 28 prev_log_number 0 --- level 0 --- --- level 1 --- --- level 2 --- 5:3244155['0000000000000000' @ 1 : 1 .. '0000000000028220' @ 28221 : 1] 7:3244177['0000000000028221' @ 28222 : 1 .. '0000000000056441' @ 56442 : 1] 9:3244156['0000000000056442' @ 56443 : 1 .. '0000000000084662' @ 84663 : 1] 11:3244178['0000000000084663' @ 84664 : 1 .. '0000000000112883' @ 112884 : 1] 13:3244158['0000000000112884' @ 112885 : 1 .. '0000000000141104' @ 141105 : 1] 15:3244176['0000000000141105' @ 141106 : 1 .. '0000000000169325' @ 169326 : 1] 17:3244156['0000000000169326' @ 169327 : 1 .. '0000000000197546' @ 197547 : 1] 19:3244178['0000000000197547' @ 197548 : 1 .. '0000000000225767' @ 225768 : 1] 21:3244155['0000000000225768' @ 225769 : 1 .. '0000000000253988' @ 253989 : 1] 23:3244179['0000000000253989' @ 253990 : 1 .. '0000000000282209' @ 282210 : 1] 25:3244157['0000000000282210' @ 282211 : 1 .. '0000000000310430' @ 310431 : 1] 27:3244176['0000000000310431' @ 310432 : 1 .. '0000000000338651' @ 338652 : 1] 29:3244156['0000000000338652' @ 338653 : 1 .. '0000000000366872' @ 366873 : 1] --- level 3 --- --- level 4 --- --- level 5 --- --- level 6 --- Test Plan: run on test directory created by dbbench Reviewers: heyongqiang Reviewed By: heyongqiang CC: hustliubo Differential Revision: https://reviews.facebook.net/D4743 | 18 August 2012, 05:36:59 UTC |
| 680e571 | heyongqiang | 18 August 2012, 01:10:09 UTC | add compaction log Summary: Summary: add compaction summary to log log looks like: 2012/08/17-18:18:32.557334 7fdcaa2bb700 Compaction summary: Base level 0, input file:[11 9 7 ],[] Test Plan: tested via db_test Reviewers: dhruba Differential Revision: https://reviews.facebook.net/D4749 | 18 August 2012, 02:29:39 UTC |
| 29c99fc | heyongqiang | 17 August 2012, 23:23:15 UTC | Merge branch 'master' of https://github.com/facebook/leveldb | 17 August 2012, 23:23:15 UTC |
| 20ee76b | heyongqiang | 17 August 2012, 23:06:05 UTC | use ts as suffix for LOG.old files Summary: as subject and only maintain 10 log files. Test Plan: new test in db_test Reviewers: dhruba Differential Revision: https://reviews.facebook.net/D4731 | 17 August 2012, 23:22:04 UTC |
| b56ff5e | Dhruba Borthakur | 16 August 2012, 18:03:07 UTC | Expose new configration variables via the thrift api Summary: Expose new configration variables via the thrift api. when compiing for fbcode, always build thrift server Test Plan: none. Reviewers: heyongqiang Reviewed By: heyongqiang Differential Revision: https://reviews.facebook.net/D4689 | 17 August 2012, 17:29:12 UTC |
| 7c0b5ec | Dhruba Borthakur | 16 August 2012, 05:52:39 UTC | leveldb thrift server uses an incorrect root directory Summary: The offsets used to speficy hostname and dbdir from the command line were not handled correctly. Test Plan: none. Reviewers: heyongqiang Reviewed By: heyongqiang Differential Revision: https://reviews.facebook.net/D4683 | 16 August 2012, 18:10:08 UTC |
| f16e393 | heyongqiang | 15 August 2012, 22:44:23 UTC | add more options to db_ben Summary: as subject Test Plan: run db_bench with new options Reviewers: dhruba Differential Revision: https://reviews.facebook.net/D4677 | 16 August 2012, 00:42:33 UTC |
| fcb2ea4 | heyongqiang | 15 August 2012, 23:38:08 UTC | disable data sync options needs to be checked when doing level-0 dump Summary: Summary: as subject Test Plan: use db_bench Reviewers: dhruba Differential Revision: https://reviews.facebook.net/D4671 | 15 August 2012, 23:39:02 UTC |
| fe6119b | Dhruba Borthakur | 14 August 2012, 21:52:48 UTC | "make all check" fails Summary: If I run "make all check" it fails to build the leveldb-thrift-server tests. This is because the vanilla build is not supposed to build thrift and/or hdfs extensions. Remove the server test from auto-running. The thrift/hdfs build instructions are more elaborate and I would like to avoid it for embedded database builds. Test Plan: build and run Reviewers: heyongqiang Reviewed By: heyongqiang Differential Revision: https://reviews.facebook.net/D4641 | 14 August 2012, 22:55:38 UTC |
| c3096af | Dhruba Borthakur | 03 August 2012, 22:20:58 UTC | Introduce a new option disableDataSync for opening the database. If this is set to true, then the data written to newly created data files are not sycned to disk, instead depend on the OS to flush dirty data to stable storage. This option is good for bulk Test Plan: manual tests Task ID: # Blame Rev: Differential Revision: https://reviews.facebook.net/D4515 | 03 August 2012, 22:23:53 UTC |
| c44be54 | Dhruba Borthakur | 03 August 2012, 07:58:26 UTC | Make leveldb server not exit immediately. Summary: Test Plan: Reviewers: CC: Task ID: # Blame Rev: | 03 August 2012, 08:00:35 UTC |
| 2d04ab5 | Dhruba Borthakur | 03 August 2012, 07:33:47 UTC | Latest release is 1.4.1.fb Summary: Test Plan: Reviewers: CC: Task ID: # Blame Rev: | 03 August 2012, 07:33:47 UTC |
| 88c515b | Dhruba Borthakur | 03 August 2012, 07:27:48 UTC | Implement taoAssocRangeGet(). Summary: Test Plan: Reviewers: CC: Task ID: # Blame Rev: | 03 August 2012, 07:27:48 UTC |
| 0c98fdc | Dhruba Borthakur | 02 August 2012, 21:09:13 UTC | Encode id1Type, id2Type and data version the assoc payload. Summary: Test Plan: Reviewers: CC: Task ID: # Blame Rev: | 02 August 2012, 21:09:13 UTC |
| a3a8f83 | Dhruba Borthakur | 02 August 2012, 00:28:35 UTC | Fix "gcc -Wall" warnings for assoc-thrift code. Summary: Test Plan: Reviewers: CC: Task ID: # Blame Rev: | 02 August 2012, 00:28:35 UTC |
| d82cfe1 | Dhruba Borthakur | 01 August 2012, 23:55:39 UTC | Checkin pre-compiled snappy libraries for leveldb. Summary: Test Plan: Reviewers: CC: Task ID: # Blame Rev: | 01 August 2012, 23:55:39 UTC |
| b40ad06 | Dhruba Borthakur | 01 August 2012, 23:17:43 UTC | Implement the FB-Assoc API via thrift. Summary: Test Plan: Reviewers: CC: Task ID: # Blame Rev: | 01 August 2012, 23:18:24 UTC |
| 4abf94e | Dhruba Borthakur | 17 July 2012, 07:44:35 UTC | Check in compiled binaries for snappy. Summary: Test Plan: Reviewers: CC: Task ID: # Blame Rev: | 17 July 2012, 07:44:35 UTC |
| 71fa013 | Dhruba Borthakur | 17 July 2012, 07:28:12 UTC | Modify symbolic links for snappy builds. Summary: Test Plan: Reviewers: CC: Task ID: # Blame Rev: | 17 July 2012, 07:32:38 UTC |
| 4274e6c | Dhruba Borthakur | 17 July 2012, 07:29:06 UTC | Added README for snappy builds Summary: Test Plan: Reviewers: CC: Task ID: # Blame Rev: | 17 July 2012, 07:30:12 UTC |
| 21f2fe4 | Dhruba Borthakur | 17 July 2012, 07:18:23 UTC | Build snappy for leveldb. Summary: Test Plan: Reviewers: CC: Task ID: # Blame Rev: | 17 July 2012, 07:18:23 UTC |
| d550320 | Dhruba Borthakur | 11 July 2012, 21:08:46 UTC | Share a single cache for all the DBs served by this server. Summary: Test Plan: Reviewers: CC: Task ID: # Blame Rev: | 11 July 2012, 21:08:46 UTC |
| bfaa1bd | Dhruba Borthakur | 10 July 2012, 17:19:49 UTC | Cleanup thrift header files. Move thrift api from thrift/leveldb.thrift to thrift/if/leveldb.thrift Summary: Test Plan: Reviewers: CC: Task ID: # Blame Rev: | 10 July 2012, 17:19:49 UTC |
| d11b637 | Dhruba Borthakur | 10 July 2012, 00:31:12 UTC | bits_per_key is already configurable. It defines how many bloom bits will be used for every key in the database. My change in this patch is to make the Hash code that is used for blooms to be confgurable. In fact, one can specify a modified HashCode that inspects only parts of the Key to generate the Hash (used by booms). Test Plan: none Differential Revision: https://reviews.facebook.net/D4059 | 10 July 2012, 06:06:07 UTC |
| 80c6638 | Dhruba Borthakur | 03 July 2012, 05:45:59 UTC | Create leveldb server via Thrift. Summary: First draft. Unit tests pass. Test Plan: unit tests attached Reviewers: heyongqiang Reviewed By: heyongqiang Differential Revision: https://reviews.facebook.net/D3969 | 07 July 2012, 16:42:39 UTC |
| 22ee777 | heyongqiang | 06 July 2012, 18:42:09 UTC | add flush interface to DB Summary: as subject. The flush will flush everything in the db. Test Plan: new test in db_test.cc Reviewers: dhruba Reviewed By: dhruba Differential Revision: https://reviews.facebook.net/D4029 | 06 July 2012, 19:11:19 UTC |
| a347d4a | heyongqiang | 05 July 2012, 20:39:28 UTC | add disable WAL option Summary: add disable WAL option Test Plan: new testcase in db_test.cc Reviewers: dhruba Reviewed By: dhruba Differential Revision: https://reviews.facebook.net/D4011 | 05 July 2012, 22:06:56 UTC |
| 7600228 | heyongqiang | 03 July 2012, 00:30:32 UTC | fix compile warning Summary: as subject Test Plan: compile Reviewers: dhruba Reviewed By: dhruba Differential Revision: https://reviews.facebook.net/D3957 | 03 July 2012, 00:37:45 UTC |
| daa816c | heyongqiang | 29 June 2012, 02:26:43 UTC | add bzip2 compression Summary: add bzip2 compression Test Plan: testcases in table_test Reviewers: dhruba Reviewed By: dhruba Differential Revision: https://reviews.facebook.net/D3909 | 29 June 2012, 17:27:28 UTC |
| 054a565 | heyongqiang | 28 June 2012, 06:41:33 UTC | add zlib compression Summary: add zlib compression Test Plan: Will add more testcases Reviewers: dhruba Reviewed By: dhruba Differential Revision: https://reviews.facebook.net/D3873 | 28 June 2012, 23:28:57 UTC |
| 4e4b681 | heyongqiang | 23 June 2012, 02:30:03 UTC | Make some variables configurable for each db instance Summary: Make configurable 'targetFileSize', 'targetFileSizeMultiplier', 'maxBytesForLevelBase', 'maxBytesForLevelMultiplier', 'expandedCompactionFactor', 'maxGrandParentOverlapFactor' Test Plan: N/A Reviewers: dhruba Reviewed By: dhruba Differential Revision: https://reviews.facebook.net/D3801 | 27 June 2012, 21:36:31 UTC |
| 2067d22 | Dhruba Borthakur | 20 June 2012, 22:15:25 UTC | Support for java access to leveldb via leveldbjni. Test Plan: unit tests. Differential Revision: https://reviews.facebook.net/D3735 | 20 June 2012, 22:17:03 UTC |
| 636e3ec | Dhruba Borthakur | 19 June 2012, 08:47:15 UTC | Release version 1.4.0.fb. Summary: Test Plan: Reviewers: CC: Task ID: # Blame Rev: | 19 June 2012, 08:47:15 UTC |
| e9bc777 | Dhruba Borthakur | 19 June 2012, 07:56:07 UTC | While creating the leveldb shared library, use the shared library for jemalloc. Summary: Test Plan: Reviewers: CC: Task ID: # Blame Rev: | 19 June 2012, 07:56:07 UTC |
| 7e6c9b9 | Dhruba Borthakur | 15 June 2012, 20:48:59 UTC | A version of snappy that uses jemalloc and gcc-4.6.2, It is here for ease of use by leveldb code. Test Plan: Run benchmark Differential Revision: https://reviews.facebook.net/D3687 | 15 June 2012, 20:54:30 UTC |
| a35e574 | Dhruba Borthakur | 08 June 2012, 08:11:14 UTC | Make Leveldb save data into HDFS files. You have to set USE_HDFS in your environment variable to compile leveldb with HDFS support. Test Plan: Run benchmark. Differential Revision: https://reviews.facebook.net/D3549 | 14 June 2012, 07:29:01 UTC |
| 338939e | Dhruba Borthakur | 31 May 2012, 00:04:35 UTC | Print log message when we are throttling writes. Summary: Added option --writes=xxx to specify the number of keys that we want to overwrite in the benchmark. Task ID: # Blame Rev: Test Plan: Revert Plan: Reviewers: adsharma CC: sc Differential Revision: https://reviews.facebook.net/D3465 | 01 June 2012, 21:03:37 UTC |
| f50ece6 | Dhruba Borthakur | 30 May 2012, 06:18:16 UTC | Fix table-cache size bug, gather table-cache statistics and prevent readahead done by fs. Summary: Summary: The db_bench test was not using the specified value for the max-file-open. Fixed. The fs readhead is switched off. Gather statistics about the table cache and print it out at the end of the tets run. Test Plan: Revert Plan: Reviewers: adsharma, sc Reviewed By: adsharma Differential Revision: https://reviews.facebook.net/D3441 | 30 May 2012, 23:42:45 UTC |
| 8f293b6 | Dhruba Borthakur | 24 May 2012, 22:54:02 UTC | Support --bufferedio=[0,1] from db_bench. If bufferedio = 0, then the read code path clears the OS page cache after the IO is completed. The default remains as bufferedio=1 Summary: Task ID: # Blame Rev: Test Plan: Revert Plan: Differential Revision: https://reviews.facebook.net/D3429 | 29 May 2012, 20:29:44 UTC |
| 33a3c6f | Dhruba Borthakur | 22 May 2012, 19:18:23 UTC | Ability to make the benchmark issue a large number of IOs. This is helpful to populate many gigabytes of data for benchmarking at scale. Summary: Task ID: # Blame Rev: Test Plan: Revert Plan: Differential Revision: https://reviews.facebook.net/D3333 | 22 May 2012, 19:20:09 UTC |
| 3b86a51 | Dhruba Borthakur | 19 May 2012, 07:05:48 UTC | Ability to switch on checksum verification from benchmark. Summary: Task ID: # Blame Rev: Test Plan: Revert Plan: Differential Revision: https://reviews.facebook.net/D3309 | 19 May 2012, 07:13:50 UTC |
| a2a0e35 | Dhruba Borthakur | 17 May 2012, 00:22:33 UTC | Add support to specify the number of shards for the Block cache. By default, the block cache is sharded into 16 parts. Summary: Task ID: # Blame Rev: Test Plan: Revert Plan: Differential Revision: https://reviews.facebook.net/D3273 | 17 May 2012, 00:23:49 UTC |
| 95af128 | Arun Sharma | 04 May 2012, 01:10:17 UTC | SSE4 optimization Summary: This speeds up CRC computation significantly on hardware that supports it. Enabled via -msse4. Note: the binary won't be usable on older CPUs that don't support the instruction. Test Plan: crc32c_test Reviewers: dhruba Reviewed By: dhruba Differential Revision: https://reviews.facebook.net/D3201 | 15 May 2012, 17:10:01 UTC |
| 8d41351 | Dhruba Borthakur | 15 May 2012, 01:17:06 UTC | Ability to switch to gcc 4.1.6 and also use jemalloc. Thanks for Chip for large amounts of help. Summary: Task ID: # Blame Rev: Test Plan: Revert Plan: Reviewers: chip Reviewed By: chip CC: sc, adsharma Differential Revision: https://reviews.facebook.net/D3225 | 15 May 2012, 05:15:09 UTC |
| 921a484 | Arun Sharma | 04 May 2012, 00:26:14 UTC | Optimize for lp64 Summary: Some code reorganization in-preparation for replacing with a hardware instruction. * Use u64 for some of the key types * Use an ALIGN macro so code is easier to read Test Plan: crc32c_test Reviewers: dhruba Reviewed By: dhruba Differential Revision: https://reviews.facebook.net/D3135 | 14 May 2012, 22:40:11 UTC |
| 37d0dcb | Dhruba Borthakur | 11 May 2012, 18:32:37 UTC | Use the elapsed time (instead of the per-thread time) to compute ops/sec. Summary: Task ID: # Blame Rev: Test Plan: Revert Plan: Differential Revision: https://reviews.facebook.net/D3147 | 11 May 2012, 19:43:31 UTC |
| 90b2924 | Arun Sharma | 04 May 2012, 21:40:36 UTC | skiplist: optimize for sequential insert pattern Summary: skiplist doesn't cache the location of the last insert and becomes CPU bound when the input data has sequential keys. Notes on thread safety: ::Insert() already requires external synchronization. So this change is not making it any worse. Test Plan: skiplist_test Reviewers: dhruba Reviewed By: dhruba Differential Revision: https://reviews.facebook.net/D3129 | 11 May 2012, 16:57:40 UTC |
| cc6c325 | Dhruba Borthakur | 10 May 2012, 06:32:20 UTC | Support arcdiff. Summary: Task ID: # Blame Rev: Test Plan: Revert Plan: Differential Revision: https://reviews.facebook.net/D3105 | 10 May 2012, 06:35:05 UTC |
| 85584d4 | Sanjay Ghemawat | 17 April 2012, 15:36:46 UTC | Added bloom filter support. In particular, we add a new FilterPolicy class. An instance of this class can be supplied in Options when opening a database. If supplied, the instance is used to generate summaries of keys (e.g., a bloom filter) which are placed in sstables. These summaries are consulted by DB::Get() so we can avoid reading sstable blocks that are guaranteed to not contain the key we are looking for. This change provides one implementation of FilterPolicy based on bloom filters. Other changes: - Updated version number to 1.4. - Some build tweaks. - C binding for CompactRange. - A few more benchmarks: deleteseq, deleterandom, readmissing, seekrandom. - Minor .gitignore update. | 17 April 2012, 15:36:46 UTC |
| bc1ee4d | Sanjay Ghemawat | 30 March 2012, 20:15:49 UTC | build shared libraries; updated version to 1.3; add Status accessors | 30 March 2012, 20:15:49 UTC |
| a1ad4d1 | Sanjay Ghemawat | 21 March 2012, 17:28:03 UTC | Build fixes and cleanups: (1) Separate out C++ and CC flags (fixes c_test compilation) (2) Move snappy/perftools detection to script (3) Fix db_bench_sqlite3 and db_bench_tree_db build rules | 21 March 2012, 17:28:03 UTC |
| 9013f13 | Sanjay Ghemawat | 15 March 2012, 16:14:00 UTC | use mmap on 64-bit machines to speed-up reads; small build fixes | 15 March 2012, 16:14:00 UTC |
| 583f149 | Sanjay Ghemawat | 09 March 2012, 15:51:04 UTC | fix LOCK file deletion to prevent crash on windows | 09 March 2012, 15:51:04 UTC |
| d79762e | Sanjay Ghemawat | 09 March 2012, 00:23:21 UTC | added group commit; drastically speeds up mult-threaded synchronous write workloads | 09 March 2012, 00:23:21 UTC |
| 015d26f | Sanjay Ghemawat | 05 March 2012, 18:35:46 UTC | add .gitignore; support for building on a few BSD variants | 05 March 2012, 18:35:46 UTC |
| 239ac9d | Sanjay Ghemawat | 02 February 2012, 17:34:14 UTC | avoid very large compactions; fix build on Linux | 02 February 2012, 17:34:14 UTC |
| 3c8be10 | Sanjay Ghemawat | 25 January 2012, 22:56:52 UTC | fixed issues 66 (leaking files on disk error) and 68 (no sync of CURRENT file) | 25 January 2012, 22:56:52 UTC |
| c8c5866 | Hans Wennborg | 30 November 2011, 10:59:40 UTC | Makefile fixes for systems with $CXX other than g++. - Makefile: Use $(CXX) for compiling C++ files, don't override the environment's value of $CXX - build_detect_platform: use $CXX instead of g++. Based on bug report from Theo Schlossnagle: http://code.google.com/p/leveldb/issues/detail?id=46 (Sync with uptream at 25807040.) | 30 November 2011, 10:59:40 UTC |
| 42fb47f | Hans Wennborg | 14 November 2011, 17:06:16 UTC | Pass system's CFLAGS, remove exit time destructor, sstable bug fix. - Pass system's values of CFLAGS,LDFLAGS. Don't override OPT if it's already set. Original patch by Alessio Treglia <alessio@debian.org>: http://code.google.com/p/leveldb/issues/detail?id=27#c6 - Remove 1 exit time destructor from leveldb. See http://crbug.com/101600 - Fix problem where sstable building code would pass an internal key to the user comparator. (Sync with uptream at 25436817.) | 14 November 2011, 17:06:16 UTC |
| 36a5f8e | Hans Wennborg | 31 October 2011, 17:22:06 UTC | A number of fixes: - Replace raw slice comparison with a call to user comparator. Added test for custom comparators. - Fix end of namespace comments. - Fixed bug in picking inputs for a level-0 compaction. When finding overlapping files, the covered range may expand as files are added to the input set. We now correctly expand the range when this happens instead of continuing to use the old range. For example, suppose L0 contains files with the following ranges: F1: a .. d F2: c .. g F3: f .. j and the initial compaction target is F3. We used to search for range f..j which yielded {F2,F3}. However we now expand the range as soon as another file is added. In this case, when F2 is added, we expand the range to c..j and restart the search. That picks up file F1 as well. This change fixes a bug related to deleted keys showing up incorrectly after a compaction as described in Issue 44. (Sync with upstream @25072954) | 31 October 2011, 17:22:06 UTC |
| 299cced | Gabor Cselle | 05 October 2011, 23:30:28 UTC | A number of bugfixes: - Added DB::CompactRange() method. Changed manual compaction code so it breaks up compactions of big ranges into smaller compactions. Changed the code that pushes the output of memtable compactions to higher levels to obey the grandparent constraint: i.e., we must never have a single file in level L that overlaps too much data in level L+1 (to avoid very expensive L-1 compactions). Added code to pretty-print internal keys. - Fixed bug where we would not detect overlap with files in level-0 because we were incorrectly using binary search on an array of files with overlapping ranges. Added "leveldb.sstables" property that can be used to dump all of the sstables and ranges that make up the db state. - Removing post_write_snapshot support. Email to leveldb mailing list brought up no users, just confusion from one person about what it meant. - Fixing static_cast char to unsigned on BIG_ENDIAN platforms. Fixes Issue 35 and Issue 36. - Comment clarification to address leveldb Issue 37. - Change license in posix_logger.h to match other files. - A build problem where uint32 was used instead of uint32_t. Sync with upstream @24408625 | 05 October 2011, 23:30:28 UTC |
| 26db4d9 | Hans Wennborg | 26 September 2011, 16:37:09 UTC | Sync with upstream @24213649. Adding GNU/kFreeBSD support. As requested here: http://code.google.com/p/leveldb/issues/detail?id=38 Use uint64_t instead of size_t in MemEnvTest. As pointed out at http://code.google.com/p/leveldb/issues/detail?id=41 | 26 September 2011, 16:37:09 UTC |
| 213a68e | Hans Wennborg | 12 September 2011, 09:21:10 UTC | Sync with upstream @23860137. Fix GCC -Wshadow warnings in LevelDB's public header files, reported by Dustin. Add in-memory Env implementation (helpers/memenv/*). This enables users to create LevelDB databases in-memory. Initialize ShardedLRUCache::last_id_ to zero. This fixes a Valgrind warning. (Also delete port/sha1_* which were removed upstream some time ago.) | 12 September 2011, 09:21:10 UTC |
| 7263023 | gabor@google.com | 01 September 2011, 19:08:02 UTC | Bugfixes: for Get(), don't hold mutex while writing log. - Fix bug in Get: when it triggers a compaction, it could sometimes mark the compaction with the wrong level (if there was a gap in the set of levels examined for the Get). - Do not hold mutex while writing to the log file or to the MANIFEST file. Added a new benchmark that runs a writer thread concurrently with reader threads. Percentiles ------------------------------ micros/op: avg median 99 99.9 99.99 99.999 max ------------------------------------------------------ before: 42 38 110 225 32000 42000 48000 after: 24 20 55 65 130 1100 7000 - Fixed race in optimized Get. It should have been using the pinned memtables, not the current memtables. git-svn-id: https://leveldb.googlecode.com/svn/trunk@50 62dab493-f737-651d-591e-8d6aee1b9529 | 01 September 2011, 19:08:02 UTC |
| e3584f9 | gabor@google.com | 22 August 2011, 21:08:51 UTC | Bugfix for issue 33; reduce lock contention in Get(), parallel benchmarks. - Fix for issue 33 (non-null-terminated result from leveldb_property_value()) - Support for running multiple instances of a benchmark in parallel. - Reduce lock contention on Get(): (1) Do not hold the lock while searching memtables. (2) Shard block and table caches 16-ways. Benchmark for evaluating this change: $ db_bench --benchmarks=fillseq1,readrandom --threads=$n (fillseq1 is a small hack to make sure fillseq runs once regardless of number of threads specified on the command line). git-svn-id: https://leveldb.googlecode.com/svn/trunk@49 62dab493-f737-651d-591e-8d6aee1b9529 | 22 August 2011, 21:08:51 UTC |
| ab323f7 | gabor@google.com | 16 August 2011, 01:21:01 UTC | Bugfixes for iterator and documentation. - Fix bug in Iterator::Prev where it would return the wrong key. Fixes issues 29 and 30. - Added a tweak to testharness to allow running just some tests. - Fixing two minor documentation errors based on issues 28 and 25. - Cleanup; fix namespaces of export-to-C code. Also fix one "const char*" vs "char*" mismatch. git-svn-id: https://leveldb.googlecode.com/svn/trunk@48 62dab493-f737-651d-591e-8d6aee1b9529 | 16 August 2011, 01:21:01 UTC |
| a05525d | dgrogan@chromium.org | 06 August 2011, 00:19:37 UTC | @23023120 git-svn-id: https://leveldb.googlecode.com/svn/trunk@47 62dab493-f737-651d-591e-8d6aee1b9529 | 06 August 2011, 00:19:37 UTC |
| 021ee9c | gabor@google.com | 05 August 2011, 20:40:49 UTC | C binding for leveldb, better readseq benchmark for SQLite. - Added a C binding for LevelDB. May be useful as a stable ABI that can be used by programs that keep leveldb in a shared library, or for JNI API. - Replaced SQLite's readseq benchmark to a more efficient version. SQLite readseq speeds increased by about a factor of 2x from the previous version. Also updated benchmark page to reflect readseq speed up. git-svn-id: https://leveldb.googlecode.com/svn/trunk@46 62dab493-f737-651d-591e-8d6aee1b9529 | 05 August 2011, 20:40:49 UTC |
| 1bfbe76 | gabor@google.com | 29 July 2011, 21:35:05 UTC | Improved benchmark, fixed bugs and SQLite parameters. - Based on suggestions on the sqlite-users mailing list, we removed the superfluous index on the primary key for SQLite's benchmarks, and turned write-ahead logging ("WAL") on. This led to performance improvements for SQLite. - Based on a suggestion by Florian Weimer on the leveldb mailing list, we disabled hard drive write-caching via hdparm when testing synchronous writes. This led to performance losses for LevelDB and Kyoto TreeDB. - Fixed a mistake in 2.A.->Random where the bar sizes were switched for Kyoto TreeDB and SQLite. git-svn-id: https://leveldb.googlecode.com/svn/trunk@45 62dab493-f737-651d-591e-8d6aee1b9529 | 29 July 2011, 21:35:05 UTC |
| b9ef914 | gabor@google.com | 27 July 2011, 14:29:59 UTC | Minor typos in benchmark page. git-svn-id: https://leveldb.googlecode.com/svn/trunk@44 62dab493-f737-651d-591e-8d6aee1b9529 | 27 July 2011, 14:29:59 UTC |
| e8dee34 | gabor@google.com | 27 July 2011, 04:39:46 UTC | Minor edit in benchmark page. (Baseline comparison does not make sense for large values.) git-svn-id: https://leveldb.googlecode.com/svn/trunk@43 62dab493-f737-651d-591e-8d6aee1b9529 | 27 July 2011, 04:39:46 UTC |
| 3cc2738 | gabor@google.com | 27 July 2011, 01:56:52 UTC | Setting SVN mime-type for benchmark page. git-svn-id: https://leveldb.googlecode.com/svn/trunk@42 62dab493-f737-651d-591e-8d6aee1b9529 | 27 July 2011, 01:56:52 UTC |
| e301f17 | gabor@google.com | 27 July 2011, 01:49:08 UTC | Adding doctype to benchmark page so Google Code displays it as HTML. git-svn-id: https://leveldb.googlecode.com/svn/trunk@41 62dab493-f737-651d-591e-8d6aee1b9529 | 27 July 2011, 01:49:08 UTC |
| f122c6d | gabor@google.com | 27 July 2011, 01:46:25 UTC | Adding FreeBSD support, removing Chromium files, adding benchmark. - LevelDB patch for FreeBSD. This resolves Issue 22. Contributed by dforsythe (thanks!). - Removing Chromium-specific files. They are now going to live in the Chromium repository. - Adding a benchmark page comparing LevelDB performance to SQLite and Kyoto Cabinet's TreeDB, along with code to generate the benchmarks. Thanks to Kevin Tseng for compiling the benchmarks, and Scott Hess and Mikio Hirabayashi for their help and advice. git-svn-id: https://leveldb.googlecode.com/svn/trunk@40 62dab493-f737-651d-591e-8d6aee1b9529 | 27 July 2011, 01:46:25 UTC |
| 60bd801 | gabor@google.com | 21 July 2011, 02:40:18 UTC | Speed up Snappy uncompression, new Logger interface. - Removed one copy of an uncompressed block contents changing the signature of Snappy_Uncompress() so it uncompresses into a flat array instead of a std::string. Speeds up readrandom ~10%. - Instead of a combination of Env/WritableFile, we now have a Logger interface that can be easily overridden applications that want to supply their own logging. - Separated out the gcc and Sun Studio parts of atomic_pointer.h so we can use 'asm', 'volatile' keywords for Sun Studio. git-svn-id: https://leveldb.googlecode.com/svn/trunk@39 62dab493-f737-651d-591e-8d6aee1b9529 | 21 July 2011, 02:40:18 UTC |
| 6872ace | gabor@google.com | 19 July 2011, 23:36:47 UTC | Sun Studio support, and fix for test related memory fixes. - LevelDB patch for Sun Studio Based on a patch submitted by Theo Schlossnagle - thanks! This fixes Issue 17. - Fix a couple of test related memory leaks. git-svn-id: https://leveldb.googlecode.com/svn/trunk@38 62dab493-f737-651d-591e-8d6aee1b9529 | 19 July 2011, 23:36:47 UTC |
| 6699c7e | gabor@google.com | 15 July 2011, 00:20:57 UTC | Small tweaks and bugfixes for Issue 18 and 19. Slight tweak to the no-overlap optimization: only push to level 2 to reduce the amount of wasted space when the same small key range is being repeatedly overwritten. Fix for Issue 18: Avoid failure on Windows by avoiding deletion of lock file until the end of DestroyDB(). Fix for Issue 19: Disregard sequence numbers when checking for overlap in sstable ranges. This fixes issue 19: when writing the same key over and over again, we would generate a sequence of sstables that were never merged together since their sequence numbers were disjoint. Don't ignore map/unmap error checks. Miscellaneous fixes for small problems Sanjay found while diagnosing issue/9 and issue/16 (corruption_testr failures). - log::Reader reports the record type when it finds an unexpected type. - log::Reader no longer reports an error when it encounters an expected zero record regardless of the setting of the "checksum" flag. - Added a missing forward declaration. - Documented a side-effects of larger write buffer sizes (longer recovery time). git-svn-id: https://leveldb.googlecode.com/svn/trunk@37 62dab493-f737-651d-591e-8d6aee1b9529 | 15 July 2011, 00:20:57 UTC |
| ed154f6 | gabor@google.com | 30 June 2011, 23:17:03 UTC | Fixed a snappy compression wrapper bug (passing wrong variable). Change atomic_pointer.h to prefer a memory barrier based implementation over a <cstdatomic> based implementation for the following reasons: (1) On a x86-32-bit gcc-4.4 build, <ctdatomic> was corrupting the AtomicPointer. (2) On a x86-64-bit gcc build, a <ctstdatomic> based acquire-load takes ~15ns as opposed to the ~1ns for a memory-barrier based implementation. Fixes issue 9 (corruption_test fails) http://code.google.com/p/leveldb/issues/detail?id=9 Fixes issue 16 (CorruptionTest.MissingDescriptor fails) http://code.google.com/p/leveldb/issues/detail?id=16 git-svn-id: https://leveldb.googlecode.com/svn/trunk@36 62dab493-f737-651d-591e-8d6aee1b9529 | 30 June 2011, 23:17:03 UTC |
| 85f0ab1 | gabor@google.com | 29 June 2011, 22:53:17 UTC | Fixing Makefile issue reported in Issue 15 (misspelled flag) git-svn-id: https://leveldb.googlecode.com/svn/trunk@35 62dab493-f737-651d-591e-8d6aee1b9529 | 29 June 2011, 22:53:17 UTC |
| f57e233 | gabor@google.com | 29 June 2011, 00:30:50 UTC | Platform detection during build, plus compatibility patches for machines without <cstdatomic>. This revision adds two major changes: 1. build_detect_platform which generates build_config.mk with platform-dependent flags for the build process 2. /port/atomic_pointer.h with anAtomicPointerimplementation for platforms without <cstdatomic> Some of this code is loosely based on patches submitted to the LevelDB mailing list at https://groups.google.com/forum/#!forum/leveldb Tip of the hat to Dave Smith and Edouard A, who both sent patches. The presence of Snappy (http://code.google.com/p/snappy/) and cstdatomic are now both detected in the build_detect_platform script (1.) which gets executing during make. For (2.), instead of broadly importing atomicops_* from Chromium or the Google performance tools, we chose to just implement AtomicPointer and the limited atomic load and store operations it needs. This resulted in much less code and fewer files - everything is contained in atomic_pointer.h. git-svn-id: https://leveldb.googlecode.com/svn/trunk@34 62dab493-f737-651d-591e-8d6aee1b9529 | 29 June 2011, 00:30:50 UTC |
| e0cbd24 | gabor@google.com | 22 June 2011, 18:45:39 UTC | Fixing issue 11: version_set_test.cc was missing git-svn-id: https://leveldb.googlecode.com/svn/trunk@33 62dab493-f737-651d-591e-8d6aee1b9529 | 22 June 2011, 18:45:39 UTC |
| ccf0fcd | gabor@google.com | 22 June 2011, 02:36:45 UTC | A number of smaller fixes and performance improvements: - Implemented Get() directly instead of building on top of a full merging iterator stack. This speeds up the "readrandom" benchmark by up to 15-30%. - Fixed an opensource compilation problem. Added --db=<name> flag to control where the database is placed. - Automatically compact a file when we have done enough overlapping seeks to that file. - Fixed a performance bug where we would read from at least one file in a level even if none of the files overlapped the key being read. - Makefile fix for Mac OSX installations that have XCode 4 without XCode 3. - Unified the two occurrences of binary search in a file-list into one routine. - Found and fixed a bug where we would unnecessarily search the last file when looking for a key larger than all data in the level. - A fix to avoid the need for trivial move compactions and therefore gets rid of two out of five syncs in "fillseq". - Removed the MANIFEST file write when switching to a new memtable/log-file for a 10-20% improvement on fill speed on ext4. - Adding a SNAPPY setting in the Makefile for folks who have Snappy installed. Snappy compresses values and speeds up writes. git-svn-id: https://leveldb.googlecode.com/svn/trunk@32 62dab493-f737-651d-591e-8d6aee1b9529 | 22 June 2011, 02:36:45 UTC |
| 80e5b0d | hans@chromium.org | 07 June 2011, 14:40:26 UTC | sync with upstream @21706995 Fixed race condition reported by Dave Smit (dizzyd@dizzyd,com) on the leveldb mailing list. We were not signalling waiters after a trivial move from level-0. The result was that in some cases (hard to reproduce), a write would get stuck forever waiting for the number of level-0 files to drop below its hard limit. The new code is simpler: there is just one condition variable instead of two, and the condition variable is signalled after every piece of background work finishes. Also, all compaction work (including for manual compactions) is done in the background thread, and therefore we can remove the "compacting_" variable. git-svn-id: https://leveldb.googlecode.com/svn/trunk@31 62dab493-f737-651d-591e-8d6aee1b9529 | 07 June 2011, 14:40:26 UTC |
| c4f5514 | dgrogan@chromium.org | 02 June 2011, 00:00:37 UTC | sync with upstream @21627589 Minor changes: * Reformat the bodies of the iterator interface routines in IteratorWrapper to make them a bit easier to read * Switched the default in the leveldb makefile to be optimized mode, rather than debug mode * Fix build problem in chromium port git-svn-id: https://leveldb.googlecode.com/svn/trunk@30 62dab493-f737-651d-591e-8d6aee1b9529 | 02 June 2011, 00:00:37 UTC |
| 740d8b3 | dgrogan@chromium.org | 28 May 2011, 00:53:58 UTC | Update from upstream @21551990 * Patch LevelDB to build for OSX and iOS * Fix race condition in memtable iterator deletion. * Other small fixes. git-svn-id: https://leveldb.googlecode.com/svn/trunk@29 62dab493-f737-651d-591e-8d6aee1b9529 | 28 May 2011, 00:53:58 UTC |
| da79909 | dgrogan@chromium.org | 21 May 2011, 02:17:43 UTC | sync with upstream @ 21409451 Check the NEWS file for details of what changed. git-svn-id: https://leveldb.googlecode.com/svn/trunk@28 62dab493-f737-651d-591e-8d6aee1b9529 | 21 May 2011, 02:17:43 UTC |
| 3c11133 | dgrogan@chromium.org | 03 May 2011, 03:10:59 UTC | make windows include /Iport\win in dependent projects git-svn-id: https://leveldb.googlecode.com/svn/trunk@27 62dab493-f737-651d-591e-8d6aee1b9529 | 03 May 2011, 03:10:59 UTC |
| be9f061 | dgrogan@chromium.org | 21 April 2011, 01:54:51 UTC | pull in hans' mac build fix git-svn-id: https://leveldb.googlecode.com/svn/trunk@26 62dab493-f737-651d-591e-8d6aee1b9529 | 21 April 2011, 01:54:51 UTC |
| ccb2cbe | dgrogan@chromium.org | 20 April 2011, 22:50:04 UTC | fix build on at least linux git-svn-id: https://leveldb.googlecode.com/svn/trunk@25 62dab493-f737-651d-591e-8d6aee1b9529 | 20 April 2011, 22:50:04 UTC |
| ba6dac0 | dgrogan@chromium.org | 20 April 2011, 22:48:11 UTC | @20776309 * env_chromium.cc should not export symbols. * Fix MSVC warnings. * Removed large value support. * Fix broken reference to documentation file git-svn-id: https://leveldb.googlecode.com/svn/trunk@24 62dab493-f737-651d-591e-8d6aee1b9529 | 20 April 2011, 22:48:11 UTC |
| 69c6d38 | dgrogan@chromium.org | 19 April 2011, 23:11:15 UTC | reverting disastrous MOE commit, returning to r21 git-svn-id: https://leveldb.googlecode.com/svn/trunk@23 62dab493-f737-651d-591e-8d6aee1b9529 | 19 April 2011, 23:11:15 UTC |
| b743906 | dgrogan@chromium.org | 19 April 2011, 23:01:25 UTC | Revision created by MOE tool push_codebase. MOE_MIGRATION= git-svn-id: https://leveldb.googlecode.com/svn/trunk@22 62dab493-f737-651d-591e-8d6aee1b9529 | 19 April 2011, 23:01:25 UTC |
| b409afe | dgrogan@chromium.org | 18 April 2011, 23:15:58 UTC | chmod a-x git-svn-id: https://leveldb.googlecode.com/svn/trunk@21 62dab493-f737-651d-591e-8d6aee1b9529 | 18 April 2011, 23:15:58 UTC |
| f779e7a | dgrogan@chromium.org | 12 April 2011, 19:38:58 UTC | @20602303. Default file permission is now 755. git-svn-id: https://leveldb.googlecode.com/svn/trunk@20 62dab493-f737-651d-591e-8d6aee1b9529 | 12 April 2011, 19:38:58 UTC |
