| 146b771 | Tamir Duberstein | 18 May 2017, 05:49:15 UTC | Fix mingw compilation with -DNDEBUG Summary: This was exposed by a48a62d, which made NDEBUG the default for cmake builds. Closes https://github.com/facebook/rocksdb/pull/2315 Differential Revision: D5079583 Pulled By: sagar0 fbshipit-source-id: c614e96a40df016a834a62b6236852265e7ee4db | 18 May 2017, 05:56:48 UTC |
| a36220c | Aaron Gao | 18 May 2017, 01:46:03 UTC | fix unity test Summary: unity test will fail even if we have the same function names in different anonymous namespaces in different files. Closes https://github.com/facebook/rocksdb/pull/2321 Differential Revision: D5083783 Pulled By: lightmark fbshipit-source-id: 1347aaf866900af30d23cdd4f29c1b96f17352af | 18 May 2017, 01:56:55 UTC |
| 0ebdd70 | hyunwoo | 17 May 2017, 23:38:05 UTC | fixed typo Summary: fixed typo Closes https://github.com/facebook/rocksdb/pull/2312 Differential Revision: D5079631 Pulled By: sagar0 fbshipit-source-id: e4c8d1d89b244ee69e9dea1dd013227cc5241026 | 17 May 2017, 23:41:49 UTC |
| 8032f4c | Siying Dong | 17 May 2017, 22:49:06 UTC | Remove -pie in TSAN Summary: -pic seems to be not working in gcc-5 and it is curently broken. Remove it to fix the build. Closes https://github.com/facebook/rocksdb/pull/2320 Differential Revision: D5082775 Pulled By: siying fbshipit-source-id: 5055f987353f1417643a394e7ce05905670410a4 | 17 May 2017, 22:56:35 UTC |
| 362ba9b | Aaron Gao | 17 May 2017, 19:34:39 UTC | Release RocksDB 5.5.0 Summary: change history.md and version Closes https://github.com/facebook/rocksdb/pull/2317 Differential Revision: D5080484 Pulled By: lightmark fbshipit-source-id: 8d70b3b52dc0d34fefc0d34f91d379c27ac13ed3 | 17 May 2017, 19:42:20 UTC |
| ba685a4 | Mikhail Antonov | 17 May 2017, 18:32:26 UTC | Support ingest_behind for IngestExternalFile Summary: First cut for early review; there are few conceptual points to answer and some code structure issues. For conceptual points - - restriction-wise, we're going to disallow ingest_behind if (use_seqno_zero_out=true || disable_auto_compaction=false), the user is responsible to properly open and close DB with required params - we wanted to ingest into reserved bottom most level. Should we fail fast if bottom level isn't empty, or should we attempt to ingest if file fits there key-ranges-wise? - Modifying AssignLevelForIngestedFile seems the place we we'd handle that. On code structure - going to refactor GenerateAndAddExternalFile call in the test class to allow passing instance of IngestionOptions, that's just going to incur lots of changes at callsites. Closes https://github.com/facebook/rocksdb/pull/2144 Differential Revision: D4873732 Pulled By: lightmark fbshipit-source-id: 81cb698106b68ef8797f564453651d50900e153a | 17 May 2017, 18:42:42 UTC |
| 01ab7b5 | Sagar Vemuri | 17 May 2017, 17:10:22 UTC | Add ROCKSDB_LIBRARY_API macro to a few C APIs, to fix windows build Summary: Windows build in AppVeyor is broken, I believe due to https://github.com/facebook/rocksdb/pull/2254. Error messages: ``` c_test.obj : error LNK2019: unresolved external symbol rocksdb_get_pinned referenced in function CheckPinGet [C:\projects\rocksdb\build\c_test.vcxproj] c_test.obj : error LNK2019: unresolved external symbol rocksdb_get_pinned_cf referenced in function CheckPinGetCF [C:\projects\rocksdb\build\c_test.vcxproj] c_test.obj : error LNK2019: unresolved external symbol rocksdb_pinnableslice_destroy referenced in function CheckPinGet [C:\projects\rocksdb\build\c_test.vcxproj] c_test.obj : error LNK2019: unresolved external symbol rocksdb_pinnableslice_value referenced in function CheckPinGet [C:\projects\rocksdb\build\c_test.vcxproj] C:\projects\rocksdb\build\Debug\c_test.exe : fatal error LNK1120: 4 unresolved externals [C:\projects\rocksdb\build\c_test.vcxproj] ``` See, for example: https://ci.appveyor.com/project/Facebook/rocksdb/build/1.0.4420 Closes https://github.com/facebook/rocksdb/pull/2309 Differential Revision: D5076992 Pulled By: sagar0 fbshipit-source-id: bf4ca063a53b5a9042ba9f655f7c60c268ea5748 | 17 May 2017, 17:22:41 UTC |
| cb9392a | boolean5 | 17 May 2017, 05:57:05 UTC | add Transactions and Checkpoint to C API Summary: I've added functions to the C API to support Transactions as requested in #1637 and to support Checkpoint. I have also added the corresponding tests to c_test.c For now, the following is omitted: 1. Optimistic Transactions 2. The column family variation of functions Closes https://github.com/facebook/rocksdb/pull/2236 Differential Revision: D4989510 Pulled By: yiwu-arbug fbshipit-source-id: 518cb39f76d5e9ec9690d633fcdc014b98958071 | 17 May 2017, 05:59:43 UTC |
| 445f123 | Yi Wu | 16 May 2017, 18:50:43 UTC | s/std::snprintf/snprintf Summary: Looks like std::snprintf is not available on all platforms (e.g. MSVC 2010). Change it back to snprintf, where we have a macro in port.h to workaround compatibility. Closes https://github.com/facebook/rocksdb/pull/2308 Differential Revision: D5070988 Pulled By: yiwu-arbug fbshipit-source-id: bedfc1660bab0431c583ad434b7e68265e1211b1 | 16 May 2017, 19:01:04 UTC |
| cd593c2 | Yi Wu | 16 May 2017, 18:20:03 UTC | Fix travis java_test Summary: Travis java_test is failing because `make clean` happens after `build_version.cc` is generated. Fixing it. Closes #2299 Closes https://github.com/facebook/rocksdb/pull/2307 Differential Revision: D5070781 Pulled By: yiwu-arbug fbshipit-source-id: 18d3ea11d602048f4786b8c7a41f49c2aee774cd | 16 May 2017, 18:21:24 UTC |
| f720796 | hyunwoo | 16 May 2017, 17:56:31 UTC | fixed typo Summary: fixed exisitng -> existing Closes https://github.com/facebook/rocksdb/pull/2305 Differential Revision: D5070169 Pulled By: yiwu-arbug fbshipit-source-id: 8c8450acf50757b767cf78b78314018395738d96 | 16 May 2017, 18:07:58 UTC |
| a48a62d | Nikhil Benesch | 16 May 2017, 17:31:24 UTC | define NDEBUG in CMake non-debug builds Summary: This brings CMake builds further in line with builds that go through the normal Makefile. Closes https://github.com/facebook/rocksdb/pull/2300 Differential Revision: D5064631 Pulled By: yiwu-arbug fbshipit-source-id: 7b2b2d5299f575f87badcf590cc95e040f14d52d | 16 May 2017, 18:07:58 UTC |
| 1ca723d | siddontang | 16 May 2017, 17:24:03 UTC | C API: support pinnable get Summary: Closes https://github.com/facebook/rocksdb/pull/2254 Differential Revision: D5053590 Pulled By: yiwu-arbug fbshipit-source-id: 2f365a031b3a2947b4fba21d26d4f8f52af9b9f0 | 16 May 2017, 18:07:58 UTC |
| 2ef15b8 | Andrew Kryczka | 16 May 2017, 17:04:41 UTC | Core-local stats blog post Summary: as titled Closes https://github.com/facebook/rocksdb/pull/2294 Differential Revision: D5070009 Pulled By: siying fbshipit-source-id: 88e41839322e7f7d4b00f7057c58bd8e7d012dad | 16 May 2017, 18:07:58 UTC |
| 4f9e69c | 赵星宇 | 15 May 2017, 21:46:01 UTC | fix log err Summary: Closes https://github.com/facebook/rocksdb/pull/2206 Differential Revision: D5054222 Pulled By: yiwu-arbug fbshipit-source-id: d8742bda1bf3e76d7b68eeb86df4608031b5cbc8 | 15 May 2017, 23:15:38 UTC |
| 11c5d47 | Nikhil Benesch | 15 May 2017, 21:42:32 UTC | cross-platform compatibility improvements Summary: We've had a couple CockroachDB users fail to build RocksDB on exotic platforms, so I figured I'd try my hand at solving these issues upstream. The problems stem from a) `USE_SSE=1` being too aggressive about turning on SSE4.2, even on toolchains that don't support SSE4.2 and b) RocksDB attempting to detect support for thread-local storage based on OS, even though it can vary by compiler on the same OS. See the individual commit messages for details. Regarding SSE support, this PR should change virtually nothing for non-CMake based builds. `make`, `PORTABLE=1 make`, `USE_SSE=1 make`, and `PORTABLE=1 USE_SSE=1 make` function exactly as before, except that SSE support will be automatically disabled when a simple SSE4.2-using test program fails to compile, as it does on OpenBSD. (OpenBSD's ports GCC supports SSE4.2, but its binutils do not, so `__SSE_4_2__` is defined but an SSE4.2-using program will fail to assemble.) A warning is emitted in this case. The CMake build is modified to support the same set of options, except that `USE_SSE` is spelled `FORCE_SSE42` because `USE_SSE` is rather useless now that we can automatically detect SSE support, and I figure changing options in the CMake build is less disruptive than changing the non-CMake build. I've tested these changes on all the platforms I can get my hands on (macOS, Windows MSVC, Windows MinGW, and OpenBSD) and it all works splendidly. Let me know if there's anything you object to—I obviously don't mean to break any of your build pipelines in the process of fixing ours downstream. Closes https://github.com/facebook/rocksdb/pull/2199 Differential Revision: D5054042 Pulled By: yiwu-arbug fbshipit-source-id: 938e1fc665c049c02ae15698e1409155b8e72171 | 15 May 2017, 23:15:38 UTC |
| d004333 | Jeff Li | 15 May 2017, 20:49:00 UTC | Put lib files into suitable path in RPM package Summary: Currently, the RPM package will install the lib and header files into `/usr/package/lib` and `/usr/package/include` which is not in the default search paths. It is reasonable to install them under `/usr/lib` and `/usr/include` so that no extra configuration is required. Closes https://github.com/facebook/rocksdb/pull/2221 Differential Revision: D5054030 Pulled By: yiwu-arbug fbshipit-source-id: 1d23de5ff21f07e6738c9dfa04429acd7a839143 | 15 May 2017, 21:05:46 UTC |
| 86d5492 | Yi Wu | 15 May 2017, 20:40:32 UTC | Fix build error with blob DB. Summary: snprintf is in <stdio.h> and not in namespace std. Closes https://github.com/facebook/rocksdb/pull/2287 Reviewed By: anirbanr-fb Differential Revision: D5054752 Pulled By: yiwu-arbug fbshipit-source-id: 356807ec38f3c7d95951cdb41f31a3d3ae0714d4 | 15 May 2017, 21:05:46 UTC |
| 254c468 | Adam Retter | 15 May 2017, 19:27:46 UTC | Fix the RocksJava Release on Windows Summary: This was previously broken accidentally by https://github.com/facebook/rocksdb/pull/2107 Closes https://github.com/facebook/rocksdb/issues/2293 Closes https://github.com/facebook/rocksdb/pull/2296 Differential Revision: D5061248 Pulled By: sagar0 fbshipit-source-id: 7f58fee754723a7052d2a7f9d3d0369051c3cc5c | 15 May 2017, 19:37:01 UTC |
| 7a47b43 | Sergei Vorobev | 15 May 2017, 19:11:22 UTC | Fix .gitignore pattern Summary: `java/**.asc` is not a correct gitignore pattern See https://git-scm.com/docs/gitignore for the list of allowed `**` patterns It seems reasonable to assume that intention is `java/**/*.asc` The reason why it bothers me is the fact that ripgrep parses .gitignore files and complains about invalid pattern https://github.com/BurntSushi/ripgrep Closes https://github.com/facebook/rocksdb/pull/2214 Differential Revision: D5063030 Pulled By: yiwu-arbug fbshipit-source-id: ddd6682b81f03134be15f20fd596130776b69695 | 15 May 2017, 19:23:10 UTC |
| fa5a15c | Adam Retter | 15 May 2017, 17:45:14 UTC | Make sure that zstd is statically linked correctly in the Java static build Summary: Closes https://github.com/facebook/rocksdb/issues/2280 Closes https://github.com/facebook/rocksdb/pull/2292 Differential Revision: D5061259 Pulled By: sagar0 fbshipit-source-id: eec89111d114c04beee5870a4eb4b51857754783 | 15 May 2017, 18:12:08 UTC |
| 3fa9a39 | Andrew Kryczka | 12 May 2017, 21:59:57 UTC | Add GetAllKeyVersions API Summary: - Introduced an include/ file dedicated to db-related debug functions to avoid making db.h more complex - Added debugging function, `GetAllKeyVersions()`, to return a listing of internal data for a range of user keys. The new `struct KeyVersion` exposes data similar to internal key without exposing any internal type. - Migrated the "ldb idump" subcommand to use this function - The API takes an inclusive-exclusive range to match behavior of "ldb idump". This will be quite annoying for users who want to query a single user key's versions :(. Closes https://github.com/facebook/rocksdb/pull/2232 Differential Revision: D4976007 Pulled By: ajkr fbshipit-source-id: cab375da53a7595d6575af2b7e3b776aa3ad793e | 12 May 2017, 22:54:06 UTC |
| 1a60982 | Adam Retter | 12 May 2017, 18:58:05 UTC | Simplified instructions for CentOS Summary: Minimal changes for improvements to INSTALL.md for CentOS/RHEL Closes https://github.com/facebook/rocksdb/pull/2266 Differential Revision: D5053530 Pulled By: yiwu-arbug fbshipit-source-id: c05f30299f8efec949b9001a73969d649536ea8a | 12 May 2017, 19:23:18 UTC |
| a5cc7ec | Adam Retter | 12 May 2017, 18:23:14 UTC | Facility for cross-building RocksJava using Docker Summary: As an alternative to Vagrant, we can now also use Docker to cross-build RocksDB. The advantages are: 1. The Docker images are fixed; they include all the latest updates and build tools. 2. The Vagrant image, required scripts that ran for every build that would update CentOS and install the buildtools. This lead to slow repeatable builds, we don't need to do this with Docker as they are already in the provided images. The Docker images I have used have their Docker build files here: https://github.com/evolvedbinary/docker-rocksjava and the images themselves are available from Docker hub: https://hub.docker.com/r/evolvedbinary/rocksjava/ I have added the following targets to the `Makefile`: 1. `rocksdbjavastaticreleasedocker` this uses Docker to perform the cross-builds. It is basically the Docker version of the existing Vagrant `rocksdbjavastaticrelease` target. 2. `rocksdbjavastaticpublishdocker` delegates to `rocksdbjavastaticreleasedocker` and then `rocksdbjavastaticpublishcentral` to upload the artiacts to Maven Central. Equivalent to the existing Vagrant target: `rocksdbjavastaticpublish` Closes https://github.com/facebook/rocksdb/pull/2278 Differential Revision: D5048206 Pulled By: yiwu-arbug fbshipit-source-id: 78fa96ef9d966fe09638ed01de282cd4e31961a9 | 12 May 2017, 18:41:21 UTC |
| ccd3ddd | Maysam Yabandeh | 12 May 2017, 17:42:37 UTC | Blog post for partitioned index/filters Summary: Closes https://github.com/facebook/rocksdb/pull/2285 Differential Revision: D5053096 Pulled By: maysamyabandeh fbshipit-source-id: c696a6b82391d039c446d16222313f78341c37e8 | 12 May 2017, 18:01:30 UTC |
| b145c34 | Maysam Yabandeh | 12 May 2017, 17:16:36 UTC | Update blog authors Summary: Closes https://github.com/facebook/rocksdb/pull/2284 Differential Revision: D5052835 Pulled By: maysamyabandeh fbshipit-source-id: afe5fdb4b2d659cf612446a666dbc1d11afc3b5d | 12 May 2017, 18:01:30 UTC |
| bbe9ee7 | Andrew Kryczka | 12 May 2017, 16:26:40 UTC | core-local array type conversions Summary: try to clean up the type conversions and hope it passes on windows. one interesting thing I learned is that bitshift operations are special: in `x << y`, the result type depends only on the type of `x`, unlike most arithmetic operations where the result type depends on both operands' types. Closes https://github.com/facebook/rocksdb/pull/2277 Differential Revision: D5050145 Pulled By: ajkr fbshipit-source-id: f3309e77526ac9612c632bf93a62d99757af9a29 | 12 May 2017, 16:28:07 UTC |
| c2be434 | Adam Retter | 11 May 2017, 22:17:26 UTC | Build and link with ZStd when creating the static RocksJava build Summary: Closes https://github.com/facebook/rocksdb/pull/2279 Differential Revision: D5048161 Pulled By: yiwu-arbug fbshipit-source-id: 43742ff93137e0a35ea7e855692c9e9a0cd41968 | 11 May 2017, 22:27:10 UTC |
| c61e72c | Yi Wu | 11 May 2017, 16:43:59 UTC | Add missing files of blob_db to CMake file Summary: Some of the file from #2269 didn't add to CMake file. Adding them to fix window build. Closes https://github.com/facebook/rocksdb/pull/2276 Differential Revision: D5043487 Pulled By: yiwu-arbug fbshipit-source-id: 4eba853e9d92574353abce21d77d30e47ce43d3d | 11 May 2017, 16:52:22 UTC |
| 3907c94 | Yi Wu | 11 May 2017, 05:46:24 UTC | Fix ColumnFamilyTest:BulkAddDrop Summary: Fix ColumnFamilyTest:BulkAddDrop not deleted CF handles at the end, causing ASAN failure. Closes https://github.com/facebook/rocksdb/pull/2275 Differential Revision: D5040724 Pulled By: yiwu-arbug fbshipit-source-id: 86cd4070c944d01173a3cc36462bb800698af192 | 11 May 2017, 06:05:44 UTC |
| cda5fde | Andrew Kryczka | 11 May 2017, 01:16:31 UTC | CoreLocalArray class Summary: Moved the logic for core-local array out of ConcurrentArena and into a separate class because I want to reuse it for core-local stats. Closes https://github.com/facebook/rocksdb/pull/2256 Differential Revision: D5011518 Pulled By: ajkr fbshipit-source-id: a75a7b8f7b7a42fd6273489ada405f14c6be196a | 11 May 2017, 01:25:36 UTC |
| 9394966 | Andrew Kryczka | 11 May 2017, 00:52:34 UTC | update TARGETS Summary: address siying's comment in #2272. Closes https://github.com/facebook/rocksdb/pull/2274 Differential Revision: D5039489 Pulled By: ajkr fbshipit-source-id: 3e2d957d3469c13d0e33ededa59320c4c3f24ef6 | 11 May 2017, 00:57:28 UTC |
| 4e83b80 | Matt Kohl | 10 May 2017, 23:47:22 UTC | title: Bulkoading -> title: Bulkloading Summary: Closes https://github.com/facebook/rocksdb/pull/2219 Differential Revision: D4986642 Pulled By: lightmark fbshipit-source-id: c9328991e742768fb5caa0e88e022afb514f0c65 | 10 May 2017, 23:56:45 UTC |
| d85ff49 | Anirban Rahut | 10 May 2017, 21:54:35 UTC | Blob storage pr Summary: The final pull request for Blob Storage. Closes https://github.com/facebook/rocksdb/pull/2269 Differential Revision: D5033189 Pulled By: yiwu-arbug fbshipit-source-id: 6356b683ccd58cbf38a1dc55e2ea400feecd5d06 | 10 May 2017, 22:14:44 UTC |
| 492fc49 | Aaron Gao | 10 May 2017, 18:53:28 UTC | fix readampbitmap tests Summary: fix test failure of ReadAmpBitmap and ReadAmpBitmapLiveInCacheAfterDBClose. test ReadAmpBitmapLiveInCacheAfterDBClose individually and make check Closes https://github.com/facebook/rocksdb/pull/2271 Differential Revision: D5038133 Pulled By: lightmark fbshipit-source-id: 803cd6f45ccfdd14a9d9473c8af311033e164be8 | 10 May 2017, 19:29:23 UTC |
| be421b0 | Andrew Kryczka | 10 May 2017, 18:50:10 UTC | portable sched_getcpu calls Summary: - added a feature test in build_detect_platform to check whether sched_getcpu() is available. glibc offers it only on some platforms (e.g., linux but not mac); this way should be easier than maintaining a list of platforms on which it's available. - refactored PhysicalCoreID() to be simpler / less repetitive. ordered the conditional compilation clauses from most-to-least preferred Closes https://github.com/facebook/rocksdb/pull/2272 Differential Revision: D5038093 Pulled By: ajkr fbshipit-source-id: 81d7db3cc620250de220bdeb3194b2b3d7673de7 | 10 May 2017, 19:29:23 UTC |
| 0f559ab | Lovro Puzar | 10 May 2017, 18:22:44 UTC | Add NO_UPDATE_BUILD_VERSION option to makefile Summary: When building rocksdb in fbcode using `make`, util/build_version.cc is always updated (gitignore/hgignore doesn't apply because the file is already checked into fbcode). To use the rocksdb makefile from our own makefile, I would like an option to prevent the metadata update, which is of no value for us. Closes https://github.com/facebook/rocksdb/pull/2264 Differential Revision: D5037846 Pulled By: siying fbshipit-source-id: 9fa005725c5ecb31d9cbe2e738cbee209591f08a | 10 May 2017, 18:27:40 UTC |
| 3a04a25 | Min Wei | 10 May 2017, 17:19:52 UTC | Flink state Summary: This is to address the issue reported in https://github.com/facebook/rocksdb/issues/1988 The fix is simple. A typo. Closes https://github.com/facebook/rocksdb/pull/2267 Differential Revision: D5037149 Pulled By: siying fbshipit-source-id: 1bb585c7a753ef77c81c4b92deafbed8e21fe8ff | 10 May 2017, 18:12:38 UTC |
| 35df23f | Dmitri Smirnov | 10 May 2017, 17:18:26 UTC | Fix suite exclisions Summary: Closes https://github.com/facebook/rocksdb/pull/2250 Differential Revision: D5037141 Pulled By: siying fbshipit-source-id: 53a0ab5553422839bb2df6a7badde54810774a3e | 10 May 2017, 18:12:38 UTC |
| e7cea86 | Adam Retter | 10 May 2017, 15:54:46 UTC | Fixes the CentOS 5 cross-building of RocksJava Summary: Updates to CentOS 5 have been archived as CentOS 5 is EOL. We now pull the updates from the vault. This is a stop gap solution, I will send a PR in a couple days which uses fixed Docker containers (with the updates pre-installed) instead. sagar0 Here you go :-) Closes https://github.com/facebook/rocksdb/pull/2270 Differential Revision: D5033637 Pulled By: sagar0 fbshipit-source-id: a9312dd1bc18bfb8653f06ffa0a1512b4415720d | 10 May 2017, 15:57:54 UTC |
| 259a00e | Aaron Gao | 10 May 2017, 08:32:52 UTC | unbiase readamp bitmap Summary: Consider BlockReadAmpBitmap with bytes_per_bit = 32. Suppose bytes [a, b) were used, while bytes [a-32, a) and [b+1, b+33) weren't used; more formally, the union of ranges passed to BlockReadAmpBitmap::Mark() contains [a, b) and doesn't intersect with [a-32, a) and [b+1, b+33). Then bits [floor(a/32), ceil(b/32)] will be set, and so the number of useful bytes will be estimated as (ceil(b/32) - floor(a/32)) * 32, which is on average equal to b-a+31. An extreme example: if we use 1 byte from each block, it'll be counted as 32 bytes from each block. It's easy to remove this bias by slightly changing the semantics of the bitmap. Currently each bit represents a byte range [i*32, (i+1)*32). This diff makes each bit represent a single byte: i*32 + X, where X is a random number in [0, 31] generated when bitmap is created. So, e.g., if you read a single byte at random, with probability 31/32 it won't be counted at all, and with probability 1/32 it will be counted as 32 bytes; so, on average it's counted as 1 byte. *But there is one exception: the last bit will always set with the old way.* (*) - assuming read_amp_bytes_per_bit = 32. Closes https://github.com/facebook/rocksdb/pull/2259 Differential Revision: D5035652 Pulled By: lightmark fbshipit-source-id: bd98b1b9b49fbe61f9e3781d07f624e3cbd92356 | 10 May 2017, 08:49:52 UTC |
| a620966 | Jos Collin | 10 May 2017, 01:43:23 UTC | port: updated PhysicalCoreID() Summary: Updated PhysicalCoreID() to use sched_getcpu() on x86_64 for glibc >= 2.22. Added a new function named GetCPUID() that calls sched_getcpu(), to avoid repeated code. This change is done as per the comments of PR: https://github.com/facebook/rocksdb/pull/2230 Signed-off-by: Jos Collin <jcollin@redhat.com> Closes https://github.com/facebook/rocksdb/pull/2260 Differential Revision: D5025734 Pulled By: ajkr fbshipit-source-id: f4cca68c12573cafcf8531e7411a1e733bbf8eef | 10 May 2017, 02:06:39 UTC |
| df035b6 | Aaron Gao | 09 May 2017, 22:44:48 UTC | Print compaction_options_universal.stop_style in LOG file Summary: Print compaction_options_universal.stop_style in LOG file ./db_bench --benchmarks=fillseq and read the log Closes https://github.com/facebook/rocksdb/pull/2268 Differential Revision: D5032438 Pulled By: lightmark fbshipit-source-id: 0e72fcd96a1caaf3cab20e86d39c75fbebf5ce37 | 09 May 2017, 22:57:44 UTC |
| 4897eb2 | Islam AbdelRahman | 09 May 2017, 16:49:41 UTC | dont skip IO for filter blocks Summary: Based on my experience with linkbench, We should not skip loading bloom filter blocks when they are not available in block cache when using Iterator::Seek Actually I am not sure why this behavior existed in the first place Closes https://github.com/facebook/rocksdb/pull/2255 Differential Revision: D5010721 Pulled By: maysamyabandeh fbshipit-source-id: 0af545a06ac4baeecb248706ec34d009c2480ca4 | 09 May 2017, 16:52:02 UTC |
| 3f73d54 | Changjian Gao | 09 May 2017, 05:09:06 UTC | Add C API to set max_file_opening_threads option Summary: Add `rocksdb_options_set_max_file_opening_threads()` API Closes https://github.com/facebook/rocksdb/pull/2184 Differential Revision: D4923090 Pulled By: lightmark fbshipit-source-id: c4ddce17733d999d426d02f7202b33a46ed6faed | 09 May 2017, 05:49:32 UTC |
| 0b69e50 | Gunnar Kudrjavets | 08 May 2017, 23:04:19 UTC | Define CACHE_LINE_SIZE only when it's not defined Summary: RocksDB is compiled as part of MyRocks (MySQL storage engine) build. MySQL already defines `CACHE_LINE_SIZE` and therefore we're getting a conflict. Change RocksDB definition to be more cognizant of this. Closes https://github.com/facebook/rocksdb/pull/2257 Differential Revision: D5013188 Pulled By: gunnarku fbshipit-source-id: cfa76fe99f90dcd82aa09204e2f1f35e07a82b41 | 08 May 2017, 23:12:28 UTC |
| 2cd0077 | Yi Wu | 08 May 2017, 05:12:55 UTC | Add bulk create/drop column family API Summary: Adding DB::CreateColumnFamilie() and DB::DropColumnFamilies() to bulk create/drop column families. This is to address the problem creating/dropping 1k column families takes minutes. The bottleneck is we persist options files for every single column family create/drop, and it parses the persisted options file for verification, which take a lot CPU time. The new APIs simply create/drop column families individually, and persist options file once at the end. This improves create 1k column families to within ~0.1s. Further improvement can be merge manifest write to one IO. Closes https://github.com/facebook/rocksdb/pull/2248 Differential Revision: D5001578 Pulled By: yiwu-arbug fbshipit-source-id: d4e00bda671451e0b314c13e12ad194b1704aa03 | 08 May 2017, 06:20:46 UTC |
| 40af238 | Maysam Yabandeh | 06 May 2017, 03:10:56 UTC | Object lifetime in cache Summary: Any non-raw-data dependent object must be destructed before the table closes. There was a bug of not doing that for filter object. This patch fixes the bug and adds a unit test to prevent such bugs in future. Closes https://github.com/facebook/rocksdb/pull/2246 Differential Revision: D5001318 Pulled By: maysamyabandeh fbshipit-source-id: 6d8772e58765485868094b92964da82ef9730b6d | 06 May 2017, 06:20:01 UTC |
| fdaefa0 | Tamir Duberstein | 05 May 2017, 23:09:24 UTC | travis: add Windows cross-compilation Summary: - downcase includes for case-sensitive filesystems - give targets the same name (librocksdb) on all platforms With this patch it is possible to cross-compile RocksDB for Windows from a Linux host using mingw. cc yuslepukhin orgads Closes https://github.com/facebook/rocksdb/pull/2107 Differential Revision: D4849784 Pulled By: siying fbshipit-source-id: ad26ed6b4d393851aa6551e6aa4201faba82ef60 | 06 May 2017, 06:20:01 UTC |
| a30a696 | Aaron Gao | 05 May 2017, 22:01:04 UTC | do not read next datablock if upperbound is reached Summary: Now if we have iterate_upper_bound set, we continue read until get a key >= upper_bound. For a lot of cases that neighboring data blocks have a user key gap between them, our index key will be a user key in the middle to get a shorter size. For example, if we have blocks: [a b c d][f g h] Then the index key for the first block will be 'e'. then if upper bound is any key between 'd' and 'e', for example, d1, d2, ..., d99999999999, we don't have to read the second block and also know that we have done our iteration by reaching the last key that smaller the upper bound already. This diff can reduce RA in most cases. Closes https://github.com/facebook/rocksdb/pull/2239 Differential Revision: D4990693 Pulled By: lightmark fbshipit-source-id: ab30ea2e3c6edf3fddd5efed3c34fcf7739827ff | 06 May 2017, 06:20:01 UTC |
| 2d42cf5 | Aaron Gao | 05 May 2017, 18:58:10 UTC | Roundup read bytes in ReadaheadRandomAccessFile Summary: Fix alignment in ReadaheadRandomAccessFile Closes https://github.com/facebook/rocksdb/pull/2253 Differential Revision: D5012336 Pulled By: lightmark fbshipit-source-id: 10d2c829520cb787227ef653ef63d5d701725778 | 05 May 2017, 19:14:14 UTC |
| 264d3f5 | Siying Dong | 05 May 2017, 01:14:29 UTC | Allow IntraL0 compaction in FIFO Compaction Summary: Allow an option for users to do some compaction in FIFO compaction, to pay some write amplification for fewer number of files. Closes https://github.com/facebook/rocksdb/pull/2163 Differential Revision: D4895953 Pulled By: siying fbshipit-source-id: a1ab608dd0627211f3e1f588a2e97159646e1231 | 05 May 2017, 01:16:13 UTC |
| 8c3a180 | Andrew Kryczka | 05 May 2017, 01:03:22 UTC | Set lower-bound on dynamic level sizes Summary: Changed dynamic leveling to stop setting the base level's size bound below `max_bytes_for_level_base`. Behavior for config where `max_bytes_for_level_base == level0_file_num_compaction_trigger * write_buffer_size` and same amount of data in L0 and base-level: - Before #2027, compaction scoring would favor base-level due to dividing by size smaller than `max_bytes_for_level_base`. - After #2027, L0 and Lbase get equal scores. The disadvantage is L0 is often compacted before reaching the num files trigger since `write_buffer_size` can be bigger than the dynamically chosen base-level size. This increases write-amp. - After this diff, L0 and Lbase still get equal scores. Now it takes `level0_file_num_compaction_trigger` files of size `write_buffer_size` to trigger L0 compaction by size, fixing the write-amp problem above. Closes https://github.com/facebook/rocksdb/pull/2123 Differential Revision: D4861570 Pulled By: ajkr fbshipit-source-id: 467ddef56ed1f647c14d86bb018bcb044c39b964 | 05 May 2017, 01:16:12 UTC |
| 7c1c8ce | Andrew Kryczka | 05 May 2017, 00:40:29 UTC | Avoid calling fallocate with UINT64_MAX Summary: When user doesn't set a limit on compaction output file size, let's use the sum of the input files' sizes. This will avoid passing UINT64_MAX as fallocate()'s length. Reported in #2249. Test setup: - command: `TEST_TMPDIR=/data/rocksdb-test/ strace -e fallocate ./db_compaction_test --gtest_filter=DBCompactionTest.ManualCompactionUnknownOutputSize` - filesystem: xfs before this diff: `fallocate(10, 01, 0, 1844674407370955160) = -1 ENOSPC (No space left on device)` after this diff: `fallocate(10, 01, 0, 1977) = 0` Closes https://github.com/facebook/rocksdb/pull/2252 Differential Revision: D5007275 Pulled By: ajkr fbshipit-source-id: 4491404a6ae8a41328aede2e2d6f4d9ac3e38880 | 05 May 2017, 00:43:22 UTC |
| a45e98a | Leonidas Galanis | 04 May 2017, 17:28:22 UTC | max_open_files dynamic set, follow up Summary: Followup to make 0x40000 a TableCache constant that indicates infinite capacity Closes https://github.com/facebook/rocksdb/pull/2247 Differential Revision: D5001349 Pulled By: lgalanis fbshipit-source-id: ce7bd2e54b0975bb9f8680fdaa0f8bb0e7ae81a2 | 04 May 2017, 17:42:45 UTC |
| 6b99dbe | Aaron Gao | 04 May 2017, 08:21:31 UTC | fix memory alignment with logical sector size Summary: we align the buffer with logical sector size and should not test it with page size, which is usually 4k. Closes https://github.com/facebook/rocksdb/pull/2245 Differential Revision: D5001842 Pulled By: lightmark fbshipit-source-id: a7135fcf6351c6db363e8908956b1e193a4a6291 | 04 May 2017, 08:30:13 UTC |
| e7ae4a3 | Leonidas Galanis | 04 May 2017, 03:46:17 UTC | Max open files mutable Summary: Makes max_open_files db option dynamically set-able by SetDBOptions. During the call of SetDBOptions we call SetCapacity on the table cache, which is a LRUCache. Closes https://github.com/facebook/rocksdb/pull/2185 Differential Revision: D4979189 Pulled By: yiwu-arbug fbshipit-source-id: ca7e8dc5e3619c79434f579be4847c0f7e56afda | 04 May 2017, 04:13:14 UTC |
| 60847a3 | Jos Collin | 03 May 2017, 19:48:23 UTC | port: updated PhysicalCoreID() Summary: Checked the return value of __get_cpuid(). Implemented the else case where the arch is different from i386 and x86_64. Pulled By: ajkr Differential Revision: D4973496 fbshipit-source-id: c40fdef5840364c2a79b1d11df0db5d4ec3d6a4a | 03 May 2017, 20:08:55 UTC |
| b551104 | siddontang | 03 May 2017, 17:54:07 UTC | support PopSavePoint for WriteBatch Summary: Try to fix https://github.com/facebook/rocksdb/issues/1969 Closes https://github.com/facebook/rocksdb/pull/2170 Differential Revision: D4907333 Pulled By: yiwu-arbug fbshipit-source-id: 417b420ff668e6c2fd0dad42a94c57385012edc5 | 03 May 2017, 17:57:45 UTC |
| 498693c | Adam Retter | 03 May 2017, 03:33:25 UTC | Remove orphaned Java classes Summary: Remove orphaned Java classes that were missed from commit 9385fd7 Closes https://github.com/facebook/rocksdb/issues/2210 Closes https://github.com/facebook/rocksdb/pull/2212 Differential Revision: D4986199 Pulled By: maysamyabandeh fbshipit-source-id: 74e866cd8661c172a4754c2c945e30d650f6fa05 | 03 May 2017, 04:13:49 UTC |
| 5e2ebf2 | Tamir Duberstein | 02 May 2017, 22:09:40 UTC | travis: add CMake compilation Summary: siying Closes https://github.com/facebook/rocksdb/pull/2233 Differential Revision: D4987113 Pulled By: siying fbshipit-source-id: f07ecd3f7d9f5366a2b1665ce28ba10d74405557 | 02 May 2017, 22:11:59 UTC |
| af6fe69 | Siying Dong | 02 May 2017, 22:01:07 UTC | Fix an issue of manual / auto compaction data race Summary: A data race between a manual and an auto compaction can cause a scheduled automatic compaction to be cancelled and never rescheduled again. This may cause a condition of hanging forever. Fix this by always making sure the cancelled compaction is put back to the compaction queue. Closes https://github.com/facebook/rocksdb/pull/2238 Differential Revision: D4984591 Pulled By: siying fbshipit-source-id: 3ab153886403c7b991896dcb2158b96cac12f227 | 02 May 2017, 22:11:59 UTC |
| 6798d1f | Maysam Yabandeh | 02 May 2017, 20:39:09 UTC | Revert "Delete filter before closing the table" Summary: This reverts commit 89833577a80ad7a2cbf6b99c5957f572b3548152. Closes https://github.com/facebook/rocksdb/pull/2240 Differential Revision: D4986982 Pulled By: maysamyabandeh fbshipit-source-id: 56c4c07b7b5b7c6fe122d5c2f2199d221c8510c0 | 02 May 2017, 20:46:39 UTC |
| 8983357 | Maysam Yabandeh | 02 May 2017, 01:17:01 UTC | Delete filter before closing the table Summary: Some filters such as partitioned filter have pointers to the table for which they are created. Therefore is they are stored in the block cache, the should be forcibly erased from block cache before closing the table, which would result into deleting the object. Otherwise the destructor will be called later when the cache is lazily erasing the object, which having the parent table no longer existent it could result into undefined behavior. Update: there will be still cases the filter is not removed from the cache since the table has not kept a pointer to the cache handle to be able to forcibly release it later. We make sure that the filter destructor does not access the table pointer to get around such cases. Closes https://github.com/facebook/rocksdb/pull/2207 Differential Revision: D4941591 Pulled By: maysamyabandeh fbshipit-source-id: 56fbab2a11cf447e1aa67caa30b58d7bd7ce5bbd | 02 May 2017, 02:19:37 UTC |
| 47a09b0 | Maysam Yabandeh | 02 May 2017, 00:43:52 UTC | Avoid pinning when row cache is accessed Summary: With row cache being enabled, table cache is doing a short circuit for reading data. This path needs to be updated to take advantage of pinnable slice. In the meanwhile we disabling pinning in this path. Closes https://github.com/facebook/rocksdb/pull/2237 Differential Revision: D4982389 Pulled By: maysamyabandeh fbshipit-source-id: 542630d0cf23cfb1f0c397da82e7053df7966591 | 02 May 2017, 01:12:46 UTC |
| aeaba07 | Siying Dong | 01 May 2017, 23:05:21 UTC | Remove an assert that causes TSAN failure. Summary: ColumnFamilyData::ConstructNewMemtable is called out of DB mutex, and it asserts current_ is not empty, but current_ should only be accessed inside DB mutex. Remove this assert to make TSAN happy. Closes https://github.com/facebook/rocksdb/pull/2235 Differential Revision: D4978531 Pulled By: siying fbshipit-source-id: 423685a7dae88ed3faaa9e1b9ccb3427ac704a4b | 01 May 2017, 23:35:15 UTC |
| 0b90aa9 | Siying Dong | 01 May 2017, 23:01:16 UTC | Set VALGRIND_VER Summary: VALGRIND_VER was left empty after moving the environment to GCC-5. Set it back. Closes https://github.com/facebook/rocksdb/pull/2234 Differential Revision: D4978534 Pulled By: siying fbshipit-source-id: f0640d58e8f575f75fb3f8b92e686c9e0b6a59bb | 01 May 2017, 23:35:15 UTC |
| a2b0521 | Siying Dong | 29 April 2017, 00:00:24 UTC | Make PosixLogger::flush_pending_ atomic Summary: TSAN sometimes complaints data race of PosixLogger::flush_pending_. Make it atomic. Closes https://github.com/facebook/rocksdb/pull/2231 Differential Revision: D4973397 Pulled By: siying fbshipit-source-id: 571e886e3eca3231705919d573e250c1c1ec3764 | 29 April 2017, 00:07:56 UTC |
| da4b207 | Yi Wu | 28 April 2017, 19:33:30 UTC | Fix WriteBatchWithIndex address use after scope error Summary: Fix use after scope error caught by ASAN. Closes https://github.com/facebook/rocksdb/pull/2228 Differential Revision: D4968028 Pulled By: yiwu-arbug fbshipit-source-id: a2a266c98634237494ab4fb2d666bc938127aeb2 | 28 April 2017, 20:12:10 UTC |
| d616ebe | Siying Dong | 28 April 2017, 00:50:56 UTC | Add GPLv2 as an alternative license. Summary: Closes https://github.com/facebook/rocksdb/pull/2226 Differential Revision: D4967547 Pulled By: siying fbshipit-source-id: dd3b58ae1e7a106ab6bb6f37ab5c88575b125ab4 | 28 April 2017, 01:06:12 UTC |
| 4439b45 | Joel Marcey | 28 April 2017, 00:36:35 UTC | Add documentation license Summary: Closes https://github.com/facebook/rocksdb/pull/2224 Differential Revision: D4966524 Pulled By: siying fbshipit-source-id: 99f28471bcfc83b0a9df871d5247f62d4a6540ce | 28 April 2017, 01:06:11 UTC |
| 0ca3ead | Aaron Gao | 27 April 2017, 21:15:30 UTC | add GetRootDB() in DeleteFilesInRange Summary: In case users cast a subclass of db* into dbimpl* Closes https://github.com/facebook/rocksdb/pull/2222 Differential Revision: D4964486 Pulled By: lightmark fbshipit-source-id: 0ccdc08ee8e7a193dfbbe0218c3cbfd795662ca1 | 27 April 2017, 21:33:17 UTC |
| cdad04b | Dmitri Smirnov | 27 April 2017, 19:19:55 UTC | Remove double buffering on RandomRead on Windows. Summary: Remove double buffering on RandomRead on Windows. With more logic appear in file reader/write Read no longer obeys forwarding calls to Windows implementation. Previously direct_io (unbuffered) was only available on Windows but now is supported as generic. We remove intermediate buffering on Windows. Remove random_access_max_buffer_size option which was windows specific. Non-zero values for that opton introduced unnecessary lock contention. Remove Env::EnableReadAhead(), Env::ShouldForwardRawRequest() that are no longer necessary. Add aligned buffer reads for cases when requested reads exceed read ahead size. Closes https://github.com/facebook/rocksdb/pull/2105 Differential Revision: D4847770 Pulled By: siying fbshipit-source-id: 8ab48f8e854ab498a4fd398a6934859792a2788f | 27 April 2017, 19:30:05 UTC |
| e15382c | Siying Dong | 27 April 2017, 00:12:28 UTC | Disable two flaky tests Summary: Closes https://github.com/facebook/rocksdb/pull/2217 Differential Revision: D4959351 Pulled By: siying fbshipit-source-id: ce7c3a430bae0d15e06b3d5c958ebce969d08564 | 27 April 2017, 00:13:46 UTC |
| 2150cc1 | Aaron Gao | 26 April 2017, 22:28:50 UTC | fix WritableFile buffer size in direct IO Summary: �fix the buffer size in case of ppl use buffer size as their block_size. Closes https://github.com/facebook/rocksdb/pull/2198 Differential Revision: D4956878 Pulled By: lightmark fbshipit-source-id: 8bb0dc9c133887aadcd625d5261a3d1110b71473 | 26 April 2017, 22:57:02 UTC |
| efc361e | Andrew Kryczka | 26 April 2017, 22:19:50 UTC | Add user stats Reset API Summary: It resets all the ticker and histogram stats to zero. Needed to change the locking a bit since Reset() is the only operation that manipulates multiple tickers/histograms together, and that operation should be seen as atomic by other operations that access tickers/histograms. Closes https://github.com/facebook/rocksdb/pull/2213 Differential Revision: D4952232 Pulled By: ajkr fbshipit-source-id: c0475c3e4c7b940120d53891b69c3091149a0679 | 26 April 2017, 22:57:01 UTC |
| 6616e4d | Aaron Gao | 26 April 2017, 21:21:04 UTC | add prefetch to PosixRandomAccessFile in buffered io Summary: Every time after a compaction/flush finish, we issue user reads to put the table into block cache which includes a couple of IO that read footer, index blocks, meta block, etc. So we implement Prefetch here to reduce IO. Closes https://github.com/facebook/rocksdb/pull/2196 Differential Revision: D4931782 Pulled By: lightmark fbshipit-source-id: 5a13d58dcab209964352322217193bbf7ff78149 | 26 April 2017, 21:47:23 UTC |
| f6a27d0 | Andrew Kryczka | 26 April 2017, 21:18:58 UTC | Extract statistics tests into separate file Summary: I'm going to add more DB tests for statistics as currently we have very few. I started a file dedicated to this purpose and moved the existing stats-specific tests there. Closes https://github.com/facebook/rocksdb/pull/2211 Differential Revision: D4951558 Pulled By: ajkr fbshipit-source-id: 05d11c35079c40ecabdfd2cf5556ccb761f694a4 | 26 April 2017, 21:47:23 UTC |
| 7eddecc | Aaron Gao | 26 April 2017, 20:28:39 UTC | support bulk loading with universal compaction Summary: Support buck load with universal compaction. More test cases to be added. Closes https://github.com/facebook/rocksdb/pull/2202 Differential Revision: D4935360 Pulled By: lightmark fbshipit-source-id: cc3ca1b6f42faa503207dab1408d6bcf393ee5b5 | 26 April 2017, 20:41:32 UTC |
| 3b4d1b7 | Aaron Gao | 26 April 2017, 08:09:05 UTC | add <sys/sysmacros.h> to avoid warning with glibc 2.25 Summary: https://github.com/facebook/rocksdb/issues/2152 Closes https://github.com/facebook/rocksdb/pull/2208 Differential Revision: D4945577 Pulled By: lightmark fbshipit-source-id: 4e679150f2c9443d3be0b6008b26b65fabbda75a | 26 April 2017, 08:26:55 UTC |
| e5e545a | Andrew Kryczka | 24 April 2017, 21:57:27 UTC | Reunite checkpoint and backup core logic Summary: These code paths forked when checkpoint was introduced by copy/pasting the core backup logic. Over time they diverged and bug fixes were sometimes applied to one but not the other (like fix to include all relevant WALs for 2PC), or it required extra effort to fix both (like fix to forge CURRENT file). This diff reunites the code paths by extracting the core logic into a function, CreateCustomCheckpoint(), that is customizable via callbacks to implement both checkpoint and backup. Related changes: - flush_before_backup is now forcibly enabled when 2PC is enabled - Extracted CheckpointImpl class definition into a header file. This is so the function, CreateCustomCheckpoint(), can be called by internal rocksdb code but not exposed to users. - Implemented more functions in DummyDB/DummyLogFile (in backupable_db_test.cc) that are used by CreateCustomCheckpoint(). Closes https://github.com/facebook/rocksdb/pull/1932 Differential Revision: D4622986 Pulled By: ajkr fbshipit-source-id: 157723884236ee3999a682673b64f7457a7a0d87 | 24 April 2017, 22:06:46 UTC |
| 72c21fb | Aaron Gao | 24 April 2017, 20:38:29 UTC | call GetRootDB() before cast to DBImpl* in CancelAllBackgroundWork Summary: User could call this with wrapper class of DB or DBImpl Closes https://github.com/facebook/rocksdb/pull/2200 Differential Revision: D4935530 Pulled By: lightmark fbshipit-source-id: df9cb61d67d0f3bbcf62f714d77523a459a92883 | 24 April 2017, 20:47:17 UTC |
| 4c9447d | Maysam Yabandeh | 24 April 2017, 18:21:47 UTC | Add erase option to release cache Summary: This is useful when we put the entries in the block cache for accounting purposes and do not expect it to be used after it is released. If the cache does not erase the item in such cases not only the performance of cache is negatively affected but the item's destructor not being called at the time of release might violate the assumptions about the lifetime of the object. The new change adds a force_erase option to the Release method and returns a boolean to indicate whehter the item is successfully deleted. Closes https://github.com/facebook/rocksdb/pull/2180 Differential Revision: D4916032 Pulled By: maysamyabandeh fbshipit-source-id: 94409a346069923cac9de8e57adc313b4ed46f28 | 24 April 2017, 18:28:36 UTC |
| 04d5897 | Tomas Kolda | 22 April 2017, 03:41:37 UTC | AIX and Solaris Sparc Support Summary: Replacement of #2147 The change was squashed due to a lot of conflicts. Closes https://github.com/facebook/rocksdb/pull/2194 Differential Revision: D4929799 Pulled By: siying fbshipit-source-id: 5cd49c254737a1d5ac13f3c035f128e86524c581 | 22 April 2017, 03:48:04 UTC |
| afff995 | Maysam Yabandeh | 22 April 2017, 00:30:30 UTC | Respect deprecated flag in table options Summary: Closes https://github.com/facebook/rocksdb/pull/2197 Differential Revision: D4932434 Pulled By: maysamyabandeh fbshipit-source-id: 6c83c12d6d47e3f0640ab84954944215968f266f | 22 April 2017, 00:42:10 UTC |
| 066cfba | Nick Anderson | 21 April 2017, 19:42:46 UTC | Adding -noprofile to CMakeLists for Windows Summary: In the off chance you have a `Microsoft.PowerShell_profile.ps1` profile the `execute_process` will load one's profile during running `cmake`, which will cause garbage output to be written to the `build_version.cc` file. If you add in a `-noprofile` flag, or you suppress output in your pshell profile, this is mitigated. Closes https://github.com/facebook/rocksdb/pull/2168 Differential Revision: D4927003 Pulled By: siying fbshipit-source-id: 83861752d6cf2627dd864eedd2acaa8aa8a6232e | 21 April 2017, 21:11:49 UTC |
| cb885bc | Aaron Gao | 21 April 2017, 02:44:55 UTC | set compaction_iterator earliest_snapshot to max if no snapshot Summary: It is a potential bug that will be triggered if we ingest files before inserting the first key into an empty db. 0 is a special value reserved to indicate the concept of non-existence. But not good for seqno in this case because 0 is a valid seqno for ingestion(bulk loading) Closes https://github.com/facebook/rocksdb/pull/2183 Differential Revision: D4919827 Pulled By: lightmark fbshipit-source-id: 237eea40f88bd6487b66806109d90065dc02c362 | 21 April 2017, 02:56:59 UTC |
| 7534ba7 | Siying Dong | 20 April 2017, 22:56:50 UTC | StackableDB should pass ResetStats() Summary: Closes https://github.com/facebook/rocksdb/pull/2190 Differential Revision: D4922688 Pulled By: siying fbshipit-source-id: eaa3d122f8d389ae0508ec8b61f7780fd8b0a7ef | 20 April 2017, 23:11:56 UTC |
| c1fbf91 | Tomas Kolda | 20 April 2017, 17:59:04 UTC | Fixing Solaris Sparc crash due to cached TLS Summary: Workaround for Solaris gcc binary. Program is crashing, because when TLS of perf context that is used twice on same frame, it is damaged thus Segmentation fault. Issue: #2153 Closes https://github.com/facebook/rocksdb/pull/2187 Differential Revision: D4922274 Pulled By: siying fbshipit-source-id: 549105ebce9a8ce08a737f4d6b9f2312ebcde9a8 | 20 April 2017, 18:06:09 UTC |
| 963eeba | Siying Dong | 20 April 2017, 17:34:01 UTC | Revert how check_format_compatible.sh checkout release branches. Summary: In a previous commit, I changed the way to checkout release branches from "git checkout <branch_name>" to "git checkout origin/<branch_name>". However, this doesn't seem to work in our CI environment. Revert it. Closes https://github.com/facebook/rocksdb/pull/2189 Differential Revision: D4922294 Pulled By: siying fbshipit-source-id: 482c17f9b05e6ccb190876b050682fe5a458103d | 20 April 2017, 18:06:09 UTC |
| 97005db | Siying Dong | 20 April 2017, 17:16:13 UTC | tools/check_format_compatible.sh to cover option file loading too Summary: tools/check_format_compatible.sh will check a newer version of RocksDB can open option files generated by older version releases. In order to achieve that, a new parameter "--try_load_options" is added to ldb. With this parameter set, if option file exists, we load the option file and use it to open the DB. With this opiton set, we can validate option loading logic. Closes https://github.com/facebook/rocksdb/pull/2178 Differential Revision: D4914989 Pulled By: siying fbshipit-source-id: db114f7724fcb41e5e9483116d84d7c4b8389ca4 | 20 April 2017, 17:26:37 UTC |
| 8f61967 | Maysam Yabandeh | 20 April 2017, 15:34:42 UTC | Add cpu usage to regression benchmarks (4th attempt) Summary: Tested by running it on a remote machine. I could not run it on the particular remote machine which has a different location for time command since it is busy and the script does not allow concurrent runs. So I tested it by hacking the script and replacing the command with "\$(hostname)" and confirmed that the scripts prints out the host name of the remote machine. Closes https://github.com/facebook/rocksdb/pull/2181 Differential Revision: D4921654 Pulled By: maysamyabandeh fbshipit-source-id: 8abb5ea9f7234f3c50a749576ccbb47ff605beb9 | 20 April 2017, 16:31:09 UTC |
| df74b77 | Andrew Kryczka | 19 April 2017, 20:15:16 UTC | Limit backups opened Summary: This was requested by a customer who wants to proactively monitor whether any valid backups are available. The existing performance was poor because Open() serially reads every small meta-file (one per backup), which was slow on HDFS. Now we only read the minimum number of meta-files to find `max_valid_backups_to_open` valid backups. The customer mentioned above can just set it to one. Closes https://github.com/facebook/rocksdb/pull/2151 Differential Revision: D4882564 Pulled By: ajkr fbshipit-source-id: cb0edf9e8ac693e4d5f24902e725a011ed8c0c2f | 19 April 2017, 20:26:47 UTC |
| 1dd7760 | Andrew Kryczka | 19 April 2017, 18:44:40 UTC | Change L0 compaction score using level size Summary: The goal is to avoid the problem of small number of L0 files triggering compaction to base level (which increased write-amp), while still allowing L0 compaction-by-size (so intra-L0 compactions cause score to increase). Closes https://github.com/facebook/rocksdb/pull/2172 Differential Revision: D4908552 Pulled By: ajkr fbshipit-source-id: 4b170142b2b368e24bd7948b2a6f24c69fabf73d | 19 April 2017, 19:00:01 UTC |
| 927bbab | Maysam Yabandeh | 19 April 2017, 18:02:09 UTC | Revert "Add cpu usage to regression benchmarks (3rd attempt)" Summary: This reverts commit 476e80be80903158ac0af486aad630efff70b036. Closes https://github.com/facebook/rocksdb/pull/2177 Differential Revision: D4914830 Pulled By: maysamyabandeh fbshipit-source-id: 039299348ceb325aa721eb35e3a26e890f84ee74 | 19 April 2017, 18:11:12 UTC |
| 8e84a38 | Maysam Yabandeh | 19 April 2017, 03:19:34 UTC | Re-add index_per_partition but as deprecated Summary: index_per_partition should have deprecated deprecated instead of being removed. It is causing backward compatibility issues. Closes https://github.com/facebook/rocksdb/pull/2173 Differential Revision: D4910947 Pulled By: maysamyabandeh fbshipit-source-id: 5c52939381847d232ede6866606f67f2b4b857ae | 19 April 2017, 03:35:46 UTC |
| 1553659 | Siying Dong | 19 April 2017, 01:31:15 UTC | Add more recent versions to tools/check_format_compatible.sh Summary: Need to add more recent versions to tools/check_format_compatible.sh to meka sure backward and forward compatibility. Closes https://github.com/facebook/rocksdb/pull/2175 Differential Revision: D4911585 Pulled By: siying fbshipit-source-id: 943e6488757efb11bb6720d811c7ba949915c9de | 19 April 2017, 01:57:11 UTC |
| 966ebb0 | Yi Wu | 19 April 2017, 00:12:50 UTC | Hide event listeners from lite build Summary: Fixing lite build failure introduce by #2169. Closes https://github.com/facebook/rocksdb/pull/2174 Reviewed By: sagar0 Differential Revision: D4910619 Pulled By: yiwu-arbug fbshipit-source-id: 5213b7b7431cc258688793c8c28153025588d8d9 | 19 April 2017, 00:26:19 UTC |
| 476e80b | Maysam Yabandeh | 19 April 2017, 00:09:47 UTC | Add cpu usage to regression benchmarks (3rd attempt) Summary: Tested by running rocks/tools/debug_regression_test.sh and verifying the local output: ``` cat local/rocks_regression_tests/OPTIONS-viewstate-66-1262-5000/2017-04-13-11-34-51/SUMMARY.csv commit id, benchmark, user@host,num-dbs,key-range,key-size,value-size,compress-rate,ops-per-thread,num-threads, cache-size,flushes,compactions,ops-per-s, p50, p75, p99, p99.9, p99.99,debug,real-sec,user-sec,sys-sec d2dce5611ab4b9a671ef518f648adf84d49e5356, readrandom, root@localhost, 12, 5000, 66, 1262, 50, 312, 16, 1073741824, 4, 16, 138458, 9380, 11530, 55200, 16803200, 32504000, 1, 0, 0, 0 d2dce5611ab4b9a671ef518f648adf84d49e5356, readwhilewriting, root@localhost, 12, 5000, 66, 1262, 50, 312, 16, 1073741824, 4, 16, 104511, Closes https://github.com/facebook/rocksdb/pull/2157 Differential Revision: D4909238 Pulled By: maysamyabandeh fbshipit-source-id: dc7bb8569c3c33b9f7c4ba47a757b24d27bb3b31 | 19 April 2017, 00:11:25 UTC |
| c49d704 | Siying Dong | 18 April 2017, 23:30:51 UTC | Add DB:ResetStats() Summary: Add a function to allow users to reset internal stats without restarting the DB. Closes https://github.com/facebook/rocksdb/pull/2167 Differential Revision: D4907939 Pulled By: siying fbshipit-source-id: ab2dd85b88aabe9380da7485320a1d460d3e1f68 | 18 April 2017, 23:56:48 UTC |
