Revision 7242dae7fe5818eed1cc655c4444d54173c5ca8c authored by Adam Retter on 03 February 2020, 20:28:25 UTC, committed by Facebook Github Bot on 03 February 2020, 20:30:13 UTC
Summary: This is a redesign of the API for RocksJava comparators with the aim of improving performance. It also simplifies the class hierarchy. **NOTE**: This breaks backwards compatibility for existing 3rd party Comparators implemented in Java... so we need to consider carefully which release branches this goes into. Previously when implementing a comparator in Java the developer had a choice of subclassing either `DirectComparator` or `Comparator` which would use direct and non-direct byte-buffers resepectively (via `DirectSlice` and `Slice`). In this redesign there we have eliminated the overhead of using the Java Slice classes, and just use `ByteBuffer`s. The `ComparatorOptions` supplied when constructing a Comparator allow you to choose between direct and non-direct byte buffers by setting `useDirect`. In addition, the `ComparatorOptions` now allow you to choose whether a ByteBuffer is reused over multiple comparator calls, by setting `maxReusedBufferSize > 0`. When buffers are reused, ComparatorOptions provides a choice of mutex type by setting `useAdaptiveMutex`. --- [JMH benchmarks previously indicated](https://github.com/facebook/rocksdb/pull/6241#issue-356398306) that the difference between C++ and Java for implementing a comparator was ~7x slowdown in Java. With these changes, when reusing buffers and guarding access to them via mutexes the slowdown is approximately the same. However, these changes offer a new facility to not reuse mutextes, which reduces the slowdown to ~5.5x in Java. We also offer a `thread_local` mechanism for reusing buffers, which reduces slowdown to ~5.2x in Java (closes https://github.com/facebook/rocksdb/pull/4425). These changes also form a good base for further optimisation work such as further JNI lookup caching, and JNI critical. --- These numbers were captured without jemalloc. With jemalloc, the performance improves for all tests, and the Java slowdown reduces to between 4.8x and 5.x. ``` ComparatorBenchmarks.put native_bytewise thrpt 25 124483.795 ± 2032.443 ops/s ComparatorBenchmarks.put native_reverse_bytewise thrpt 25 114414.536 ± 3486.156 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_adaptive-mutex thrpt 25 17228.250 ± 1288.546 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_non-adaptive-mutex thrpt 25 16035.865 ± 1248.099 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_thread-local thrpt 25 21571.500 ± 871.521 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_adaptive-mutex thrpt 25 23613.773 ± 8465.660 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_non-adaptive-mutex thrpt 25 16768.172 ± 5618.489 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_thread-local thrpt 25 23921.164 ± 8734.742 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_no-reuse thrpt 25 17899.684 ± 839.679 ops/s ComparatorBenchmarks.put java_bytewise_direct_no-reuse thrpt 25 22148.316 ± 1215.527 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_adaptive-mutex thrpt 25 11311.126 ± 820.602 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_non-adaptive-mutex thrpt 25 11421.311 ± 807.210 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_thread-local thrpt 25 11554.005 ± 960.556 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_adaptive-mutex thrpt 25 22960.523 ± 1673.421 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_non-adaptive-mutex thrpt 25 18293.317 ± 1434.601 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_thread-local thrpt 25 24479.361 ± 2157.306 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_no-reuse thrpt 25 7942.286 ± 626.170 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_no-reuse thrpt 25 11781.955 ± 1019.843 ops/s ``` Pull Request resolved: https://github.com/facebook/rocksdb/pull/6252 Differential Revision: D19331064 Pulled By: pdillinger fbshipit-source-id: 1f3b794e6a14162b2c3ffb943e8c0e64a0c03738
1 parent 800d24d
| File | Mode | Size |
|---|---|---|
| buckifier | ||
| build_tools | ||
| cache | ||
| cmake | ||
| coverage | ||
| db | ||
| db_stress_tool | ||
| docs | ||
| env | ||
| examples | ||
| file | ||
| hdfs | ||
| include | ||
| java | ||
| logging | ||
| memory | ||
| memtable | ||
| monitoring | ||
| options | ||
| port | ||
| table | ||
| test_util | ||
| third-party | ||
| tools | ||
| trace_replay | ||
| util | ||
| utilities | ||
| .clang-format | -rw-r--r-- | 138 bytes |
| .gitignore | -rw-r--r-- | 875 bytes |
| .lgtm.yml | -rw-r--r-- | 67 bytes |
| .travis.yml | -rw-r--r-- | 3.7 KB |
| .watchmanconfig | -rw-r--r-- | 130 bytes |
| AUTHORS | -rw-r--r-- | 322 bytes |
| CMakeLists.txt | -rw-r--r-- | 39.4 KB |
| CODE_OF_CONDUCT.md | -rw-r--r-- | 3.3 KB |
| CONTRIBUTING.md | -rw-r--r-- | 706 bytes |
| COPYING | -rw-r--r-- | 17.7 KB |
| DEFAULT_OPTIONS_HISTORY.md | -rw-r--r-- | 1.5 KB |
| DUMP_FORMAT.md | -rw-r--r-- | 763 bytes |
| HISTORY.md | -rw-r--r-- | 104.7 KB |
| INSTALL.md | -rw-r--r-- | 7.5 KB |
| LANGUAGE-BINDINGS.md | -rw-r--r-- | 1.1 KB |
| LICENSE.Apache | -rw-r--r-- | 11.1 KB |
| LICENSE.leveldb | -rw-r--r-- | 1.5 KB |
| Makefile | -rw-r--r-- | 74.5 KB |
| README.md | -rw-r--r-- | 1.8 KB |
| ROCKSDB_LITE.md | -rw-r--r-- | 1.0 KB |
| TARGETS | -rw-r--r-- | 35.0 KB |
| USERS.md | -rw-r--r-- | 6.8 KB |
| Vagrantfile | -rw-r--r-- | 1017 bytes |
| WINDOWS_PORT.md | -rw-r--r-- | 12.5 KB |
| appveyor.yml | -rw-r--r-- | 3.1 KB |
| defs.bzl | -rw-r--r-- | 1.4 KB |
| issue_template.md | -rw-r--r-- | 294 bytes |
| src.mk | -rw-r--r-- | 33.5 KB |
| thirdparty.inc | -rw-r--r-- | 7.8 KB |
Computing file changes ...
