https://github.com/facebook/rocksdb
Revision 9aa3b6f9ae840a2fe8da8eecd146d968b7b972d8 authored by Changyu Bi on 13 March 2023, 18:06:59 UTC, committed by Facebook GitHub Bot on 13 March 2023, 18:06:59 UTC
Summary: CreateColumnFamilyWithImport() did not support range tombstones for two reasons: 1. it uses point keys of a input file to determine its boundary (smallest and largest internal key), which means range tombstones outside of the point key range will be effectively dropped. 2. it does not handle files with no point keys. Also included a fix in external_sst_file_ingestion_job.cc where the blocks read in `GetIngestedFileInfo()` can be added to block cache now (issue fixed in https://github.com/facebook/rocksdb/pull/6429). This PR adds support for exporting and importing column family with range tombstones. The main change is to add smallest internal key and largest internal key to `SstFileMetaData` that will be part of the output of `ExportColumnFamily()`. Then during `CreateColumnFamilyWithImport(...,const ExportImportFilesMetaData& metadata,...)`, file boundaries can be set from `metadata` directly. This is needed since when file boundaries are extended by range tombstones, sometimes they cannot be deduced from a file's content alone. Pull Request resolved: https://github.com/facebook/rocksdb/pull/11252 Test Plan: - added unit tests that fails before this change Closes https://github.com/facebook/rocksdb/issues/11245 Reviewed By: ajkr Differential Revision: D43577443 Pulled By: cbi42 fbshipit-source-id: 6bff78e583cc50c44854994dea0a8dd519398f2f
1 parent fbd603d
Tip revision: 9aa3b6f9ae840a2fe8da8eecd146d968b7b972d8 authored by Changyu Bi on 13 March 2023, 18:06:59 UTC
Support range deletion tombstones in `CreateColumnFamilyWithImport` (#11252)
Support range deletion tombstones in `CreateColumnFamilyWithImport` (#11252)
Tip revision: 9aa3b6f
| File | Mode | Size |
|---|---|---|
| .circleci | ||
| .github | ||
| buckifier | ||
| build_tools | ||
| cache | ||
| cmake | ||
| coverage | ||
| db | ||
| db_stress_tool | ||
| docs | ||
| env | ||
| examples | ||
| file | ||
| fuzz | ||
| include | ||
| java | ||
| logging | ||
| memory | ||
| memtable | ||
| microbench | ||
| monitoring | ||
| options | ||
| plugin | ||
| port | ||
| table | ||
| test_util | ||
| third-party | ||
| tools | ||
| trace_replay | ||
| util | ||
| utilities | ||
| .clang-format | -rw-r--r-- | 138 bytes |
| .gitignore | -rw-r--r-- | 1.0 KB |
| .lgtm.yml | -rw-r--r-- | 67 bytes |
| .watchmanconfig | -rw-r--r-- | 130 bytes |
| AUTHORS | -rw-r--r-- | 322 bytes |
| CMakeLists.txt | -rw-r--r-- | 56.8 KB |
| CODE_OF_CONDUCT.md | -rw-r--r-- | 3.3 KB |
| CONTRIBUTING.md | -rw-r--r-- | 706 bytes |
| COPYING | -rw-r--r-- | 17.7 KB |
| DEFAULT_OPTIONS_HISTORY.md | -rw-r--r-- | 1.5 KB |
| DUMP_FORMAT.md | -rw-r--r-- | 763 bytes |
| HISTORY.md | -rw-r--r-- | 274.2 KB |
| INSTALL.md | -rw-r--r-- | 8.3 KB |
| LANGUAGE-BINDINGS.md | -rw-r--r-- | 1.3 KB |
| LICENSE.Apache | -rw-r--r-- | 11.1 KB |
| LICENSE.leveldb | -rw-r--r-- | 1.5 KB |
| Makefile | -rw-r--r-- | 99.3 KB |
| PLUGINS.md | -rw-r--r-- | 705 bytes |
| README.md | -rw-r--r-- | 2.0 KB |
| TARGETS | -rw-r--r-- | 658.3 KB |
| USERS.md | -rw-r--r-- | 8.4 KB |
| Vagrantfile | -rw-r--r-- | 1017 bytes |
| WINDOWS_PORT.md | -rw-r--r-- | 12.5 KB |
| common.mk | -rw-r--r-- | 963 bytes |
| crash_test.mk | -rw-r--r-- | 4.2 KB |
| issue_template.md | -rw-r--r-- | 294 bytes |
| rocksdb.pc.in | -rw-r--r-- | 292 bytes |
| src.mk | -rw-r--r-- | 45.8 KB |
| thirdparty.inc | -rw-r--r-- | 7.8 KB |
Computing file changes ...
