swh:1:snp:5115096b921df712aeb2a08114fede57fb3331fb
Revision 07030c6f4a08c18f99051faf288330d14e57daec authored by Cheng Chang on 08 December 2020, 18:56:50 UTC, committed by Facebook GitHub Bot on 08 December 2020, 18:58:04 UTC
Summary: Consider the case: 1. All column families are flushed, so all WALs become obsolete, but no WAL is removed from disk yet because the removal is asynchronous, a VersionEdit is written to MANIFEST indicating that WALs before a certain WAL number are obsolete, let's say this number is 3; 2. `SyncWAL` is called, so all the on-disk WALs are synced, and if track_and_verify_wal_in_manifest=true, the WALs will be tracked in MANIFEST, let's say the WAL numbers are 1 and 2; 3. DB crashes; 4. During DB recovery, when replaying MANIFEST, we first see that WAL with number < 3 are obsolete, then we see that WAL 1 and 2 are synced, so according to current implementation of `WalSet`, the `WalSet` will be recovered to include WAL 1 and 2; 5. WAL 1 and 2 are asynchronously deleted from disk, then the WAL verification algorithm fails with `Corruption: missing WAL`. The above case is reproduced in a new unit test `DBBasicTestTrackWal::DoNotTrackObsoleteWal`. The fix is to maintain the upper bound of the obsolete WAL numbers, any WAL with number less than the maintained number is considered to be obsolete, so shouldn't be tracked even if they are later synced. The number is maintained in `WalSet`. Pull Request resolved: https://github.com/facebook/rocksdb/pull/7725 Test Plan: 1. a new unit test `DBBasicTestTrackWal::DoNotTrackObsoleteWal` is added. 2. run `make crash_test` on devserver. Reviewed By: riversand963 Differential Revision: D25238914 Pulled By: cheng-chang fbshipit-source-id: f5dccd57c3d89f19565ec5731f2d42f06d272b72
1 parent 11c4be2
Tip revision: 19076c95aa2bcee55c26fcf0960cc844ad86ee9c authored by Levi Tamasi on 21 January 2021, 22:18:25 UTC
Update HISTORY.md for PR 7888 (#7890)
Update HISTORY.md for PR 7888 (#7890)
Tip revision: 19076c9
File | Mode | Size |
---|---|---|
.circleci | ||
.github | ||
buckifier | ||
build_tools | ||
cache | ||
cmake | ||
coverage | ||
db | ||
db_stress_tool | ||
docs | ||
env | ||
examples | ||
file | ||
fuzz | ||
hdfs | ||
include | ||
java | ||
logging | ||
memory | ||
memtable | ||
monitoring | ||
options | ||
port | ||
table | ||
test_util | ||
third-party | ||
tools | ||
trace_replay | ||
util | ||
utilities | ||
.clang-format | -rw-r--r-- | 138 bytes |
.gitignore | -rw-r--r-- | 1.0 KB |
.lgtm.yml | -rw-r--r-- | 67 bytes |
.travis.yml | -rw-r--r-- | 10.0 KB |
.watchmanconfig | -rw-r--r-- | 130 bytes |
AUTHORS | -rw-r--r-- | 322 bytes |
CMakeLists.txt | -rw-r--r-- | 45.2 KB |
CODE_OF_CONDUCT.md | -rw-r--r-- | 3.3 KB |
CONTRIBUTING.md | -rw-r--r-- | 706 bytes |
COPYING | -rw-r--r-- | 17.7 KB |
DEFAULT_OPTIONS_HISTORY.md | -rw-r--r-- | 1.5 KB |
DUMP_FORMAT.md | -rw-r--r-- | 763 bytes |
HISTORY.md | -rw-r--r-- | 146.0 KB |
INSTALL.md | -rw-r--r-- | 7.5 KB |
LANGUAGE-BINDINGS.md | -rw-r--r-- | 1.2 KB |
LICENSE.Apache | -rw-r--r-- | 11.1 KB |
LICENSE.leveldb | -rw-r--r-- | 1.5 KB |
Makefile | -rw-r--r-- | 86.0 KB |
README.md | -rw-r--r-- | 2.0 KB |
ROCKSDB_LITE.md | -rw-r--r-- | 1.0 KB |
TARGETS | -rw-r--r-- | 52.5 KB |
USERS.md | -rw-r--r-- | 7.3 KB |
Vagrantfile | -rw-r--r-- | 1017 bytes |
WINDOWS_PORT.md | -rw-r--r-- | 12.5 KB |
appveyor.yml | -rw-r--r-- | 3.3 KB |
defs.bzl | -rw-r--r-- | 1.9 KB |
issue_template.md | -rw-r--r-- | 294 bytes |
src.mk | -rw-r--r-- | 39.6 KB |
thirdparty.inc | -rw-r--r-- | 7.8 KB |
Computing file changes ...