https://github.com/voldemort/voldemort
Revision 5f51537c1b478d6b2e1eab5a97f7112437a0e17d authored by Felix GV on 29 October 2015, 01:51:20 UTC, committed by Felix GV on 29 October 2015, 23:35:10 UTC
Given a data directory, the ReadOnlyFileValidator class checks a few things: - Index files are well formed: - Well sorted - No negative offset - Data files are well formed: - Number of records for a given key hash is > 0 - Key and value sizes are > 0 - Index and data files match together: - No gaps in data files, when traversing them with the offsets from index files - Raw keys in data files match the hashes found in their index files It then prints a summary of the amount and kinds of problems found. Disclaimers: - This tool only works with Voldemort Read-Only V2 formatted files. - This tool does NOT rely on the checksum in the .metadata file. - This tool only inspects the structural integrity of index and data files, and whether they match together. - If a problem is detected, you're in trouble for sure, but if no problems are detected, that doesn't mean an unsupported problem didn't slip through undetected (i.e.: false negatives are possible but false positives are not). Also added a bash run script in bin/read-only-file-validator.sh
1 parent ec50a64
Tip revision: 5f51537c1b478d6b2e1eab5a97f7112437a0e17d authored by Felix GV on 29 October 2015, 01:51:20 UTC
Added a validator for Read-Only files.
Added a validator for Read-Only files.
Tip revision: 5f51537
| File | Mode | Size |
|---|---|---|
| .settings | ||
| bin | ||
| clients | ||
| config | ||
| contrib | ||
| docs | ||
| example | ||
| gradle | ||
| private-lib | ||
| src | ||
| test | ||
| voldemort-contrib | ||
| .gitignore | -rw-r--r-- | 242 bytes |
| CONTRIBUTORS | -rw-r--r-- | 659 bytes |
| LICENSE | -rw-r--r-- | 11.1 KB |
| NOTES | -rw-r--r-- | 2.5 KB |
| NOTICE | -rw-r--r-- | 8.1 KB |
| README.md | -rw-r--r-- | 4.4 KB |
| build.gradle | -rw-r--r-- | 20.0 KB |
| build.xml | -rw-r--r-- | 1.7 KB |
| gradle.properties | -rw-r--r-- | 1.1 KB |
| gradlew | -rwxr-xr-x | 5.0 KB |
| gradlew.bat | -rw-r--r-- | 2.3 KB |
| release_notes.txt | -rw-r--r-- | 46.1 KB |
| settings.gradle | -rw-r--r-- | 149 bytes |
| tomcat-tasks.properties | -rw-r--r-- | 420 bytes |
| web.xml | -rw-r--r-- | 1.1 KB |
Computing file changes ...
