https://github.com/voldemort/voldemort
Revision 30bbdd5a306d1b87a8e92cd9254c8a1c132ed2db authored by Arunachalam Thirupathi on 09 May 2014, 21:22:07 UTC, committed by ARUNACHALAM THIRUPATHI on 10 May 2014, 18:20:57 UTC
Fix the new admin metadata check command and did the same change for old admin command. 1) If check metadata fails it says true or false but not the actual failure. 2) Check metadata sometimes incorrectly reports failure depending on the order in which stores are returned. 3) Check metadata will reprot success if a store is completely missing from one of the nodes. It does not track number of times retrieved. With this commit the check metadata is done using a map. For stores check The map will be from storeName to (map from store defintion to nodeName) for other The map will be from object of that key to nodename. This map will be used for consistency check. This map will group non matching information to groups for easier analysis. The lenght of the value in the map will be evaluated for missing information, solving the problem 2 and 3 listed above
1 parent b5fcb89
Tip revision: 30bbdd5a306d1b87a8e92cd9254c8a1c132ed2db authored by Arunachalam Thirupathi on 09 May 2014, 21:22:07 UTC
check metadata incorrect results, hard analysis
check metadata incorrect results, hard analysis
Tip revision: 30bbdd5
NOTES
Getting Started
For the most up-to-date information see http://project-voldemort.com
## checkout and build
jkreps@jkreps-md:/tmp > svn co svn+ssh://cm01.corp/lirepo/voldemort/trunk voldemort
jkreps@jkreps-md:/tmp > cd voldemort
jkreps@jkreps-md:/tmp/voldemort > ant
## start single node cluster and connect to table named ﾒtestﾓ
jkreps@jkreps-md:/tmp/voldemort > ./bin/voldemort-server.sh config/single_node_cluster &
jkreps@jkreps-md:/tmp/voldemort> ./bin/voldemort-shell.sh test tcp://localhost:6666
## run some random commands to put and get strings
> help
Commands:
put key value -- Associate the given value with the key.
get key -- Retrieve the value associated with the key.
delete key -- Remove all values associated with the key.
preflist key -- Get node preference list for given key.
help -- Print this message.
exit -- Exit from this shell.
> put "hello" "there"
> get "hello"
version(0:1): "there"
> preflist "hello"
Node 0
host: localhost
port: 6666
available: yes
last checked: 4614 ms ago
Example usage in example/java/voldemort/example/VoldemortExample.java.
Example configurations in config/
Code layout
annotations - Helper annotations
client - Code specific to the client
cluster - domain model for a voldemort cluster
routing - Code specific to routing requests
serialization - Code for turning bytes into objects and vice versa
server - Code for handling client requests
store - All the store implementations used by both client and server
utils - Helpers!
versioning - Vector clock stuff
xml - Code for serializing the configuration data...should probably be move to serialization
Background Resources
- Amazon Dynamo Paper -- http://s3.amazonaws.com/AllThingsDistributed/sosp/amazon-dynamo-sosp2007.pdf
- http://www.allthingsdistributed.com/2007/12/eventually_consistent.html
- OpenDHT and Bamboo papers
- BDB Performance: http://www.oracle.com/technology/products/berkeley-db/pdf/berkeley-db-perf.pdf
- Origin of vector clocks: http://research.microsoft.com/users/lamport/pubs/time-clocks.pdf
- Brewer's conjecture: http://citeseer.ist.psu.edu/544596.html
Current build is from r19
Supporting other clients
- Each store is available via all protcols, they are seperated by port
- Wire format vs. protocol (HTTP vs. Tcp/IP), protocol buffers
- How to abstract wire format?
- A given serialization type may or may not be supported by the client language
Socket servers share threadpool?
A WireProtocol takes bytes and creates a voldemort request, and takes objects to create a voldemort response.
Likewise the client does the opposite
Connector.handleRequest()
Computing file changes ...
