voldemort.utils
Class ConsistencyCheck

java.lang.Object
  extended by voldemort.utils.ConsistencyCheck

public class ConsistencyCheck
extends java.lang.Object


Nested Class Summary
protected static class ConsistencyCheck.ClusterNode
          Used to track nodes that may share the same nodeId in different clusters
protected static class ConsistencyCheck.ConsistencyLevel
           
protected static class ConsistencyCheck.HashedValue
          A class to save version and value hash It is used to compare versions by the value hash
protected static class ConsistencyCheck.KeyFetchTracker
          A class to track what keys have been fetched and what keys will not appear any more.
protected static class ConsistencyCheck.Reporter
          Used to report bad keys, progress, and statistics
protected static class ConsistencyCheck.RetentionChecker
          A checker to determine if a key is to be cleaned according to retention policy
 
Constructor Summary
ConsistencyCheck(java.util.List<java.lang.String> urls, java.lang.String storeName, int partitionId, java.io.BufferedWriter badKeyWriter)
           
 
Method Summary
static void cleanIneligibleKeys(java.util.Map<ByteArray,java.util.Map<Version,java.util.Set<ConsistencyCheck.ClusterNode>>> keyVersionNodeSetMap, int requiredWrite)
          Determine if a key version is invalid by comparing the version's existance and required writes configuration
 void connect()
          Connect to the clusters using given urls and start fetching process on correct nodes
static ConsistencyCheck.ConsistencyLevel determineConsistency(java.util.Map<Version,java.util.Set<ConsistencyCheck.ClusterNode>> versionNodeSetMap, int replicationFactor)
          Determine the consistency level of a key
 ConsistencyCheck.Reporter execute()
          Run consistency check on connected key-value iterators
static java.lang.String keyVersionToString(ByteArray key, java.util.Map<Version,java.util.Set<ConsistencyCheck.ClusterNode>> versionMap, java.lang.String storeName, java.lang.Integer partitionId)
          Convert a key-version-nodeSet information to string
static void main(java.lang.String[] args)
           
 void recordFetch(ConsistencyCheck.ClusterNode clusterNode, ByteArray key, Versioned<byte[]> versioned)
           
 void trySweepAll()
           
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

ConsistencyCheck

public ConsistencyCheck(java.util.List<java.lang.String> urls,
                        java.lang.String storeName,
                        int partitionId,
                        java.io.BufferedWriter badKeyWriter)
Method Detail

connect

public void connect()
             throws java.lang.Exception
Connect to the clusters using given urls and start fetching process on correct nodes

Throws:
java.lang.Exception - When no such store is found

execute

public ConsistencyCheck.Reporter execute()
                                  throws java.io.IOException
Run consistency check on connected key-value iterators

Returns:
Results in form of ConsistencyCheckStats
Throws:
java.io.IOException

trySweepAll

public void trySweepAll()

recordFetch

public void recordFetch(ConsistencyCheck.ClusterNode clusterNode,
                        ByteArray key,
                        Versioned<byte[]> versioned)

determineConsistency

public static ConsistencyCheck.ConsistencyLevel determineConsistency(java.util.Map<Version,java.util.Set<ConsistencyCheck.ClusterNode>> versionNodeSetMap,
                                                                     int replicationFactor)
Determine the consistency level of a key

Parameters:
versionNodeSetMap - A map that maps version to set of PrefixNodes
replicationFactor - Total replication factor for the set of clusters
Returns:
ConsistencyLevel Enum

cleanIneligibleKeys

public static void cleanIneligibleKeys(java.util.Map<ByteArray,java.util.Map<Version,java.util.Set<ConsistencyCheck.ClusterNode>>> keyVersionNodeSetMap,
                                       int requiredWrite)
Determine if a key version is invalid by comparing the version's existance and required writes configuration

Parameters:
keyVersionNodeSetMap - A map that contains keys mapping to a map that maps versions to set of PrefixNodes
requiredWrite - Required Write configuration

main

public static void main(java.lang.String[] args)
                 throws java.lang.Exception
Throws:
java.lang.Exception

keyVersionToString

public static java.lang.String keyVersionToString(ByteArray key,
                                                  java.util.Map<Version,java.util.Set<ConsistencyCheck.ClusterNode>> versionMap,
                                                  java.lang.String storeName,
                                                  java.lang.Integer partitionId)
Convert a key-version-nodeSet information to string

Parameters:
key - The key
versionMap - mapping versions to set of PrefixNodes
storeName - store's name
partitionId - partition scanned
Returns:
a string that describe the information passed in


Jay Kreps, Roshan Sumbaly, Alex Feinberg, Bhupesh Bansal, Lei Gao, Chinmay Soman, Vinoth Chandar, Zhongjie Wu