voldemort.store.metadata
Class MetadataStore

java.lang.Object
  extended by voldemort.store.AbstractStore<K,V,T>
      extended by voldemort.store.AbstractStorageEngine<ByteArray,byte[],byte[]>
          extended by voldemort.store.metadata.MetadataStore
All Implemented Interfaces:
StorageEngine<ByteArray,byte[],byte[]>, Store<ByteArray,byte[],byte[]>

public class MetadataStore
extends AbstractStorageEngine<ByteArray,byte[],byte[]>

MetadataStore maintains metadata for Voldemort Server.
Metadata is persisted as strings in inner store for ease of readability.
Metadata Store keeps an in memory write-through-cache for performance.


Nested Class Summary
static class MetadataStore.VoldemortState
           
 
Field Summary
static java.lang.String CLUSTER_KEY
           
static java.util.Set<java.lang.String> GOSSIP_KEYS
           
static java.util.Set<java.lang.Object> METADATA_KEYS
           
static java.lang.String METADATA_STORE_NAME
           
static java.lang.String NODE_ID_KEY
           
static java.util.Set<java.lang.String> OPTIONAL_KEYS
           
 java.util.concurrent.locks.Lock readLock
           
static java.lang.String REBALANCING_SOURCE_CLUSTER_XML
           
static java.lang.String REBALANCING_SOURCE_STORES_XML
           
static java.lang.String REBALANCING_STEAL_INFO
           
static java.util.Set<java.lang.String> REQUIRED_KEYS
           
static java.lang.String SERVER_STATE_KEY
           
static java.lang.String STORE_DEFINITIONS_STORE_NAME
           
static java.lang.String STORES_KEY
           
static java.lang.String SYSTEM_STORES_KEY
           
 java.util.concurrent.locks.Lock writeLock
           
 
Constructor Summary
MetadataStore(Store<java.lang.String,java.lang.String,java.lang.String> innerStore, int nodeId)
           
MetadataStore(Store<java.lang.String,java.lang.String,java.lang.String> innerStore, StorageEngine<java.lang.String,java.lang.String,java.lang.String> storeDefinitionsStorageEngine, int nodeId)
           
 
Method Summary
 void addMetadataStoreListener(java.lang.String storeName, MetadataStoreListener listener)
           
 void addRebalancingState(RebalanceTaskInfo stealInfo)
          Add the steal information to the rebalancer state
 void addStoreDefinition(StoreDefinition storeDef)
          Function to add a new Store to the Metadata store.
 void cleanAllRebalancingState()
           
 void close()
          Close the store.
 boolean delete(ByteArray key, Version version)
          Delete all entries prior to the given version
 void deleteRebalancingState(RebalanceTaskInfo stealInfo)
          Delete the partition steal information from the rebalancer state
 void deleteStoreDefinition(java.lang.String storeName)
          Function to delete the specified store from Metadata store.
 ClosableIterator<Pair<ByteArray,Versioned<byte[]>>> entries()
          Get an iterator over pairs of entries in the store.
 ClosableIterator<Pair<ByteArray,Versioned<byte[]>>> entries(int partition)
          Get an iterator over pairs of entries in a store's partition.
 java.util.List<Versioned<byte[]>> get(ByteArray keyBytes, byte[] transforms)
          Get the value associated with the given key
 java.util.List<Versioned<byte[]>> get(java.lang.String key, java.lang.String transforms)
           
 java.util.Map<ByteArray,java.util.List<Versioned<byte[]>>> getAll(java.lang.Iterable<ByteArray> keys, java.util.Map<ByteArray,byte[]> transforms)
          Get the values associated with the given keys and returns them in a Map of keys to a list of versioned values.
 java.lang.Object getCapability(StoreCapabilityType capability)
          Get some capability of the store.
 Cluster getCluster()
           
 java.lang.String getName()
           
 int getNodeId()
           
 RebalancerState getRebalancerState()
           
 Cluster getRebalancingSourceCluster()
           
 java.util.List<StoreDefinition> getRebalancingSourceStores()
           
 RoutingStrategy getRoutingStrategy(java.lang.String storeName)
           
 MetadataStore.VoldemortState getServerStateLocked()
           
 MetadataStore.VoldemortState getServerStateUnlocked()
           
 StoreDefinition getStoreDef(java.lang.String storeName)
           
 java.util.List<StoreDefinition> getStoreDefList()
           
 java.util.List<StoreDefinition> getSystemStoreDefList()
           
 java.util.List<Version> getVersions(ByteArray key)
          Get the versions associated with the given key.
 boolean isValidStore(java.lang.String name)
          Utility function to validate if the given store name exists in the store name list managed by MetadataStore.
 ClosableIterator<ByteArray> keys()
          Get an iterator over keys in the store.
 ClosableIterator<ByteArray> keys(int partition)
          Get an iterator over keys in the store's partition Note that the iterator need not be threadsafe, and that it must be manually closed after use.
 void put(ByteArray keyBytes, Versioned<byte[]> valueBytes, byte[] transforms)
          A write through put to inner-store.
 void put(java.lang.String key, java.lang.Object value)
          helper function to read current version and put() after incrementing it for local node.
 void put(java.lang.String key, Versioned<java.lang.Object> value)
          helper function to convert strings to bytes as needed.
static MetadataStore readFromDirectory(java.io.File dir, int nodeId)
           
 void removeMetadataStoreListener(java.lang.String storeName)
           
 void truncate()
          Truncate all entries in the store
 void updateStoreDefinitions(Versioned<byte[]> valueBytes)
          Function to update store definitions.
 
Methods inherited from class voldemort.store.AbstractStorageEngine
beginBatchModifications, endBatchModifications, getAndLock, isPartitionAware, isPartitionScanSupported, multiVersionPut, putAndUnlock, releaseLock, resolveAndConstructVersionsToPersist
 
Methods inherited from class voldemort.store.AbstractStore
delete, get, getAll, put
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 
Methods inherited from interface voldemort.store.Store
delete, get, getAll, put
 

Field Detail

METADATA_STORE_NAME

public static final java.lang.String METADATA_STORE_NAME
See Also:
Constant Field Values

STORE_DEFINITIONS_STORE_NAME

public static final java.lang.String STORE_DEFINITIONS_STORE_NAME
See Also:
Constant Field Values

CLUSTER_KEY

public static final java.lang.String CLUSTER_KEY
See Also:
Constant Field Values

STORES_KEY

public static final java.lang.String STORES_KEY
See Also:
Constant Field Values

SYSTEM_STORES_KEY

public static final java.lang.String SYSTEM_STORES_KEY
See Also:
Constant Field Values

SERVER_STATE_KEY

public static final java.lang.String SERVER_STATE_KEY
See Also:
Constant Field Values

NODE_ID_KEY

public static final java.lang.String NODE_ID_KEY
See Also:
Constant Field Values

REBALANCING_STEAL_INFO

public static final java.lang.String REBALANCING_STEAL_INFO
See Also:
Constant Field Values

REBALANCING_SOURCE_CLUSTER_XML

public static final java.lang.String REBALANCING_SOURCE_CLUSTER_XML
See Also:
Constant Field Values

REBALANCING_SOURCE_STORES_XML

public static final java.lang.String REBALANCING_SOURCE_STORES_XML
See Also:
Constant Field Values

GOSSIP_KEYS

public static final java.util.Set<java.lang.String> GOSSIP_KEYS

REQUIRED_KEYS

public static final java.util.Set<java.lang.String> REQUIRED_KEYS

OPTIONAL_KEYS

public static final java.util.Set<java.lang.String> OPTIONAL_KEYS

METADATA_KEYS

public static final java.util.Set<java.lang.Object> METADATA_KEYS

readLock

public final java.util.concurrent.locks.Lock readLock

writeLock

public final java.util.concurrent.locks.Lock writeLock
Constructor Detail

MetadataStore

public MetadataStore(Store<java.lang.String,java.lang.String,java.lang.String> innerStore,
                     StorageEngine<java.lang.String,java.lang.String,java.lang.String> storeDefinitionsStorageEngine,
                     int nodeId)

MetadataStore

public MetadataStore(Store<java.lang.String,java.lang.String,java.lang.String> innerStore,
                     int nodeId)
Method Detail

addMetadataStoreListener

public void addMetadataStoreListener(java.lang.String storeName,
                                     MetadataStoreListener listener)

removeMetadataStoreListener

public void removeMetadataStoreListener(java.lang.String storeName)

readFromDirectory

public static MetadataStore readFromDirectory(java.io.File dir,
                                              int nodeId)

getName

public java.lang.String getName()
Specified by:
getName in interface Store<ByteArray,byte[],byte[]>
Overrides:
getName in class AbstractStore<ByteArray,byte[],byte[]>
Returns:
The name of the store.

put

public void put(java.lang.String key,
                Versioned<java.lang.Object> value)
helper function to convert strings to bytes as needed.

Parameters:
key -
value -

updateStoreDefinitions

public void updateStoreDefinitions(Versioned<byte[]> valueBytes)
Function to update store definitions. Unlike the put method, this function does not delete any existing state. It only updates the state of the stores specified in the given stores.xml

Parameters:
valueBytes - specifies the bytes of the stores.xml containing updates for the specified stores

put

public void put(java.lang.String key,
                java.lang.Object value)
helper function to read current version and put() after incrementing it for local node.

Parameters:
key -
value -

put

public void put(ByteArray keyBytes,
                Versioned<byte[]> valueBytes,
                byte[] transforms)
         throws VoldemortException
A write through put to inner-store.

Specified by:
put in interface Store<ByteArray,byte[],byte[]>
Overrides:
put in class AbstractStore<ByteArray,byte[],byte[]>
Parameters:
keyBytes - : keyName strings serialized as bytes eg. 'cluster.xml'
valueBytes - : versioned byte[] eg. UTF bytes for cluster xml definitions
Throws:
VoldemortException

close

public void close()
           throws VoldemortException
Description copied from interface: Store
Close the store.

Specified by:
close in interface Store<ByteArray,byte[],byte[]>
Overrides:
close in class AbstractStore<ByteArray,byte[],byte[]>
Throws:
VoldemortException - If closing fails.

getCapability

public java.lang.Object getCapability(StoreCapabilityType capability)
Description copied from interface: Store
Get some capability of the store. Examples would be the serializer used, or the routing strategy. This provides a mechanism to verify that the store hierarchy has some set of capabilities without knowing the precise layering.

Specified by:
getCapability in interface Store<ByteArray,byte[],byte[]>
Overrides:
getCapability in class AbstractStore<ByteArray,byte[],byte[]>
Parameters:
capability - The capability type to retrieve
Returns:
The given capaiblity

get

public java.util.List<Versioned<byte[]>> get(ByteArray keyBytes,
                                             byte[] transforms)
                                      throws VoldemortException
Description copied from interface: Store
Get the value associated with the given key

Specified by:
get in interface Store<ByteArray,byte[],byte[]>
Overrides:
get in class AbstractStore<ByteArray,byte[],byte[]>
Parameters:
keyBytes - : keyName strings serialized as bytes eg. 'cluster.xml'
Returns:
List of values (only 1 for Metadata) versioned byte[] eg. UTF bytes for cluster xml definitions
Throws:
VoldemortException

get

public java.util.List<Versioned<byte[]>> get(java.lang.String key,
                                             java.lang.String transforms)
                                      throws VoldemortException
Throws:
VoldemortException

cleanAllRebalancingState

public void cleanAllRebalancingState()

getVersions

public java.util.List<Version> getVersions(ByteArray key)
Description copied from interface: Store
Get the versions associated with the given key. This is used in a put call to write a new value for this key

Specified by:
getVersions in interface Store<ByteArray,byte[],byte[]>
Overrides:
getVersions in class AbstractStore<ByteArray,byte[],byte[]>
Parameters:
key - The key to retrieve the versions for
Returns:
List of Versions associated with this key.

getCluster

public Cluster getCluster()

getStoreDefList

public java.util.List<StoreDefinition> getStoreDefList()

getSystemStoreDefList

public java.util.List<StoreDefinition> getSystemStoreDefList()

getNodeId

public int getNodeId()

getStoreDef

public StoreDefinition getStoreDef(java.lang.String storeName)

getServerStateLocked

public MetadataStore.VoldemortState getServerStateLocked()

getServerStateUnlocked

public MetadataStore.VoldemortState getServerStateUnlocked()

getRebalancerState

public RebalancerState getRebalancerState()

getRebalancingSourceCluster

public Cluster getRebalancingSourceCluster()

getRebalancingSourceStores

public java.util.List<StoreDefinition> getRebalancingSourceStores()

getRoutingStrategy

public RoutingStrategy getRoutingStrategy(java.lang.String storeName)

addRebalancingState

public void addRebalancingState(RebalanceTaskInfo stealInfo)
Add the steal information to the rebalancer state

Parameters:
stealInfo - The steal information to add

deleteRebalancingState

public void deleteRebalancingState(RebalanceTaskInfo stealInfo)
Delete the partition steal information from the rebalancer state

Parameters:
stealInfo - The steal information to delete

addStoreDefinition

public void addStoreDefinition(StoreDefinition storeDef)
Function to add a new Store to the Metadata store. This involves 1. Create a new entry in the ConfigurationStorageEngine for STORES. 2. Update the metadata cache. 3. Re-create the 'stores.xml' key

Parameters:
storeDef - defines the new store to be created

deleteStoreDefinition

public void deleteStoreDefinition(java.lang.String storeName)
Function to delete the specified store from Metadata store. This involves 1. Remove entry from the ConfigurationStorageEngine for STORES. 2. Update the metadata cache. 3. Re-create the 'stores.xml' key

Parameters:
storeName - specifies name of the store to be deleted.

entries

public ClosableIterator<Pair<ByteArray,Versioned<byte[]>>> entries()
Description copied from interface: StorageEngine
Get an iterator over pairs of entries in the store. The key is the first element in the pair and the versioned value is the second element. Note that the iterator need not be threadsafe, and that it must be manually closed after use.

Specified by:
entries in interface StorageEngine<ByteArray,byte[],byte[]>
Overrides:
entries in class AbstractStorageEngine<ByteArray,byte[],byte[]>
Returns:
An iterator over the entries in this StorageEngine.

keys

public ClosableIterator<ByteArray> keys()
Description copied from interface: StorageEngine
Get an iterator over keys in the store. Note that the iterator need not be threadsafe, and that it must be manually closed after use.

Specified by:
keys in interface StorageEngine<ByteArray,byte[],byte[]>
Overrides:
keys in class AbstractStorageEngine<ByteArray,byte[],byte[]>
Returns:
An iterator over the keys in this StorageEngine.

entries

public ClosableIterator<Pair<ByteArray,Versioned<byte[]>>> entries(int partition)
Description copied from interface: StorageEngine
Get an iterator over pairs of entries in a store's partition. The key is the first element in the pair and the versioned value is the second element. Note that the iterator need not be threadsafe, and that it must be manually closed after use.

Specified by:
entries in interface StorageEngine<ByteArray,byte[],byte[]>
Overrides:
entries in class AbstractStorageEngine<ByteArray,byte[],byte[]>
Parameters:
partition - partition whose entries are to be fetched
Returns:
An iterator over the entries in this StorageEngine.

keys

public ClosableIterator<ByteArray> keys(int partition)
Description copied from interface: StorageEngine
Get an iterator over keys in the store's partition Note that the iterator need not be threadsafe, and that it must be manually closed after use.

Specified by:
keys in interface StorageEngine<ByteArray,byte[],byte[]>
Overrides:
keys in class AbstractStorageEngine<ByteArray,byte[],byte[]>
Parameters:
partition - partition whose keys are to be fetched
Returns:
An iterator over the keys in this StorageEngine.

truncate

public void truncate()
Description copied from interface: StorageEngine
Truncate all entries in the store

Specified by:
truncate in interface StorageEngine<ByteArray,byte[],byte[]>
Overrides:
truncate in class AbstractStorageEngine<ByteArray,byte[],byte[]>

delete

public boolean delete(ByteArray key,
                      Version version)
               throws VoldemortException
Description copied from interface: Store
Delete all entries prior to the given version

Specified by:
delete in interface Store<ByteArray,byte[],byte[]>
Overrides:
delete in class AbstractStore<ByteArray,byte[],byte[]>
Parameters:
key - The key to delete
version - The current value of the key
Returns:
True if anything was deleted
Throws:
VoldemortException

getAll

public java.util.Map<ByteArray,java.util.List<Versioned<byte[]>>> getAll(java.lang.Iterable<ByteArray> keys,
                                                                         java.util.Map<ByteArray,byte[]> transforms)
                                                                  throws VoldemortException
Description copied from interface: Store
Get the values associated with the given keys and returns them in a Map of keys to a list of versioned values. Note that the returned map will only contain entries for the keys which have a value associated with them.

Specified by:
getAll in interface Store<ByteArray,byte[],byte[]>
Overrides:
getAll in class AbstractStore<ByteArray,byte[],byte[]>
Parameters:
keys - The keys to check for.
Returns:
A Map of keys to a list of versioned values.
Throws:
VoldemortException

isValidStore

public boolean isValidStore(java.lang.String name)
Utility function to validate if the given store name exists in the store name list managed by MetadataStore. This is used by the Admin service for validation before serving a get-metadata request.

Parameters:
name - Name of the store to validate
Returns:
True if the store name exists in the 'storeNames' list. False otherwise.


Jay Kreps, Roshan Sumbaly, Alex Feinberg, Bhupesh Bansal, Lei Gao, Chinmay Soman, Vinoth Chandar, Zhongjie Wu