voldemort.store.slop
Class SlopStorageEngine

java.lang.Object
  extended by voldemort.store.AbstractStore<K,V,T>
      extended by voldemort.store.AbstractStorageEngine<ByteArray,byte[],byte[]>
          extended by voldemort.store.slop.SlopStorageEngine
All Implemented Interfaces:
StorageEngine<ByteArray,byte[],byte[]>, Store<ByteArray,byte[],byte[]>

public class SlopStorageEngine
extends AbstractStorageEngine<ByteArray,byte[],byte[]>

Tracks statistics of hints that were attempted, but not successfully pushed last time a pusher job ran; also tracks hints that have been added after the last run


Field Summary
static java.lang.String SLOP_STORE_NAME
           
 
Constructor Summary
SlopStorageEngine(StorageEngine<ByteArray,byte[],byte[]> slopEngine, Cluster cluster)
           
 
Method Summary
 StorageEngine<ByteArray,Slop,byte[]> asSlopStore()
           
 boolean beginBatchModifications()
          A lot of storage engines support efficient methods for performing large number of writes (puts/deletes) against the data source.
 void close()
          Close the store.
 boolean delete(ByteArray key, Version version)
          Delete all entries prior to the given version
 boolean endBatchModifications()
           
 ClosableIterator<Pair<ByteArray,Versioned<byte[]>>> entries()
          Get an iterator over pairs of entries in the store.
 ClosableIterator<Pair<ByteArray,Versioned<byte[]>>> entries(int partition)
          Get an iterator over pairs of entries in a store's partition.
 java.util.List<Versioned<byte[]>> get(ByteArray key, byte[] transforms)
          Get the value associated with the given key
 java.util.Map<ByteArray,java.util.List<Versioned<byte[]>>> getAll(java.lang.Iterable<ByteArray> keys, java.util.Map<ByteArray,byte[]> transforms)
          Get the values associated with the given keys and returns them in a Map of keys to a list of versioned values.
 java.lang.Object getCapability(StoreCapabilityType capability)
          Get some capability of the store.
 java.util.Map<java.lang.Integer,java.lang.Long> getOutstandingByNode()
           
 java.util.Map<java.lang.Integer,java.lang.Long> getOutstandingByZone()
           
 long getOutstandingTotal()
           
 java.util.List<Version> getVersions(ByteArray key)
          Get the versions associated with the given key.
 boolean isPartitionAware()
          Are partitions persisted in distinct files? In other words is the data stored on disk on a per-partition basis? This is really for the read-only use case in which each partition is stored in a distinct file.
 boolean isPartitionScanSupported()
          Does the storage engine support efficient scanning of a single partition?
 ClosableIterator<ByteArray> keys()
          Get an iterator over keys in the store.
 ClosableIterator<ByteArray> keys(int partition)
          Get an iterator over keys in the store's partition Note that the iterator need not be threadsafe, and that it must be manually closed after use.
 void put(ByteArray key, Versioned<byte[]> value, byte[] transforms)
          Associate the value with the key and version in this store
 void resetStats(java.util.Map<java.lang.Integer,java.lang.Long> newValues)
           
 void truncate()
          Truncate all entries in the store
 
Methods inherited from class voldemort.store.AbstractStorageEngine
multiVersionPut, resolveAndConstructVersionsToPersist
 
Methods inherited from class voldemort.store.AbstractStore
delete, get, getAll, getName, put
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 
Methods inherited from interface voldemort.store.Store
delete, get, getAll, getName, put
 

Field Detail

SLOP_STORE_NAME

public static final java.lang.String SLOP_STORE_NAME
See Also:
Constant Field Values
Constructor Detail

SlopStorageEngine

public SlopStorageEngine(StorageEngine<ByteArray,byte[],byte[]> slopEngine,
                         Cluster cluster)
Method Detail

getOutstandingTotal

public long getOutstandingTotal()

getOutstandingByNode

public java.util.Map<java.lang.Integer,java.lang.Long> getOutstandingByNode()

getOutstandingByZone

public java.util.Map<java.lang.Integer,java.lang.Long> getOutstandingByZone()

resetStats

public void resetStats(java.util.Map<java.lang.Integer,java.lang.Long> newValues)

asSlopStore

public StorageEngine<ByteArray,Slop,byte[]> asSlopStore()

entries

public ClosableIterator<Pair<ByteArray,Versioned<byte[]>>> entries()
Description copied from interface: StorageEngine
Get an iterator over pairs of entries in the store. The key is the first element in the pair and the versioned value is the second element. Note that the iterator need not be threadsafe, and that it must be manually closed after use.

Specified by:
entries in interface StorageEngine<ByteArray,byte[],byte[]>
Overrides:
entries in class AbstractStorageEngine<ByteArray,byte[],byte[]>
Returns:
An iterator over the entries in this StorageEngine.

keys

public ClosableIterator<ByteArray> keys()
Description copied from interface: StorageEngine
Get an iterator over keys in the store. Note that the iterator need not be threadsafe, and that it must be manually closed after use.

Specified by:
keys in interface StorageEngine<ByteArray,byte[],byte[]>
Overrides:
keys in class AbstractStorageEngine<ByteArray,byte[],byte[]>
Returns:
An iterator over the keys in this StorageEngine.

entries

public ClosableIterator<Pair<ByteArray,Versioned<byte[]>>> entries(int partition)
Description copied from interface: StorageEngine
Get an iterator over pairs of entries in a store's partition. The key is the first element in the pair and the versioned value is the second element. Note that the iterator need not be threadsafe, and that it must be manually closed after use.

Specified by:
entries in interface StorageEngine<ByteArray,byte[],byte[]>
Overrides:
entries in class AbstractStorageEngine<ByteArray,byte[],byte[]>
Parameters:
partition - partition whose entries are to be fetched
Returns:
An iterator over the entries in this StorageEngine.

keys

public ClosableIterator<ByteArray> keys(int partition)
Description copied from interface: StorageEngine
Get an iterator over keys in the store's partition Note that the iterator need not be threadsafe, and that it must be manually closed after use.

Specified by:
keys in interface StorageEngine<ByteArray,byte[],byte[]>
Overrides:
keys in class AbstractStorageEngine<ByteArray,byte[],byte[]>
Parameters:
partition - partition whose keys are to be fetched
Returns:
An iterator over the keys in this StorageEngine.

truncate

public void truncate()
Description copied from interface: StorageEngine
Truncate all entries in the store

Specified by:
truncate in interface StorageEngine<ByteArray,byte[],byte[]>
Overrides:
truncate in class AbstractStorageEngine<ByteArray,byte[],byte[]>

get

public java.util.List<Versioned<byte[]>> get(ByteArray key,
                                             byte[] transforms)
                                      throws VoldemortException
Description copied from interface: Store
Get the value associated with the given key

Specified by:
get in interface Store<ByteArray,byte[],byte[]>
Overrides:
get in class AbstractStore<ByteArray,byte[],byte[]>
Parameters:
key - The key to check for
Returns:
The value associated with the key or an empty list if no values are found.
Throws:
VoldemortException

getAll

public java.util.Map<ByteArray,java.util.List<Versioned<byte[]>>> getAll(java.lang.Iterable<ByteArray> keys,
                                                                         java.util.Map<ByteArray,byte[]> transforms)
                                                                  throws VoldemortException
Description copied from interface: Store
Get the values associated with the given keys and returns them in a Map of keys to a list of versioned values. Note that the returned map will only contain entries for the keys which have a value associated with them.

Specified by:
getAll in interface Store<ByteArray,byte[],byte[]>
Overrides:
getAll in class AbstractStore<ByteArray,byte[],byte[]>
Parameters:
keys - The keys to check for.
Returns:
A Map of keys to a list of versioned values.
Throws:
VoldemortException

put

public void put(ByteArray key,
                Versioned<byte[]> value,
                byte[] transforms)
         throws VoldemortException
Description copied from interface: Store
Associate the value with the key and version in this store

Specified by:
put in interface Store<ByteArray,byte[],byte[]>
Overrides:
put in class AbstractStore<ByteArray,byte[],byte[]>
Parameters:
key - The key to use
value - The value to store and its version.
Throws:
VoldemortException

delete

public boolean delete(ByteArray key,
                      Version version)
               throws VoldemortException
Description copied from interface: Store
Delete all entries prior to the given version

Specified by:
delete in interface Store<ByteArray,byte[],byte[]>
Overrides:
delete in class AbstractStore<ByteArray,byte[],byte[]>
Parameters:
key - The key to delete
version - The current value of the key
Returns:
True if anything was deleted
Throws:
VoldemortException

close

public void close()
           throws VoldemortException
Description copied from interface: Store
Close the store.

Specified by:
close in interface Store<ByteArray,byte[],byte[]>
Overrides:
close in class AbstractStore<ByteArray,byte[],byte[]>
Throws:
VoldemortException - If closing fails.

getCapability

public java.lang.Object getCapability(StoreCapabilityType capability)
Description copied from interface: Store
Get some capability of the store. Examples would be the serializer used, or the routing strategy. This provides a mechanism to verify that the store hierarchy has some set of capabilities without knowing the precise layering.

Specified by:
getCapability in interface Store<ByteArray,byte[],byte[]>
Overrides:
getCapability in class AbstractStore<ByteArray,byte[],byte[]>
Parameters:
capability - The capability type to retrieve
Returns:
The given capaiblity

getVersions

public java.util.List<Version> getVersions(ByteArray key)
Description copied from interface: Store
Get the versions associated with the given key. This is used in a put call to write a new value for this key

Specified by:
getVersions in interface Store<ByteArray,byte[],byte[]>
Overrides:
getVersions in class AbstractStore<ByteArray,byte[],byte[]>
Parameters:
key - The key to retrieve the versions for
Returns:
List of Versions associated with this key.

isPartitionAware

public boolean isPartitionAware()
Description copied from interface: StorageEngine
Are partitions persisted in distinct files? In other words is the data stored on disk on a per-partition basis? This is really for the read-only use case in which each partition is stored in a distinct file.

Specified by:
isPartitionAware in interface StorageEngine<ByteArray,byte[],byte[]>
Overrides:
isPartitionAware in class AbstractStorageEngine<ByteArray,byte[],byte[]>
Returns:
Boolean indicating if partitions are persisted in distinct files (read-only use case).

isPartitionScanSupported

public boolean isPartitionScanSupported()
Description copied from interface: StorageEngine
Does the storage engine support efficient scanning of a single partition?

Specified by:
isPartitionScanSupported in interface StorageEngine<ByteArray,byte[],byte[]>
Overrides:
isPartitionScanSupported in class AbstractStorageEngine<ByteArray,byte[],byte[]>
Returns:
true if the storage engine implements the capability. false otherwise

beginBatchModifications

public boolean beginBatchModifications()
Description copied from interface: StorageEngine
A lot of storage engines support efficient methods for performing large number of writes (puts/deletes) against the data source. This method puts the storage engine in this batch write mode

Specified by:
beginBatchModifications in interface StorageEngine<ByteArray,byte[],byte[]>
Overrides:
beginBatchModifications in class AbstractStorageEngine<ByteArray,byte[],byte[]>
Returns:
true if the storage engine took successful action to switch to 'batch-write' mode

endBatchModifications

public boolean endBatchModifications()
Specified by:
endBatchModifications in interface StorageEngine<ByteArray,byte[],byte[]>
Overrides:
endBatchModifications in class AbstractStorageEngine<ByteArray,byte[],byte[]>
Returns:
true if the storage engine successfully returned to normal mode


Jay Kreps, Roshan Sumbaly, Alex Feinberg, Bhupesh Bansal, Lei Gao, Chinmay Soman, Vinoth Chandar, Zhongjie Wu