voldemort.client.protocol.admin
Class AdminClient.BulkStreamingFetchOperations

java.lang.Object
  voldemort.client.protocol.admin.AdminClient.BulkStreamingFetchOperations

Enclosing class:

AdminClient

public class AdminClient.BulkStreamingFetchOperations
extends java.lang.Object

Encapsulates all the operations to forklift data from the cluster

Constructor Summary

AdminClient.BulkStreamingFetchOperations()

Method Summary

java.util.Iterator<Pair<ByteArray,Versioned<byte[]>>>	fetchEntries(int nodeId, java.lang.String storeName, java.util.HashMap<java.lang.Integer,java.util.List<java.lang.Integer>> replicaToPartitionList, VoldemortFilter filter, boolean fetchMasterEntries, Cluster initialCluster, long skipRecords) Fetch key/value tuples belonging to this map of replica type to partition list
java.util.Iterator<Pair<ByteArray,Versioned<byte[]>>>	fetchEntries(int nodeId, java.lang.String storeName, java.util.List<java.lang.Integer> partitionList, VoldemortFilter filter, boolean fetchMasterEntries) Legacy interface for fetching entries.
java.util.Iterator<Pair<ByteArray,Versioned<byte[]>>>	fetchEntries(int nodeId, java.lang.String storeName, java.util.List<java.lang.Integer> partitionList, VoldemortFilter filter, boolean fetchMasterEntries, long skipRecords) Legacy interface for fetching entries.
java.util.Iterator<ByteArray>	fetchKeys(int nodeId, java.lang.String storeName, java.util.HashMap<java.lang.Integer,java.util.List<java.lang.Integer>> replicaToPartitionList, VoldemortFilter filter, boolean fetchMasterEntries, Cluster initialCluster, long skipRecords) Fetch all keys belonging to the map of replica type to partition list.
java.util.Iterator<ByteArray>	fetchKeys(int nodeId, java.lang.String storeName, java.util.List<java.lang.Integer> partitionList, VoldemortFilter filter, boolean fetchMasterEntries) Legacy interface for fetching entries.
java.util.Iterator<ByteArray>	fetchKeys(int nodeId, java.lang.String storeName, java.util.List<java.lang.Integer> partitionList, VoldemortFilter filter, boolean fetchMasterEntries, long skipRecords) Legacy interface for fetching entries.
java.util.Iterator<Pair<ByteArray,Versioned<byte[]>>>	fetchOrphanedEntries(int nodeId, java.lang.String storeName) Fetches entries that don't belong to the node, based on current metadata and yet persisted on the node
java.util.Iterator<ByteArray>	fetchOrphanedKeys(int nodeId, java.lang.String storeName) Fetch all the keys on the node that don't belong to it, based on its current metadata and yet stored on the node.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

AdminClient.BulkStreamingFetchOperations

public AdminClient.BulkStreamingFetchOperations()

Method Detail

fetchOrphanedEntries

public java.util.Iterator<Pair<ByteArray,Versioned<byte[]>>> fetchOrphanedEntries(int nodeId,
                                                                                  java.lang.String storeName)

Fetches entries that don't belong to the node, based on current metadata and yet persisted on the node

Parameters:

nodeId - Id of the node to fetch from

storeName - Name of the store

Returns:

An iterator which allows entries to be streamed as they're being iterated over.

fetchEntries

public java.util.Iterator<Pair<ByteArray,Versioned<byte[]>>> fetchEntries(int nodeId,
                                                                          java.lang.String storeName,
                                                                          java.util.List<java.lang.Integer> partitionList,
                                                                          VoldemortFilter filter,
                                                                          boolean fetchMasterEntries,
                                                                          long skipRecords)

Legacy interface for fetching entries. See AdminClient#fetchEntries(int, String, HashMap, VoldemortFilter, boolean, Cluster, long) for more information.

Parameters:

nodeId - Id of the node to fetch from

storeName - Name of the store

partitionList - List of the partitions

filter - Custom filter implementation to filter out entries which should not be fetched.

fetchMasterEntries - Fetch an entry only if master replica

skipRecords - Number of records to skip

Returns:

An iterator which allows entries to be streamed as they're being iterated over.

fetchEntries

public java.util.Iterator<Pair<ByteArray,Versioned<byte[]>>> fetchEntries(int nodeId,
                                                                          java.lang.String storeName,
                                                                          java.util.List<java.lang.Integer> partitionList,
                                                                          VoldemortFilter filter,
                                                                          boolean fetchMasterEntries)

Legacy interface for fetching entries. See AdminClient#fetchEntries(int, String, HashMap, VoldemortFilter, boolean, Cluster, long) for more information.

Parameters:

nodeId - Id of the node to fetch from

storeName - Name of the store

partitionList - List of the partitions

filter - Custom filter implementation to filter out entries which should not be fetched.

fetchMasterEntries - Fetch an entry only if master replica

Returns:

An iterator which allows entries to be streamed as they're being iterated over.

fetchEntries

public java.util.Iterator<Pair<ByteArray,Versioned<byte[]>>> fetchEntries(int nodeId,
                                                                          java.lang.String storeName,
                                                                          java.util.HashMap<java.lang.Integer,java.util.List<java.lang.Integer>> replicaToPartitionList,
                                                                          VoldemortFilter filter,
                                                                          boolean fetchMasterEntries,
                                                                          Cluster initialCluster,
                                                                          long skipRecords)

Fetch key/value tuples belonging to this map of replica type to partition list

Streaming API - The server keeps sending the messages as it's iterating over the data. Once iteration has finished, the server sends an "end of stream" marker and flushes its buffer. A response indicating a VoldemortException may be sent at any time during the process.

Entries are being streamed as the iteration happens i.e. the whole result set is not buffered in memory.

Parameters:

nodeId - Id of the node to fetch from

storeName - Name of the store

replicaToPartitionList - Mapping of replica type to partition list

filter - Custom filter implementation to filter out entries which should not be fetched.

fetchMasterEntries - Fetch an entry only if master replica

initialCluster - The cluster metadata to use while making the decision to fetch entries. This is important during rebalancing where-in we want to fetch keys using an older metadata compared to the new one.

skipRecords - Number of records to skip

Returns:

An iterator which allows entries to be streamed as they're being iterated over.

fetchOrphanedKeys

public java.util.Iterator<ByteArray> fetchOrphanedKeys(int nodeId,
                                                       java.lang.String storeName)

Fetch all the keys on the node that don't belong to it, based on its current metadata and yet stored on the node. i.e all keys orphaned on the node due to say not running the repair job after a rebalance

Parameters:

nodeId - Id of the node to fetch from

storeName - Name of the store

Returns:

An iterator which allows keys to be streamed as they're being iterated over.

fetchKeys

public java.util.Iterator<ByteArray> fetchKeys(int nodeId,
                                               java.lang.String storeName,
                                               java.util.List<java.lang.Integer> partitionList,
                                               VoldemortFilter filter,
                                               boolean fetchMasterEntries,
                                               long skipRecords)

Legacy interface for fetching entries. See AdminClient#fetchKeys(int, String, HashMap, VoldemortFilter, boolean, Cluster, long) for more information.

Parameters:

nodeId - Id of the node to fetch from

storeName - Name of the store

partitionList - List of the partitions to retrieve

filter - Custom filter implementation to filter out entries which should not be fetched.

fetchMasterEntries - Fetch a key only if master replica

skipRecords - Number of keys to skip

Returns:

An iterator which allows keys to be streamed as they're being iterated over.

fetchKeys

public java.util.Iterator<ByteArray> fetchKeys(int nodeId,
                                               java.lang.String storeName,
                                               java.util.List<java.lang.Integer> partitionList,
                                               VoldemortFilter filter,
                                               boolean fetchMasterEntries)

Legacy interface for fetching entries. See AdminClient#fetchKeys(int, String, HashMap, VoldemortFilter, boolean, Cluster, long) for more information.

Parameters:

nodeId - Id of the node to fetch from

storeName - Name of the store

partitionList - List of the partitions to retrieve

filter - Custom filter implementation to filter out entries which should not be fetched.

fetchMasterEntries - Fetch a key only if master replica

Returns:

An iterator which allows keys to be streamed as they're being iterated over.

fetchKeys

public java.util.Iterator<ByteArray> fetchKeys(int nodeId,
                                               java.lang.String storeName,
                                               java.util.HashMap<java.lang.Integer,java.util.List<java.lang.Integer>> replicaToPartitionList,
                                               VoldemortFilter filter,
                                               boolean fetchMasterEntries,
                                               Cluster initialCluster,
                                               long skipRecords)

Fetch all keys belonging to the map of replica type to partition list. Identical to AdminClient#fetchEntries but only fetches the keys

Parameters:

nodeId - The node id from where to fetch the keys

storeName - The store name whose keys we want to retrieve

replicaToPartitionList - Map of replica type to corresponding partition list

filter - Custom filter

initialCluster - Cluster to use for selecting a key. If null, use the default metadata from the metadata store

skipRecords - Number of records to skip [ Used for sampling ]

Returns:

Returns an iterator of the keys