Voldemort Admin Tool Reference

New Voldemort Admin Tool

This page shows all available admin tool commands and their synopses.

To see the options corresponding to each command, please use help command.

1. How to show help menus or command-line usages

The new voldemort admin tool has 3 levels of help menus.

The top level help menu shows a list of all group commands.

Type ./bin/vadmin.sh to see it.

The group help menu shows a list of sub-commands within this command group.

Type ./bin/vadmin.sh <group-name> to see it, e.g. ./bin/vadmin.sh quota .

The sub-command help info shows description, synopsis and options for the sub-command.

Type ./bin/vadmin.sh <group-name> <sub-command> -h to see it, e.g. ./bin/vadmin.sh quota get -h .

2. Command Groups

Group Name Description
async-job Show or stop async jobs.
cleanup Cleanup orphaned data, vector clocks or slops.
debug Check keys or routing plan.
meta Get, set, clear metadata; check or sync metadata version.
quota Get, set, unset quota or reserve memory on stores.
store Add, delete, rollback, truncate stores, or truncate partitions.
stream Fetch keys or entries, or mirror data across nodes, or update entries.

To see help info for each group command, please type one of the following:

./bin/vadmin.sh <group-name>

./bin/vadmin.sh help <group-name>

./bin/vadmin.sh <group-name> -h

./bin/vadmin.sh <group-name> --help

3. Non-grouped Commands

Command Description
native-backup Backup bdb data on one single node natively.
restore-from-replica Restore data from peer replication.

Synopsis

1) Backup bdb data on one single node natively (this op needs confirmation)

./bin/vadmin.sh native-backup -d <backup-dir> -n <node-id> -s <store-name> -u <url> [--timeout <time-minute>] [--incremental] [--verify] [--confirm]

2) Restore data on one single node from its peer replication (this op needs confirmation)

./bin/vadmin.sh restore-from-replica -n <node-id> -u <url> -z <zone-id> [--parallel <num>] [--confirm]

4. Async-job Commands

Command Description
list Get async job list from nodes.
stop Stop async jobs on one node.

Synopsis

1) Get a list of async jobs on some nodes or all nodes

./bin/vadmin.sh async-job list -u <url> [-n <node-id-list> | --all-nodes]

2) Stop a list of async jobs on a particular node (this op needs confirmation)

./bin/vadmin.sh async-job stop <job-id-list> -n <node-id> -u <url> [--confirm]

4. Cleanup Commands

Command Description
orphaned-data Remove orphaned data on a single node after rebalancing is done.
vector-clocks Prune data resulting from versioned puts during rebalancing.
slops Purge slops.

Synopsis

1) Remove orphaned data on some nodes or all nodes, after rebalancing is done (this op needs confirmation)

./bin/vadmin.sh cleanup orphaned-data -u <url> [-n <node-id-list> | --all-nodes] [--confirm]

2) Prune data of stores resulting from versioned puts during rebalancing, on some nodes or all nodes (this op needs confirmation)

./bin/vadmin.sh cleanup vector-clocks -s <store-name-list> -url <url> [-n <node-id-list> | --all-nodes] [–confirm]

3) Purge slops of stores in a zone, on some nodes or all nodes (this op needs confirmation)

./bin/vadmin.sh cleanup slops -s <store-name-list> -u <url> -z <zone-id> [-n <node-id-list> | --all-nodes] [--confirm]

5. Meta Commands

Command Description
check Check if metadata is consistent across all nodes.
get Get metadata from nodes.
set Set metadata on nodes.
clear-rebalance Remove metadata related to rebalancing.
check-version Verify metadata versions on all the cluster nodes.
sync-version Synchronize metadata versions across all nodes.

Valid Metadata Keys

cluster.xml

stores.xml

server.state

node.id

rebalancing.steal.info.key

rebalancing.source.cluster.xml

rebalancing.source.stores.xml

Synopsis

1) Check if metadata is consistent across all nodes

./bin/vadmin.sh meta check (<meta-key-list> | all) -u <url>

2) Get metadata from some nodes or all nodes, optional output to directory, non-verbose by default

./bin/vadmin.sh meta get (<meta-key-list> | all) -u <url> [-d <output-dir>] [-n <node-id-list> | --all-nodes] [--verbose]

3) Set one metadata on some nodes or all nodes (this op needs confirmation)

./bin/vadmin.sh meta set <meta-key>=<meta-file> -u <url> [-n <node-id-list> | --all-nodes] [–confirm]

4) Set <cluster.xml, stores.xml> metadata pair on some nodes or all nodes (this op needs confirmation)

./bin/vadmin.sh meta set cluster.xml=<cluster-xml-file>,stores.xml=<stores-xml-file> -u <url> [-n <node-id-list> | --all-nodes] [–confirm]

./bin/vadmin.sh meta set stores.xml=<stores-xml-file>,cluster.xml=<cluster-xml-file> -u <url> [-n <node-id-list> | --all-nodes] [–confirm]

5) Remove metadata related to rebalancing on some nodes or all nodes (this op needs confirmation)

./bin/vadmin.sh meta clear-rebalance -u <url> [-n <node-id-list> | --all-nodes] [--confirm]

6) Verify metadata versions on all nodes

./bin/vadmin.sh meta check-version -u <url>

7) Synchronize metadata versions across all nodes (this op needs confirmation)

./bin/vadmin.sh meta sync-version -n <base-node-id> -u <url> [--confirm]

6. Quota Commands

Command Description
get Get quota values of stores.
set Set quota values for stores.
unset Clear quota settings for stores.
reserve-memory Reserve memory for stores.

Valid Quota Types

DELETE_THROUGHPUT

GET_THROUGHPUT

GET_ALL_THROUGHPUT

PUT_THROUGHPUT

Synopsis

1) Get quota values of given stores

./bin/vadmin.sh quota get (<quota-type-list> | all) -s <store-name-list> -u <url>

2) Set quota values for given stores (this op needs confirmation)

./bin/vadmin.sh quota set (<quota-type1>=<quota-value1>,...) -s <store-name-list> -u <url> [--confirm]

3) Clear quota settings for given stores (this op needs confirmation)

./bin/vadmin.sh quota unset (<quota-type-list> | all) -s <store-name-list> -u <url> [--confirm]

4) Reserve memory for given stores on some nodes or all nodes (this op needs confirmation)

./bin/vadmin.sh quota reserve-memory <memory-size> -s <store-name-list> -u <url> [-n <node-id-list> | --all-nodes] [--confirm]

7. Store Commands

Command Description
add Add stores from a 'stores.xml' file.
delete Delete stores.
truncate-partition Remove contents of partitions on a node.
truncate-store Remove contents of stores.
rollback-ro Rollback read-only store to a given version.

Synopsis

1) Add several stores from a 'stores.xml' file, on some nodes or all nodes

./bin/vadmin.sh store add -f <stores.xml-file-path> -u <url> [-n <node-id-list> | --all-nodes]

2) Delete stores (this op needs confirmation)

./bin/vadmin.sh store delete -s <store-name-list> -u <url> [-n <node-id-list> | --all-nodes] [--confirm]

3) Remove contents of partitions on a node (this op needs confirmation)

./bin/vadmin.sh store truncate-partition -p <partition-id-list> -n <node-id> (-s <store-name-list> | --all-stores) -u <url> [--confirm]

4) Remove contents of stores (this op needs confirmation)

./bin/vadmin.sh store truncate-store -s <store-name-list> -u <url> [-n <node-id-list> | --all-nodes] [--confirm]

5) Rollback read-only store to a given version (this op needs confirmation)

./bin/vadmin.sh store rollback-ro -s <store-name> -u <url> --version <store-version> [-n <node-id-list> | --all-nodes] [--confirm]

8. Stream Commands

Command Description
fetch-keys Fetch keys from a node.
fetch-entries Fetch entries from a node.
update-entries Update entries from file to a node.
mirror Mirror data from a node to another.

Synopsis

1) Fetch keys of some partitions or all partitions, or orphaned keys, some stores or all store, on a given node, in format of json (by default) or hex

./bin/vadmin.sh stream fetch-keys -n <node-id> (-p <partition-id-list> | --all-partitions | --orphaned) (-s <store-name-list> | --all-stores) -u <url> [-d <output-dir>] [--format json | hex]

2) Fetch entries of some partitions or all partitions, or orphaned entries, some stores or all store, on a given node, in format of json (by default) or hex

./bin/vadmin.sh stream fetch-entries -n <node-id> (-p <partition-id-list> | --all-partitions | --orphaned) (-s <store-name-list> | --all-stores) -u <url> [-d <output-dir>] [--format json | hex]

3) Update entries from files to a node; if store names are not specified, then all entries of the files will be updated; otherwise only the entries of the specified stores will be updated (this op needs confirmation)

./bin/vadmin.sh stream update-entries -d <input-dir> -n <node-id> -u <url> [-s <store-name-list>] [--confirm]

4) Mirror data from a node to another, for some stores or all stores (this op needs confirmation)

./bin/vadmin.sh stream mirror --from-url <src-url> --from-node <src-node-id> --to-url <dest-url> --to-node <dest-node-id> (-s <store-name-list> | --all-stores) [--confirm]


Upgrade from Old Admin Tool to New Admin Tool

Old Admin Command New Command Group New Admin Command
--native-backup native-backup
--restore restore-from-replica
--async get
ASYNC-JOB
 list
--async stop stop
--repair-job
CLEANUP
 orphaned-data
--prune-job vector-clocks
--purge-slops slops
--query-key
DEBUG
 query-keys
--show-routing-plan route
--check-metadata
META
 check
--clear-rebalancing-metadata clear-rebalance
--get-metadata get
--ro-metadata (COMMAND REMOVED)
--set-metadata set
--set-metadata-pair set
--synchronize-metadata-version sync-version
--verify-metadata-version check-version
--get-quota
QUOTA
 get
--set-quota set
--reserve-memory reserve-memory
--unset-quota unset
--add-stores
STORE
 add
--delete-store delete
--rollback rollback-ro
--delete-partitions truncate-partition
--truncate truncate-store
--fetch-keys
STREAM
 fetch-entries
--fetch-entries fetch-keys
--mirror-from-url mirror
--update-entries update-entries


Old Voldemort Admin Tool

Commands supported

  1. Get / set metadata
    1. Get metadata from all nodes 
      ./bin/voldemort-admin-tool.sh --get-metadata [metadata-key] --url [url]
    2. Get metadata from a particular node 
      ./bin/voldemort-admin-tool.sh --get-metadata [metadata-key] --url [url]
--node [node-id]
    3. Set metadata on all nodes [ metadata-value is path to xml files for cluster.xml | stores.xml ] 
      ./bin/voldemort-admin-tool.sh --set-metadata [metadata-key] --set-metadata-value
[metadata-value] --url [url]
    4. Set metadata for a particular node 
      ./bin/voldemort-admin-tool.sh --set-metadata [metadata-key] --set-metadata-value
[metadata-value] --url [url] --node [node-id]
    5. Check if metadata is same on all nodes 
      ./bin/voldemort-admin-tool.sh --check-metadata [metadata-key] --url [url]
  2. Add / delete stores
    1. Add store(s) on all nodes 
      ./bin/voldemort-admin-tool.sh --add-stores [xml file with store(s) to
add] --url [url]
    2. Add store(s) on single node 
      ./bin/voldemort-admin-tool.sh --add-stores [xml file with store(s) to
add] --url [url] --node [node-id]
    3. Delete store on all nodes 
      ./bin/voldemort-admin-tool.sh --delete-store [store-name] --url [url]
    4. Delete the contents of the store on all nodes 
      ./bin/voldemort-admin-tool.sh --truncate [store-name] --url [url]
    5. Delete the contents of the store on a single node 
      ./bin/voldemort-admin-tool.sh --truncate [store-name] --url [url] --node
[node-id]
    6. Delete the contents of some partitions on a single node 
      ./bin/voldemort-admin-tool.sh --delete-partitions [comma-separated list
of partitions] --url [url] --node [node-id]
    7. Delete the contents of some partitions ( of some stores ) on a single node 
      ./bin/voldemort-admin-tool.sh --delete-partitions [comma-separated list
of partitions] --url [url] --node [node-id] --stores [comma-separated list
of store names]
  3. Stream data
    1. Fetch keys from a set of partitions [ all stores ] on a node ( binary dump ) 
      ./bin/voldemort-admin-tool.sh --fetch-keys [comma-separated list of partitions
with no space] --url [url] --node [node-id]
    2. Fetch keys from a set of partitions [ all stores ] on a node ( ascii enabled ) 
      ./bin/voldemort-admin-tool.sh --fetch-keys [comma-separated list of partitions
with no space] --url [url] --node [node-id] --ascii
    3. Fetch entries from a set of partitions [ all stores ] on a node ( binary dump ) 
      ./bin/voldemort-admin-tool.sh --fetch-entries [comma-separated list of
partitions with no space] --url [url] --node [node-id]
    4. Fetch entries from a set of partitions [ all stores ] on a node ( ascii enabled ) 
      ./bin/voldemort-admin-tool.sh --fetch-entries [comma-separated list of
partitions with no space] --url [url] --node [node-id] --ascii
    5. Fetch entries from a set of partitions [ all stores ] on a node ( ascii enabled ) and output to a folder 
      ./bin/voldemort-admin-tool.sh --fetch-entries [comma-separated list of
partitions with no space] --url [url] --node [node-id] --ascii --outdir
[directory]
    6. Fetch entries from a set of partitions and some stores on a node ( ascii enabled ) 
      ./bin/voldemort-admin-tool.sh --fetch-entries [comma-separated list of
partitions with no space] --url [url] --node [node-id] --ascii --stores
[comma-separated list of store names]
    7. Update entries for a set of stores using the output from a binary dump fetch entries 
      ./bin/voldemort-admin-tool.sh --update-entries [folder path from output
of --fetch-entries --outdir] --url [url] --node [node-id] --stores [comma-separated
list of store names]
  4. Read-only operations
    1. Retrieve metadata information of read-only data for a particular node and all stores 
      ./bin/voldemort-admin-tool.sh --ro-metadata [current | max | storage-format]
--url [url] --node [node-id]
    2. Retrieve metadata information of read-only data for all nodes and a set of store 
      ./bin/voldemort-admin-tool.sh --ro-metadata [current | max | storage-format]
--url [url] --stores [comma-separated list of store names]
  5. Others
    1. Restore a particular node completely from its replicas 
      ./bin/voldemort-admin-tool.sh --restore --url [url] --node [node-id]
    2. Restore a particular node completely from its replicas ( with increased parallelism - 10 ) 
      ./bin/voldemort-admin-tool.sh --restore 10 --url [url] --node [node-id]
    3. Generates the key distribution on a per node basis [ both store wise and overall ] 
      ./bin/voldemort-admin-tool.sh --key-distribution --url [url] --node [node-id]
--ascii
Fork me on GitHub