Voldemort Admin Tool Reference
New Voldemort Admin Tool
This page shows all available admin tool commands and their synopses.
To see the options corresponding to each command, please use help command.
1. How to show help menus or command-line usages
The new voldemort admin tool has 3 levels of help menus.
The top level help menu shows a list of all group commands.
Type
./bin/vadmin.sh
to see it.
The group help menu shows a list of sub-commands within this command group.
Type
./bin/vadmin.sh <group-name>
to see it, e.g.
./bin/vadmin.sh quota
.
The sub-command help info shows description, synopsis and options for the sub-command.
Type
./bin/vadmin.sh <group-name> <sub-command> -h
to see it, e.g.
./bin/vadmin.sh quota get -h
.
2. Command Groups
Group Name | Description |
---|---|
async-job | Show or stop async jobs. |
cleanup | Cleanup orphaned data, vector clocks or slops. |
debug | Check keys or routing plan. |
meta | Get, set, clear metadata; check or sync metadata version. |
quota | Get, set, unset quota or reserve memory on stores. |
store | Add, delete, rollback, truncate stores, or truncate partitions. |
stream | Fetch keys or entries, or mirror data across nodes, or update entries. |
To see help info for each group command, please type one of the following:
./bin/vadmin.sh <group-name>
./bin/vadmin.sh help <group-name>
./bin/vadmin.sh <group-name> -h
./bin/vadmin.sh <group-name> --help
3. Non-grouped Commands
Command | Description |
---|---|
native-backup | Backup bdb data on one single node natively. |
restore-from-replica | Restore data from peer replication. |
Synopsis
1) Backup bdb data on one single node natively (this op needs confirmation)
./bin/vadmin.sh native-backup -d <backup-dir> -n <node-id>
-s <store-name> -u <url> [--timeout <time-minute>]
[--incremental] [--verify] [--confirm]
2) Restore data on one single node from its peer replication (this op needs confirmation)
./bin/vadmin.sh restore-from-replica -n <node-id> -u <url>
-z <zone-id> [--parallel <num>] [--confirm]
4. Async-job Commands
Command | Description |
---|---|
list | Get async job list from nodes. |
stop | Stop async jobs on one node. |
Synopsis
1) Get a list of async jobs on some nodes or all nodes
./bin/vadmin.sh async-job list -u <url>
[-n <node-id-list> | --all-nodes]
2) Stop a list of async jobs on a particular node (this op needs confirmation)
./bin/vadmin.sh async-job stop <job-id-list> -n <node-id> -u <url>
[--confirm]
4. Cleanup Commands
Command | Description |
---|---|
orphaned-data | Remove orphaned data on a single node after rebalancing is done. |
vector-clocks | Prune data resulting from versioned puts during rebalancing. |
slops | Purge slops. |
Synopsis
1) Remove orphaned data on some nodes or all nodes, after rebalancing is done (this op needs confirmation)
./bin/vadmin.sh cleanup orphaned-data -u <url>
[-n <node-id-list> | --all-nodes] [--confirm]
2) Prune data of stores resulting from versioned puts during rebalancing, on some nodes or all nodes (this op needs confirmation)
./bin/vadmin.sh cleanup vector-clocks -s <store-name-list> -url <url>
[-n <node-id-list> | --all-nodes] [–confirm]
3) Purge slops of stores in a zone, on some nodes or all nodes (this op needs confirmation)
./bin/vadmin.sh cleanup slops -s <store-name-list> -u <url> -z <zone-id>
[-n <node-id-list> | --all-nodes] [--confirm]
5. Meta Commands
Command | Description |
---|---|
check | Check if metadata is consistent across all nodes. |
get | Get metadata from nodes. |
set | Set metadata on nodes. |
clear-rebalance | Remove metadata related to rebalancing. |
check-version | Verify metadata versions on all the cluster nodes. |
sync-version | Synchronize metadata versions across all nodes. |
Valid Metadata Keys
cluster.xml
stores.xml
server.state
node.id
rebalancing.steal.info.key
rebalancing.source.cluster.xml
rebalancing.source.stores.xml
Synopsis
1) Check if metadata is consistent across all nodes
./bin/vadmin.sh meta check (<meta-key-list> | all) -u <url>
2) Get metadata from some nodes or all nodes, optional output to directory, non-verbose by default
./bin/vadmin.sh meta get (<meta-key-list> | all) -u <url>
[-d <output-dir>] [-n <node-id-list> | --all-nodes] [--verbose]
3) Set one metadata on some nodes or all nodes (this op needs confirmation)
./bin/vadmin.sh meta set <meta-key>=<meta-file> -u <url>
[-n <node-id-list> | --all-nodes] [–confirm]
4) Set <cluster.xml, stores.xml> metadata pair on some nodes or all nodes (this op needs confirmation)
./bin/vadmin.sh meta set cluster.xml=<cluster-xml-file>,stores.xml=<stores-xml-file>
-u <url> [-n <node-id-list> | --all-nodes] [–confirm]
./bin/vadmin.sh meta set stores.xml=<stores-xml-file>,cluster.xml=<cluster-xml-file>
-u <url> [-n <node-id-list> | --all-nodes] [–confirm]
5) Remove metadata related to rebalancing on some nodes or all nodes (this op needs confirmation)
./bin/vadmin.sh meta clear-rebalance -u <url>
[-n <node-id-list> | --all-nodes] [--confirm]
6) Verify metadata versions on all nodes
./bin/vadmin.sh meta check-version -u <url>
7) Synchronize metadata versions across all nodes (this op needs confirmation)
./bin/vadmin.sh meta sync-version -n <base-node-id> -u <url>
[--confirm]
6. Quota Commands
Command | Description |
---|---|
get | Get quota values of stores. |
set | Set quota values for stores. |
unset | Clear quota settings for stores. |
reserve-memory | Reserve memory for stores. |
Valid Quota Types
DELETE_THROUGHPUT
GET_THROUGHPUT
GET_ALL_THROUGHPUT
PUT_THROUGHPUT
Synopsis
1) Get quota values of given stores
./bin/vadmin.sh quota get (<quota-type-list> | all) -s <store-name-list>
-u <url>
2) Set quota values for given stores (this op needs confirmation)
./bin/vadmin.sh quota set (<quota-type1>=<quota-value1>,...)
-s <store-name-list> -u <url> [--confirm]
3) Clear quota settings for given stores (this op needs confirmation)
./bin/vadmin.sh quota unset (<quota-type-list> | all) -s <store-name-list>
-u <url> [--confirm]
4) Reserve memory for given stores on some nodes or all nodes (this op needs confirmation)
./bin/vadmin.sh quota reserve-memory <memory-size> -s <store-name-list> -u <url>
[-n <node-id-list> | --all-nodes] [--confirm]
7. Store Commands
Command | Description |
---|---|
add | Add stores from a 'stores.xml' file. |
delete | Delete stores. |
truncate-partition | Remove contents of partitions on a node. |
truncate-store | Remove contents of stores. |
rollback-ro | Rollback read-only store to a given version. |
Synopsis
1) Add several stores from a 'stores.xml' file, on some nodes or all nodes
./bin/vadmin.sh store add -f <stores.xml-file-path> -u <url>
[-n <node-id-list> | --all-nodes]
2) Delete stores (this op needs confirmation)
./bin/vadmin.sh store delete -s <store-name-list> -u <url>
[-n <node-id-list> | --all-nodes] [--confirm]
3) Remove contents of partitions on a node (this op needs confirmation)
./bin/vadmin.sh store truncate-partition -p <partition-id-list>
-n <node-id> (-s <store-name-list> | --all-stores)
-u <url> [--confirm]
4) Remove contents of stores (this op needs confirmation)
./bin/vadmin.sh store truncate-store -s <store-name-list> -u <url>
[-n <node-id-list> | --all-nodes] [--confirm]
5) Rollback read-only store to a given version (this op needs confirmation)
./bin/vadmin.sh store rollback-ro -s <store-name> -u <url>
--version <store-version> [-n <node-id-list> | --all-nodes] [--confirm]
8. Stream Commands
Command | Description |
---|---|
fetch-keys | Fetch keys from a node. |
fetch-entries | Fetch entries from a node. |
update-entries | Update entries from file to a node. |
mirror | Mirror data from a node to another. |
Synopsis
1) Fetch keys of some partitions or all partitions, or orphaned keys, some stores or all store, on a given node, in format of json (by default) or hex
./bin/vadmin.sh stream fetch-keys -n <node-id>
(-p <partition-id-list> | --all-partitions | --orphaned)
(-s <store-name-list> | --all-stores) -u <url>
[-d <output-dir>] [--format json | hex]
2) Fetch entries of some partitions or all partitions, or orphaned entries, some stores or all store, on a given node, in format of json (by default) or hex
./bin/vadmin.sh stream fetch-entries -n <node-id>
(-p <partition-id-list> | --all-partitions | --orphaned)
(-s <store-name-list> | --all-stores) -u <url>
[-d <output-dir>] [--format json | hex]
3) Update entries from files to a node; if store names are not specified, then all entries of the files will be updated; otherwise only the entries of the specified stores will be updated (this op needs confirmation)
./bin/vadmin.sh stream update-entries -d <input-dir> -n <node-id>
-u <url> [-s <store-name-list>] [--confirm]
4) Mirror data from a node to another, for some stores or all stores (this op needs confirmation)
./bin/vadmin.sh stream mirror --from-url <src-url> --from-node <src-node-id>
--to-url <dest-url> --to-node <dest-node-id>
(-s <store-name-list> | --all-stores) [--confirm]
Upgrade from Old Admin Tool to New Admin Tool
Old Admin Command | New Command Group | New Admin Command |
---|---|---|
--native-backup | native-backup | |
--restore | restore-from-replica | |
--async get |
ASYNC-JOB
|
list |
--async stop | stop | |
--repair-job |
CLEANUP
|
orphaned-data |
--prune-job | vector-clocks | |
--purge-slops | slops | |
--query-key |
DEBUG
|
query-keys |
--show-routing-plan | route | |
--check-metadata |
META
|
check |
--clear-rebalancing-metadata | clear-rebalance | |
--get-metadata | get | |
--ro-metadata | (COMMAND REMOVED) | |
--set-metadata | set | |
--set-metadata-pair | set | |
--synchronize-metadata-version | sync-version | |
--verify-metadata-version | check-version | |
--get-quota |
QUOTA
|
get |
--set-quota | set | |
--reserve-memory | reserve-memory | |
--unset-quota | unset | |
--add-stores |
STORE
|
add |
--delete-store | delete | |
--rollback | rollback-ro | |
--delete-partitions | truncate-partition | |
--truncate | truncate-store | |
--fetch-keys |
STREAM
|
fetch-entries |
--fetch-entries | fetch-keys | |
--mirror-from-url | mirror | |
--update-entries | update-entries |
Old Voldemort Admin Tool
Commands supported
- Get / set metadata
- Get metadata from all nodes
./bin/voldemort-admin-tool.sh --get-metadata [metadata-key] --url [url]
- Get metadata from a particular node
./bin/voldemort-admin-tool.sh --get-metadata [metadata-key] --url [url] --node [node-id]
- Set metadata on all nodes [ metadata-value is path to xml files for
cluster.xml | stores.xml ]
./bin/voldemort-admin-tool.sh --set-metadata [metadata-key] --set-metadata-value [metadata-value] --url [url]
- Set metadata for a particular node
./bin/voldemort-admin-tool.sh --set-metadata [metadata-key] --set-metadata-value [metadata-value] --url [url] --node [node-id]
- Check if metadata is same on all nodes
./bin/voldemort-admin-tool.sh --check-metadata [metadata-key] --url [url]
- Get metadata from all nodes
- Add / delete stores
- Add store(s) on all nodes
./bin/voldemort-admin-tool.sh --add-stores [xml file with store(s) to add] --url [url]
- Add store(s) on single node
./bin/voldemort-admin-tool.sh --add-stores [xml file with store(s) to add] --url [url] --node [node-id]
- Delete store on all nodes
./bin/voldemort-admin-tool.sh --delete-store [store-name] --url [url]
- Delete the contents of the store on all nodes
./bin/voldemort-admin-tool.sh --truncate [store-name] --url [url]
- Delete the contents of the store on a single node
./bin/voldemort-admin-tool.sh --truncate [store-name] --url [url] --node [node-id]
- Delete the contents of some partitions on a single node
./bin/voldemort-admin-tool.sh --delete-partitions [comma-separated list of partitions] --url [url] --node [node-id]
- Delete the contents of some partitions ( of some stores ) on a single
node
./bin/voldemort-admin-tool.sh --delete-partitions [comma-separated list of partitions] --url [url] --node [node-id] --stores [comma-separated list of store names]
- Add store(s) on all nodes
- Stream data
- Fetch keys from a set of partitions [ all stores ] on a node ( binary
dump )
./bin/voldemort-admin-tool.sh --fetch-keys [comma-separated list of partitions with no space] --url [url] --node [node-id]
- Fetch keys from a set of partitions [ all stores ] on a node ( ascii
enabled )
./bin/voldemort-admin-tool.sh --fetch-keys [comma-separated list of partitions with no space] --url [url] --node [node-id] --ascii
- Fetch entries from a set of partitions [ all stores ] on a node ( binary
dump )
./bin/voldemort-admin-tool.sh --fetch-entries [comma-separated list of partitions with no space] --url [url] --node [node-id]
- Fetch entries from a set of partitions [ all stores ] on a node ( ascii
enabled )
./bin/voldemort-admin-tool.sh --fetch-entries [comma-separated list of partitions with no space] --url [url] --node [node-id] --ascii
- Fetch entries from a set of partitions [ all stores ] on a node ( ascii
enabled ) and output to a folder
./bin/voldemort-admin-tool.sh --fetch-entries [comma-separated list of partitions with no space] --url [url] --node [node-id] --ascii --outdir [directory]
- Fetch entries from a set of partitions and some stores on a node (
ascii enabled )
./bin/voldemort-admin-tool.sh --fetch-entries [comma-separated list of partitions with no space] --url [url] --node [node-id] --ascii --stores [comma-separated list of store names]
- Update entries for a set of stores using the output from a binary dump
fetch entries
./bin/voldemort-admin-tool.sh --update-entries [folder path from output of --fetch-entries --outdir] --url [url] --node [node-id] --stores [comma-separated list of store names]
- Fetch keys from a set of partitions [ all stores ] on a node ( binary
dump )
- Read-only operations
- Retrieve metadata information of read-only data for a particular node
and all stores
./bin/voldemort-admin-tool.sh --ro-metadata [current | max | storage-format] --url [url] --node [node-id]
- Retrieve metadata information of read-only data for all nodes and a
set of store
./bin/voldemort-admin-tool.sh --ro-metadata [current | max | storage-format] --url [url] --stores [comma-separated list of store names]
- Retrieve metadata information of read-only data for a particular node
and all stores
- Others
- Restore a particular node completely from its replicas
./bin/voldemort-admin-tool.sh --restore --url [url] --node [node-id]
- Restore a particular node completely from its replicas ( with increased
parallelism - 10 )
./bin/voldemort-admin-tool.sh --restore 10 --url [url] --node [node-id]
- Generates the key distribution on a per node basis [ both store wise
and overall ]
./bin/voldemort-admin-tool.sh --key-distribution --url [url] --node [node-id] --ascii
- Restore a particular node completely from its replicas