DMTN-112: LSST DM Vault

  • Adam Thornton

Latest Revision: 2019-03-06

1   Location and basic usage

LSST Data Management maintains an instance of Hashicorp’s Vault product at . To use Vault, set VAULT_ADDR to, and set the appropriate token (described later) in the environment variable VAULT_TOKEN. Then use the Vault CLI (or other client of your choice) to access secrets under the structure allowed for your token.

Note that we are using Version 2 of the KV Secret Engine (allowing versioned secrets), and therefore you must read and write secrets with vault kv get or vault kv put rather than simple vault read or vault write commands.

2   Vault Implementation

This vault implementation is simply Seth Vargo’s Vault On GKE with the TLS certificates replaced with actual certificates rather than self-signed ones. It is running on a Kubernetes Cluster in the us-central1 region. Members of the LSST DM SQuaRE team have access to the credentials necessary to administer the cluster.

3   Taxonomy

Our intention is for Vault to be organized with secret paths under the top-level secret store (secret/) as follows:


As an example, secrets for the instance of the LSST Science Platform Notebook Aspect are stored in secret/dm/square/nublado/ Within that secret path are hub, proxy, and tls folders, which each contain a number of individual secrets, e.g. secret/dm/square/nublado/

4   Tokens

Each secret path will have two tokens created: read and write. The read token can view but not alter data, while the write token can create, update, or delete data within the secret path.

5   Token Acquisition and Revocation

To acquire a token pair, ask Adam Thornton ( (or any other SQuaRE personnel) to create a pair for your desired secret path. Assuming that the path is correctly-structured, he will give you a pair of tokens (both id and accessor) for reading and writing to the secret path.

To revoke or renew these tokens, indicate to Adam (or another SQuaRE team member) what path you want revoked or renewed, and (if revocation) whether the corresponding data should be deleted as well.

6   Administrative Tools

Administrative tools for working with Vault can be found at the LSST Vault Utils GitHub Repository or by pip-installing lsstvaultutils.

Without an administrative token (limited to SQuaRE team members), you will not be able to use the tokenadmin tool, which is the tool by which token issuance and revocation is managed.

However, with read and write tokens you can use copyk2v and copyv2k to copy secrets back and forth between Kubernetes secrets and the Vault implementation.

With a write token you will also be able to use vaultrmrf, which is exactly as dangerous as it sounds.