Running Vault on Kubernetes is generally the same as running it anywhere else. Kubernetes, as a container orchestration engine, eases some of the operational burdens and Helm charts provide the benefit of a refined interface when it comes to deploying Vault in a variety of different modes.
In this tutorial, you will set up Vault and its dependencies with a Helm chart. You will then integrate a web application that uses the Kubernetes service account token to authenticate with Vault and retrieve a secret.
This tutorial requires the Kubernetes command-line interface (CLI) and the Helm CLI installed, minikube, the Vault Helm charts, the sample web application, and additional configuration to bring it all together.
First, follow the directions to install minikube, including VirtualBox or similar.
Next, install kubectl CLI and helm CLI.
NOTE: This tutorial was last tested in May 2022 on a macOS 12.2.1 using this configuration.
Docker version.
$ docker version
Client:
Cloud integration: v1.0.22
Version: 20.10.13
## ...
Display minikube version.
$ minikube version
minikube version: v1.25.2
commit: 362d5fdc0a3dbee389b3d3f1034e8023e72bd3a7
Helm version.
$ helm version
version.BuildInfo{Version:"v3.9.0", GitCommit:"7ceeda6c585217a19a1131663d8cd1f7d641b2a7", GitTreeState:"clean", GoVersion:"go1.18.2"}
These are recommended software versions and the output displayed may vary depending on your environment and the software versions you use.
Install kubectl with Homebrew.
$ brew install kubernetes-cli
Install helm with Homebrew.
Install kubectl with Chocolatey.
$ choco install kubernetes-cli
Install helm with Chocolatey.
$ choco install kubernetes-helm
minikube is a CLI tool that provisions and manages the lifecycle of single-node Kubernetes clusters running inside Virtual Machines (VM) on your local system.
Start a Kubernetes cluster.
$ minikube start
😄 minikube v1.25.2 on Darwin 12.2.1
✨ Automatically selected the docker driver. Other choices: hyperkit, virtualbox, ssh
👍 Starting control plane node minikube in cluster minikube
🚜 Pulling base image ...
🔥 Creating docker container (CPUs=2, Memory=8100MB) ...
🐳 Preparing Kubernetes v1.23.3 on Docker 20.10.12 ...
▪ kubelet.housekeeping-interval=5m
▪ Generating certificates and keys ...
▪ Booting up control plane ...
▪ Configuring RBAC rules ...
🔎 Verifying Kubernetes components...
▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
🌟 Enabled addons: storage-provisioner, default-storageclass
🏄 Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default
The initialization process takes several minutes as it retrieves any necessary dependencies and executes various container images.
Verify the status of the minikube cluster.
$ minikube status
minikube
type: Control Plane
host: Running
kubelet: Running
apiserver: Running
kubeconfig: Configured
Vault manages the secrets that are written to these mountable volumes. To provide these secrets a single Vault server is required. For this demonstration Vault can be run in development mode to automatically handle initialization, unsealing, and setup of a KV secrets engine.
Add the HashiCorp Helm repository.
$ helm repo add hashicorp https://helm.releases.hashicorp.com
"hashicorp" has been added to your repositories
Update all the repositories to ensure helm is aware of the latest versions.
$ helm repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "secrets-store-csi-driver" chart repository
...Successfully got an update from the "hashicorp" chart repository
Update Complete. ⎈Happy Helming!⎈
To verify, search repositories for vault in charts.
$ helm search repo hashicorp/vault
NAME CHART VERSION APP VERSION DESCRIPTION
hashicorp/vault 0.20.0 1.10.3 Official HashiCorp Vault Chart
Create a file named helm-vault-raft-values.yml with the following contents:
$ cat > helm-vault-raft-values.yml <<EOF
server:
affinity: ""
ha:
enabled: true
raft:
enabled: true
setNodeId: true
config: |
cluster_name = "vault-integrated-storage"
storage "raft" {
path = "/vault/data/"
}
listener "tcp" {
address = "[::]:8200"
cluster_address = "[::]:8201"
tls_disable = "true"
}
service_registration "kubernetes" {}
EOF
Recommendation
If you are using Prometheus for monitoring and alerting, we recommend to set the cluster_name in the HCL configuration. With the Vault Helm chart, this is accomplished with the config parameter.
Install the latest version of the Vault Helm chart with Integrated Storage.
$ helm install vault hashicorp/vault --values helm-vault-raft-values.yml
Example output:
NAME: vault
LAST DEPLOYED: Wed May 18 20:19:15 2022
NAMESPACE: default
STATUS: deployed
REVISION: 1
NOTES:
Thank you for installing HashiCorp Vault!
Now that you have deployed Vault, you should look over the docs on using
Vault with Kubernetes available here:
https://www.vaultproject.io/docs/
Your release is named vault. To learn more about the release, try:
$ helm status vault
$ helm get manifest vault
This creates three Vault server instances with an Integrated Storage (Raft) backend.
The Vault pods and Vault Agent Injector pod are deployed in the default namespace.
Display all the pods within the default namespace. This may take a moment.
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
vault-0 0/1 Running 0 2m12s
vault-1 0/1 Running 0 2m12s
vault-2 0/1 Running 0 2m12s
vault-agent-injector-56b65c5cd4-k7lbt 1/1 Running 0 2m13s
Initialize vault-0 with one key share and one key threshold.
$ kubectl exec vault-0 -- vault operator init \
-key-shares=1 \
-key-threshold=1 \
-format=json > cluster-keys.json
The operator init command generates a root key that it disassembles into key shares -key-shares=1 and then sets the number of key shares required to unseal Vault -key-threshold=1. These key shares are written to the output as unseal keys in JSON format -format=json. Here the output is redirected to a file named cluster-keys.json.
Display the unseal key found in cluster-keys.json.
$ jq -r ".unseal_keys_b64[]" cluster-keys.json
rrUtT32GztRy/pVWmcH0ZQLCCXon/TxCgi40FL1Zzus=
Insecure operation
Do not run an unsealed Vault in production with a single key share and a single key threshold. This approach is only used here to simplify the unsealing process for this demonstration.Create a variable named VAULT_UNSEAL_KEY to capture the Vault unseal key.
$ VAULT_UNSEAL_KEY=$(jq -r ".unseal_keys_b64[]" cluster-keys.json)
After initialization, Vault is configured to know where and how to access the storage, but does not know how to decrypt any of it. Unsealing is the process of constructing the root key necessary to read the decryption key to decrypt the data, allowing access to the Vault.
Unseal Vault running on the vault-0 pod.
$ kubectl exec vault-0 -- vault operator unseal $VAULT_UNSEAL_KEY
Insecure operation
Providing the unseal key with the command writes the key to your shell's history. This approach is only used here to simplify the unsealing process for this demonstration.The operator unseal command reports that Vault is initialized and unsealed.
Example output:
Key Value
--- -----
Seal Type shamir
Initialized true
Sealed false
Total Shares 1
Threshold 1
Version 1.15.2
Build Date 2023-11-06T11:33:28Z
Storage Type raft
Cluster Name vault-integrated-storage
Cluster ID f9762f89-fa6b-5420-1595-0a335513d7d1
HA Enabled true
HA Cluster n/a
HA Mode standby
Active Node Address <none>
Raft Committed Index 31
Raft Applied Index 31
The Vault server is initialized and unsealed.
Join the vault-1 pod to the Raft cluster.
$ kubectl exec -ti vault-1 -- vault operator raft join http://vault-0.vault-internal:8200
Join the vault-2 pod to the Raft cluster.
$ kubectl exec -ti vault-2 -- vault operator raft join http://vault-0.vault-internal:8200
Use the unseal key from above to unseal vault-1.
$ kubectl exec -ti vault-1 -- vault operator unseal $VAULT_UNSEAL_KEY
Unseal Key (will be hidden):
Key Value
--- -----
Seal Type shamir
Initialized true
Sealed true
Total Shares 1
Threshold 1
Unseal Progress 0/1
Unseal Nonce n/a
Version 1.10.3
Storage Type raft
HA Enabled true
Use the unseal key from above to unseal vault-2.
$ kubectl exec -ti vault-2 -- vault operator unseal $VAULT_UNSEAL_KEY
The web application that you deploy in the Launch a web application section, expects Vault to store a username and password at the path secret/webapp/config. To create this secret requires you to login with the root token, enable the key-value secret engine, and store a secret username and password at that defined path.
Vault generated an initial root token when it was initialized.
Display the root token found in cluster-keys.json.
$ jq -r ".root_token" cluster-keys.json
hvs.HJmsajgGlWPTx6YNHoUljuOO
Start an interactive shell session on the vault-0 pod.
$ kubectl exec --stdin=true --tty=true vault-0 -- /bin/sh
/ $
Your system prompt is replaced with a new prompt / $.
Vault is now ready for you to login with the initial root token.
Vault requires clients to authenticate first before it allows any further actions. An unsealed Vault starts with the Token Auth Method enabled and generates an initial root token.
Login with the root token when prompted.
$ vault login
Token (will be hidden):
Success! You are now authenticated. The token information displayed below
is already stored in the token helper. You do NOT need to run "vault login"
again. Future Vault requests will automatically use this token.
Key Value
--- -----
token hvs.HJmsajgGlWPTx6YNHoUljuOO
token_accessor JVsMJHVu6rTWbPLlYmWQTq1R
token_duration ∞
token_renewable false
token_policies ["root"]
identity_policies []
policies ["root"]
Enable an instance of the kv-v2 secrets engine at the path secret.
$ vault secrets enable -path=secret kv-v2
Success! Enabled the kv-v2 secrets engine at: secret/
Learn more
This tutorial focuses on Vault's integration with Kubernetes and not interacting with the key-value secrets engine. For more information refer to the Versioned Key/value secrets engine tutorial.
Create a secret at path secret/webapp/config with a username and password.
$ vault kv put secret/webapp/config username="static-user" password="static-password"
====== Secret Path ======
secret/data/webapp/config
======= Metadata =======
Key Value
--- -----
created_time 2022-06-07T05:15:19.402740412Z
custom_metadata <nil>
deletion_time n/a
destroyed false
version 1
Verify that the secret is defined at the path secret/webapp/config.
$ vault kv get secret/webapp/config
====== Secret Path ======
secret/data/webapp/config
======= Metadata =======
Key Value
--- -----
created_time 2022-06-07T05:15:19.402740412Z
custom_metadata <nil>
deletion_time n/a
destroyed false
version 1
====== Data ======
Key Value
--- -----
password static-password
username static-user
You successfully created the secret for the web application.
Exit the vault-0 pod.
The initial root token is a privileged user that can perform any operation at any path. The web application only requires the ability to read secrets defined at a single path. This application should authenticate and be granted a token with limited access.
We recommend that root tokens are used only for initial setup of an authentication method and policies. Afterwards they should be revoked. This tutorial does not show you how to revoke the root token.
Vault provides a Kubernetes authentication method that enables clients to authenticate with a Kubernetes Service Account Token.
Start an interactive shell session on the vault-0 pod.
$ kubectl exec --stdin=true --tty=true vault-0 -- /bin/sh
/ $
Your system prompt is replaced with a new prompt / $.
Enable the Kubernetes authentication method.
$ vault auth enable kubernetes
Success! Enabled kubernetes auth method at: kubernetes/
Vault accepts this service token from any client within the Kubernetes cluster. During authentication, Vault verifies that the service account token is valid by querying a configured Kubernetes endpoint.
Configure the Kubernetes authentication method to use the location of the Kubernetes API.
For the best compatibility with recent Kubernetes versions, ensure you are using Vault v1.9.3 or greater.
$ vault write auth/kubernetes/config \
kubernetes_host="https://$KUBERNETES_PORT_443_TCP_ADDR:443"
Successful output from this command resembles this example:
Success! Data written to: auth/kubernetes/config
The environment variable KUBERNETES_PORT_443_TCP_ADDR is defined and references the internal network address of the Kubernetes host.
For a client to access the secret data defined, at secret/webapp/config, requires that the read capability be granted for the path secret/data/webapp/config. This is an example of a policy. A policy defines a set of capabilities.
Write out the policy named webapp that enables the read capability for secrets at path secret/data/webapp/config.
$ vault policy write webapp - <<EOF
path "secret/data/webapp/config" {
capabilities = ["read"]
}
EOF
Successful output from this command resembles this example:
Success! Uploaded policy: webapp
Define an auth method role that uses the webapp policy. A role binds policies and environment parameters together to create a login for the web application.
Create a Kubernetes authentication role, named webapp, that connects the Kubernetes service account name and webapp policy.
$ vault write auth/kubernetes/role/webapp \
bound_service_account_names=vault \
bound_service_account_namespaces=default \
policies=webapp \
ttl=24h
Successful output from this command resembles this example:
Success! Data written to: auth/kubernetes/role/webapp
The role connects the Kubernetes service account, vault, and namespace, default, with the Vault policy, webapp. The tokens returned after authentication are valid for 24 hours.
Exit the vault-0 pod.
You have created a web application, published it to DockerHub, and created a Kubernetes deployment that will run the application in your existing cluster. The example web application performs the single function of listening for HTTP requests. During a request it reads the Kubernetes service token, logs into Vault, and then requests the secret.
Use your preferred text editor and review the contents of deployment-01-webapp.yml.
deployment-01-webapp.yml
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: webapp
labels:
app: webapp
spec:
replicas: 1
selector:
matchLabels:
app: webapp
template:
metadata:
labels:
app: webapp
spec:
serviceAccountName: vault
containers:
- name: app
image: hashieducation/simple-vault-client:latest
imagePullPolicy: Always
env:
- name: VAULT_ADDR
value: 'http://vault:8200'
- name: JWT_PATH
value: '/var/run/secrets/kubernetes.io/serviceaccount/token'
- name: SERVICE_PORT
value: '8080'
The web application deployment defines a list of environment variables.
JWT_PATH sets the path of the JSON web token (JWT) issued by Kubernetes. This token is used by the web application to authenticate with Vault.VAULT_ADDR sets the address of the Vault service. The Helm chart defined a Kubernetes service named vault that forwards requests to its endpoints (i.e. The pods named vault-0, vault-1, and vault-2).SERVICE_PORT sets the port that the service listens for incoming HTTP requests.Deploy the webapp in Kubernetes by applying the file deployment-01-webapp.yml.
$ kubectl apply --filename deployment-01-webapp.yml
deployment.apps/webapp created
The webapp runs as a pod within the default namespace.
Get all the pods within the default namespace.
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
vault-0 1/1 Running 0 15m
vault-1 1/1 Running 0 15m
vault-2 1/1 Running 0 15m
vault-agent-injector-659b4488df-4hhpz 1/1 Running 0 15m
webapp-784b8d9979-vn57d 0/1 ContainerCreating 0 47s
The webapp pod is displayed here as the pod prefixed with webapp.
The deployment of the service requires the retrieval of the web application container from Docker Hub. This displays the STATUS of ContainerCreating. The pod reports that it is not ready (0/1).
Wait until the webapp pod is running and ready (1/1).
The webapp pod runs an HTTP service that is listening on port 8080.
In another terminal, port forward all requests made to http://localhost:8080 to the webapp pod on port 8080.
$ kubectl port-forward \
$(kubectl get pod -l app=webapp -o jsonpath="{.items[0].metadata.name}") \
8080:8080
In the original terminal, perform a curl request at http://localhost:8080.
$ curl http://localhost:8080
password:static-secret username:static-user
The web application running on port 8080 in the webapp pod:
secret/data/webapp/config pathsecret/data/webapp/config pathFirst, stop the running local Kubernetes cluster.
This deactivates minikube, and all pods still exist at this point.
Delete the local Kubernetes cluster.
Be aware that minikube delete removes the minikube deployment including all pods. Be sure you want everything removed before continuing.
$ minikube delete
🔥 Deleting "minikube" in docker ...
🔥 Deleting container "minikube" ...
🔥 Removing /Users/mrken/.minikube/machines/minikube ...
💀 Removed all traces of the "minikube" cluster.
You launched Vault in high-availability mode with a Helm chart. Learn more about the Vault Helm chart by reading the documentation or exploring the project source code.
Also, you can run vault on Kubernetes on different platforms. Explore setting up Vault in high-availability with Integrated Storage in the Vault Installation to Google Kubernetes Engine via Helm tutorial.
Then you deployed a web application that authenticated and requested a secret directly from Vault. Explore how pods can retrieve secrets through the Vault Injector service via annotations or secrets mounted on ephemeral volumes.
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4