Vault
Google Cloud Platform (GCP) auth method
Before a client can interact with Vault, it must authenticate with an auth method to acquire a token. This token has policies attached so that the behavior of the client can be governed.
![Auth Concept](/_next/image?url=https%3A%2F%2Fcontent.hashicorp.com%2Fapi%2Fassets%3Fproduct%3Dtutorials%26version%3Dmain%26asset%3Dpublic%252Fimg%252Fvault%252Fdiagram-auth-workflow.png%26width%3D1436%26height%3D340&w=3840&q=75)
NOTE: To learn the basics of Vault tokens, go through the Tokens tutorial.
Auth methods perform authentication to verify the user or machine-supplied information. Some of the supported auth methods are targeted towards users while others are targeted toward machines or apps.
Challenge
Vault supports a number of auth methods for users or system to prove their identity so that a token with appropriate policies can be obtained.
Google Cloud Platform customers require a solution that supports multiple services for both users and compute services.
Solution
The gcp auth method allows Google Cloud Platform entities to authenticate to Vault. Vault treats Google Cloud as a trusted third party and verifies authenticating entities against the Google Cloud APIs.
Prerequisites
This tutorial assumes the following:
- You have a Google Cloud Platform account with permission to:
- Create IAM service accounts, policies
- Enable GCP APIs
- Access the GCP Cloud Shell
- Create VMs
- Vault installed on your local machine
- ngrok installed and configured with an auth token (required for Vault)
Lab setup
Workstation setup
Create a temporary directory to store the files used for this tutorial.
$ mkdir ~/TUTORIAL_TEMP
Retrieve the directory path and store it in the
TUTORIAL_TEMP
environment variable.$ TUTORIAL_TEMP=$(echo ~/TUTORIAL_TEMP)
Go to the
TUTORIAL_TEMP
directory.$ cd $TUTORIAL_TEMP
Vault setup
Note
If you do not have access to an HCP Vault Dedicated cluster, visit the Create a Vault Cluster on HCP tutorial.
Launch the HCP Portal and login.
Click Vault in the left navigation pane.
In the Vault clusters pane, click vault-cluster.
Under Cluster URLs, click Public Cluster URL.
In the same terminal you set the
TUTORIAL_TEMP
environment variable, set theVAULT_ADDR
environment variable to the copied address.$ export VAULT_ADDR=<Public_Cluster_URL>
Return to the Overview page and click Generate token.
Within a few moments, a new token will be generated.
Copy the Admin Token.
Return to the terminal and set the
VAULT_TOKEN
environment variable.$ export VAULT_TOKEN=<token>
Set the
VAULT_NAMESPACE
environment variable toadmin
.$ export VAULT_NAMESPACE=admin
The
admin
namespace is the top-level namespace automatically created by HCP Vault. All CLI operations default to use the namespace defined in this environment variable.Note
For these tasks, you can use HCP Vault Dedicated's admin token. However, it is recommended that admin tokens are only used for enough initial setup or in emergencies. As a best practice, use an authentication method or token that meets the policy requirements.
Enable the KV secrets engine.
$ vault secrets enable -version=2 -path=secret kv Success! Enabled the kv secrets engine at: secret/
The Vault Dedicated server is ready.
Configure GCP services for Vault
(Persona: admin)
Warning
Resources will be provisioned during this tutorial that may result in charges to your Google Cloud Platform account.
Before Vault can authenticate GCP IAM service accounts and GCE instances, you need to configure the necessary resources in GCP. This includes enabling the required GCP APIs, creating a IAM service account and IAM policy for Vault, and creating a key for the service account that Vault will use to authenticate with GCP.
Open a browser and sign into the GCP console.
Click the terminal icon to launch the GCP Cloud Shell.
A new pane will open at the bottom of your window and provision a Cloud Shell instance.
In the Cloud Shell terminal, list the GCP project ID.
Note
If this is the first time using the Cloud Shell terminal, you will be prompted to authorize permission to use your credentials.
Click
Authorize
when prompted.$ gcloud projects list PROJECT_ID: abc-123defde860044c59e853ee2819 NAME: my-gcp-account-name PROJECT_NUMBER: 85527753462
The project ID is used when configuring both GCP and Vault.
Verify the cloud resource manager and IAM API's are enabled.
$ gcloud services list --enabled | grep 'resource\|iam\|compute' NAME: cloudresourcemanager.googleapis.com NAME: compute.googleapis.com NAME: iam.googleapis.com NAME: iamcredentials.googleapis.com
If
cloudresourcemanager.googleapis.com
,compute.googleapis.com
, oriam.googleapis.com
are not listed, expand the menu below and enable the required API(s).Enable the
iam.googleapis.com
API.$ gcloud services enable iam.googleapis.com
Enable the
compute.googleapis.com
API.$ gcloud services enable compute.googleapis.com
Enable the
cloudresourcemanager.googleapis.com
API.$ gcloud services enable cloudresourcemanager.googleapis.com
Create a service account for Vault to authenticate with GCP.
$ gcloud iam service-accounts create \ VaultServiceAccount \ --display-name="VaultServiceAccount"
Create a GCP IAM role for Vault and assign the required permissions.
Note
The permissions assigned in this tutorial are for both IAM and GCE.
If you only require one of these services, remove the unused permissions.
$ gcloud iam roles create VaultServiceRole \ --project=$DEVSHELL_PROJECT_ID \ --title=VaultServiceRole \ --stage=GA \ --permissions=iam.serviceAccounts.get,iam.serviceAccountKeys.get,compute.instances.get,compute.instanceGroups.list,iam.serviceAccounts.signJwt
Note
The
iam.serviceAccounts.signJwt
permission is required for any service account used to authenticate with Vault using the IAM method. If you are not using the IAM method, you can remove this permission.Retrieve the role name and store it in the
ROLE_NAME
environment variable.$ ROLE_NAME=$(gcloud iam roles list --project=$DEVSHELL_PROJECT_ID --format=json --filter="vault" | jq -r .[].name)
Retrieve the service account email and store it in the
SERVICE_ACCOUNT
environment variable.$ SERVICE_ACCOUNT=$(gcloud iam service-accounts list --project=$DEVSHELL_PROJECT_ID --filter=vault --format=json | jq -r .[].email)
Add the Vault service account to the GCP project and bind the role to the service account.
$ gcloud projects add-iam-policy-binding $DEVSHELL_PROJECT_ID \ --member="serviceAccount:$SERVICE_ACCOUNT" \ --role="$ROLE_NAME"
Create a service account key credential file.
$ gcloud iam service-accounts keys create VaultServiceAccountKey.json \ --iam-account=$SERVICE_ACCOUNT \ --project=$DEVSHELL_PROJECT_ID
Click the More button (represented by the vertical ellipsis) and select Download.
Click the folder icon, expand the directory, select the VaultServiceAccountKey.json file, and click Download.
Store the
VaultServiceAccountKey.json
file inTUTORIAL_TEMP
directory you created in the lab setup section.Tip
Depending on your browser settings, the file may automatically download to a default directory. To complete this tutorial, make sure the
VaultServiceAccountKey.json
file is located in the$TUTORIAL_TEMP
directory.Example OSX command:
$ mv ~/Downloads/VaultServiceAccountKey.json $TUTORIAL_TEMP/
Retrieve and copy the GCP project ID from the Cloud Shell terminal.
$ echo $DEVSHELL_PROJECT_ID
Return to the terminal where you set the
TUTORIAL_TEMP
environment variable and set theGCP_PROJECT_ID
environment variable to the value from the previous step.$ export GCP_PROJECT_ID=<actual-project-id>
You have completed the necessary configuration in GCP to support the Vault GCP secrets engine.
Configure Vault
(Persona: admin)
Note
The GCP auth method focuses on identities specific to Google Cloud and does not support authenticating arbitrary Google or Google Workspace users or generic OAuth against Google.
In the terminal where you set the
TUTORIAL_TEMP
environment variable, enable the GCP secrets engine.$ vault auth enable gcp Success! Enabled gcp auth method at: gcp/
Configure the GCP auth method to use the
VaultServiceAccountKey.json
credentials.$ vault write auth/gcp/config \ credentials=@$TUTORIAL_TEMP/VaultServiceAccountKey.json
Example output:
Success! Data written to: auth/gcp/config
Create a policy file named dev.hcl.
$ tee dev.hcl <<EOF # Read permission on the k/v secrets path "/secret/*" { capabilities = ["read", "list"] } EOF
Create a policy named
dev
with the policy defined indev.hcl
.$ vault policy write dev dev.hcl Success! Uploaded policy: dev
Create a role for GCE instances.
A role includes several parameters.
policies
: One or more comma separated Vault policies that will be applied.bound_projects
: The name of your GCP project.bound_zones
: One or more comma separated GCP zones permitted to authenticate with Vault.
$ vault write auth/gcp/role/vault-gce-auth-role \ type="gce" \ policies="dev" \ bound_projects="$GCP_PROJECT_ID" \ bound_zones="us-east1-b" \
Review the GCP auth method API documentation for a full list of supported parameters.
The Vault configuration is complete
Authenticate with Vault from GCP
(Persona: app)
To complete authentication with Vault, a signed JSON Web Token must be passed to Vault. The Vault binary simplifies this process. Refer to the generating JWTs documentation for steps on retrieving the JWT manually.
Deploy GCE VM
Return to the GCP console.
Expand the hamburger menu and navigate to Compute Engine >> VM instances.
Click CREATE INSTANCE with the following details:
- Name:
vault-auth-test
- Region:
us-east1
- Zone:
us-east1-b
- Name:
Keep all other options set to their default values and click Create.
When the instance becomes available, click SSH. A new window will open and log you into the VM.
Setup Vault CLI
The Vault binary will require access to your Vault Dedicated cluster. Ensure that you followed the Lab setup section and enabled public access. In production environments, you can connect your GCP account via an AWS transit gateway and use the Vault Dedicated private address.
Download the HashiCorp GPG key.
$ curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg
Add the HashiCorp repo.
$ echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
Install Vault.
$ sudo apt update && sudo apt install vault
Return to the HCP Portal and login if necessary.
Click Vault in the left navigation pane.
In the Vault clusters pane, click vault-cluster.
Under Cluster URLs, click Public Cluster URL.
Return to the SSH session for the GCP instance and set the
VAULT_ADDR
environment variable to the copied address.$ export VAULT_ADDR=<Public_Cluster_URL>
Set the
VAULT_NAMESPACE
environment variable toadmin
.$ export VAULT_NAMESPACE=admin
Authenticate with Vault using the
vault-gce-auth-role
role.$ vault login -method=gcp role="vault-gce-auth-role" Success! You are now authenticated. The token information displayed below is already stored in the token helper. You do NOT need to run "vault login" again. Future Vault requests will automatically use this token. Key Value --- ----- token hvs.CAESIJr5Wfq_LO0TOVb4CKqkHq1bJGeSkEX5dvdm8f6QM8MBGicKImh2cy4zTTlNaE1iZFV0T0pVZFNreG43TUhZWjAuQ0xHemgQsQY token_accessor V0eS8zohjaTBnLDQEucMsJKM.CLGzh token_duration 1h token_renewable true token_policies ["default" "dev"] identity_policies [] policies ["default" "dev"] token_meta_project_number 1060482098095 token_meta_zone us-east1-b token_meta_instance_creation_timestamp 1678121533 token_meta_instance_id 1617931539497854164 token_meta_project_id hc-c0af9a5c6c9a42328ef9b7ce806 token_meta_service_account_id 115056664069215052875 token_meta_instance_name instance-1 token_meta_role vault-gce-auth-role token_meta_service_account_email 1060482098095-compute@developer.gserviceaccount.com
You have successfully authenticated with Vault using the GCP auth method from a GCE instance.
The
dev
policy was assigned through thevault-gce-auth-role
.
Cleanup
To avoid unnecessary charges and having unused security resources in your GCP account, HCP account, or local workstation, remove the following:
Cleanup Vault
Return to the HCP Portal.
Delete any Vault clusters created for this tutorial.
Cleanup GCP
Return to the GCP console and close the Cloud Shell terminal.
Expand the hamburger menu and navigate to Compute Engine >> VM instances. Delete any GCE instances created for this tutorial.
Expand the hamburger menu and navigate to IAM & Admin >> Service accounts. Delete the VaultServiceAccount service account.
From the IAM & Admin console, click Roles and delete the VaultServiceRole role.
Cleanup local workstation
On your local workstation, switch to the terminal used to configure Vault and create an array of the local environment variables.
$ learnVariables=( \ GCP_PROJECT_ID \ GCP_SERVICE_EMAIL \ VAULT_ADDR \ VAULT_TOKEN \ VAULT_NAMESPACE )
Unset the environment variables.
for v in ${learnVariables[@]}; do unset $v done
Remove the
TUTORIAL_TEMP
directory.$ cd ~ && rm -rf $TUTORIAL_TEMP && unset TUTORIAL_TEMP
Help and reference
- Google Cloud auth method documentation
- Google Cloud API documentation
- Code examples to generate JWTs
- GCP auth method repository