HCP Terraform allows you to manage isolated, private, or on-premises infrastructure using self-hosted HCP Terraform agents. The agent polls HCP Terraform or Terraform Enterprise for any changes to your configuration and executes the changes locally, so you do not need to allow public ingress traffic to your resources. Agents allow you to control infrastructure in private environments without modifying your network perimeter.
In this tutorial, you will create an HCP Terraform agent pool, launch local Dockerized agents on your machine, and configure an HCP Terraform workspace to use the agents to manage an Nginx container on your machine. You will also manage tokens for multiple HCP Terraform agents. By the end of the tutorial, you will understand how to set up a similar configuration in your own environment.
Note
HCP Terraform Free Edition includes one self-hosted agent. Refer to HCP Terraform pricing for details.
For this tutorial, you will need:
Agent pools are groups of agents that can share tokens. When you associate a workspace with an agent pool, any agent in the pool can execute a run in that workspace.
To create an agent pool, navigate to the "Agents" panel within your HCP Terraform's "Settings" page and click "Create agent pool."
Enter education as the name, then click "Continue."
Next, HCP Terraform will prompt you to generate a token for the agent pool. Add agent1 as the description and click "Create token."
The next page displays your agent token and example commands for getting your agent running. This page only displays the token on creation — leave it open for now.
The page suggests a command to run the Docker agent that passes in environment variables with individual -e flags. In this tutorial, you will use the --env-file parameter instead, which makes it easier to manage multiple variables within a single file, so you will not have to re-enter them each time you need to relaunch the container. Using an environment variable file also prevents credentials from appearing in your process tables, granting an extra layer of security.
You can configure multiple tokens per agent pool, or have one shared token across all agents in the pool. By using unique tokens, you can revoke the token of one agent and stop its work without disrupting other agents in the pool, which you will do later in this tutorial.
Now configure a version-control-driven HCP Terraform workspace to use the agent pool.
In a new browser window, create a fork of the demo repository in your own GitHub account. This repository contains sample configuration to spin up an Nginx Docker container.
In that same browser window, create a new learn-terraform-cloud-agents workspace associated with your forked repository. For a detailed example of how to configure a VCS integration in HCP Terraform, revisit the Use VCS-Driven Workflow tutorial.
Next, navigate to your workspace's "General" settings.
Under "Execution Mode," select "Custom" and then select "Agent". Select "education" in the drop down under "Agent Pool."
Make sure to click "Save Settings" at the bottom of the page.
Create a file called agent1.list file and paste in the contents below. Update the value of TFC_AGENT_TOKEN with the token you created in the previous step. Name the agent agent1 for this exercise. This name will show up in the agent management UI and on runs, allowing you to identify specific agents in the future. If you plan on using unique tokens for each agent, use the same name for the agent and token for clarity.
TFC_AGENT_TOKEN=<YOUR TOKEN>
TFC_AGENT_NAME=agent1
The only required environment variable is TFC_AGENT_TOKEN, but the agent loads all variables in your Docker environment. You can use this as an alternative to storing credentials and environment variables in your HCP Terraform Workspace's configuration.
Start your first containerized agent on your local machine. For this exercise, you are mounting the Docker socket using -v /var/run/docker.sock:/var/run/docker.sock. Mounting the socket allows the containerized agent to use the Docker provider to manage other containers on your machine.
$ docker run --name tfc_agent --env-file agent1.list -v /var/run/docker.sock:/var/run/docker.sock hashicorp/tfc-agent:latest
[INFO] agent: Starting: name=agent1 version=0.4.1
[INFO] core: Starting: version=0.4.1
[INFO] core: Agent registered successfully with HCP Terraform: id=agent-9F5TXbYJoA7s7c18 pool-id=apool-wqx8cnrSrFn2AUfi
[INFO] agent: Core version is up to date: version=0.4.1
[INFO] core: Waiting for next job
Keep this terminal open.
Because this scenario uses the containerized Terraform to manage other Docker containers, the agent container needs to access the Docker socket. The HCP Terraform Agent runs as the non-root tfc-agent user within the container, so you need to explicitly modify the permissions for the Docker socket.
Open a new terminal window.
First, verify the group ID of the docker group on your system. It will be the third field in the output returned, in this case 281.
$ grep docker /etc/group
docker:x:281:root
Exec into the agent container as the root user.
$ docker exec -it -u 0 tfc_agent /bin/bash
Create a docker group within the container, replacing <GROUP_ID> with the docker group ID of your host.
$ groupadd -g <GROUP_ID> docker
Add the tfc-agent user, which owns the agent process, to the docker group.
$ usermod -aG docker tfc-agent
Exit out of the container.
Open a new terminal window.
Exec into the agent container as the root user.
$ docker exec -it -u 0 tfc_agent /bin/bash
Change the permissions on the Docker socket to grant the tfc-agent user read and write privileges.
$ chmod 666 /var/run/docker.sock
Exit out of the container.
Note
Mounting the Docker socket and manipulating its permissions is only necessary if you are running the Dockerized agent and need it to manage other Docker containers on the host machine.
Once the agent container launches, verify that it has registered with the pool in the HCP Terraform interface.
Agents are available as Docker containers and as standalone x86 binaries. For guidance on using the binaries, refer to the agent documentation.
In your browser, navigate back to your HCP Terraform learn-terraform-cloud-agents workspace and trigger a run by clicking "Queue plan."
The run "Execution Mode" shows that it is running in your local Docker agent. The plan details show the names of the agent pool and agent responsible for the run.
In your terminal, the Docker agent logs display the agent's Terraform actions.
[INFO] core: Job received: type=plan id=run-NpfMDfGTsnY9ai6A
[INFO] terraform: Handling run: id=run-NpfMDfGTsnY9ai6A type=plan org=hashicorp-training workspace=learn-terraform-cloud-agents
[INFO] terraform: Extracting Terraform from release archive
[INFO] terraform: Terraform CLI details: version=0.14.5
[INFO] terraform: Downloading Terraform configuration
[INFO] terraform: Running terraform init
[INFO] terraform: Running terraform plan
[INFO] terraform: Generating and uploading plan JSON
[INFO] terraform: Generating and uploading provider schemas JSON
[INFO] terraform: Persisting filesystem to remote storage
[INFO] terraform: Finished handling run
[INFO] core: Waiting for next job
Approve the proposed changes in HCP Terraform by clicking on "Confirm & Apply," and then confirming the plan.
The agent will now execute the work. The agent is self-contained and will not spin up additional containers for the Terraform execution itself — it downloads the Terraform version your configuration specifies and any providers necessary to complete the work task. In this tutorial, the agent launches an additional Docker container because that is the resource declared in the sample Terraform configuration.
[INFO] terraform: Handling run: id=run-NpfMDfGTsnY9ai6A type=apply org=hashicorp-training workspace=learn-terraform-cloud-agents
[INFO] core: Job received: type=apply id=run-NpfMDfGTsnY9ai6A
[INFO] terraform: Extracting Terraform from release archive
[INFO] terraform: Terraform CLI details: version=0.14.5
[INFO] terraform: Recovering filesystem from remote storage
[INFO] terraform: Running terraform init
[INFO] terraform: Running terraform apply
[INFO] terraform: Finished handling run
[INFO] core: Waiting for next job
Once the apply is complete, open a new terminal window and confirm that your Nginx container is running by using docker ps or visiting localhost:8000.
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c9469d90663e f6d0b4767a6c "/docker-entrypoint.…" About a minute ago Up About a minute 0.0.0.0:8000->80/tcp nginx
515ed219aa66 hashicorp/tfc-agent:latest "/bin/tfc-agent" 28 minutes ago Up 28 minutes admiring_swartz
Though this tutorial targets your local machine, you can use the same configuration on any instance of your infrastructure. While running, the agent polls HCP Terraform for any new workloads it needs to complete. Since this is a pull-based pattern, you only need to allow TCP/443 egress traffic and DNS resolution. This is a lightweight way to use HCP Terraform within your isolated network segments without needing to configure your own installation of Terraform Enterprise.
Each agent is single-threaded and can only execute the work of one run at a time. For concurrent workloads, you must provision multiple agents. Agents within a pool can share tokens, or use unique ones so you can easily revoke the token of one agent without disrupting others.
Launch a second agent with a unique token.
agent2.
This will take you through the same steps as your first token creation. You will use this second token to launch another agent.
In a new terminal, create a file named agent2.list and open it. Copy and paste the contents below, and set the value of TFC_AGENT_TOKEN to be your new token.
TFC_AGENT_TOKEN=<YOUR TOKEN>
TFC_AGENT_NAME=agent2
Now launch your second agent.
$ docker run --env-file agent2.list -v /var/run/docker.sock:/var/run/docker.sock hashicorp/tfc-agent:latest
The "Agents" page in HCP Terraform will now display 2 agents ready to retrieve and process workloads.
Any other containers that have exited will appear here as well, but do not count toward your allotted number of agents. Refer to the HCP Terraform Agent documentation for a detailed list of possible agent statuses and how they count toward your agent totals.
You can configure additional agent pools as well, allowing you to maintain agents in each of your data centers and network segments.
Next, revoke the token of one of your agents. On the education agent pool page, click "Revoke Token" for the agent2 token.
Once you revoke the token, the agent2 container will log its graceful shutdown and exit.
[ERROR] agent: Unrecoverable error, shutting down: error="GET https://app.terraform.io/api/agent/jobs: unexpected status code (401 Unauthorized): Agent token invalid"
[INFO] agent: Shutting down
[INFO] agent: Core plugin is shutting down
[ERROR] core: Failed updating status: error="PUT https://app.terraform.io/api/agent/status: unexpected status code (401 Unauthorized): Agent token invalid"
[INFO] core: Shutdown complete
Graceful shutdown complete
Meanwhile, the agent1 container will continue running. This demonstrates the advantage of using unique tokens for your agents: you can revoke a token without disrupting any other agents. The state of agent2 will change in the "Agents" dashboard in HCP Terraform.
The exited agent is in an "Exited" state but will expire out of the list and your agent allocation count within 2 hours. If this agent comes back online before that timeout, it will return to an "Idle" state. Only agents in the "Idle," "Busy," or "Exited" states count against your purchased agent limit.
Now that you have created and used HCP Terraform agents to create resources in an isolated environment, clean up all resources created in this tutorial.
Destroy the infrastructurePrior to removing the agent pool, remove the resources created by queueing a destroy plan in your workspace. Follow the Destroy Infrastructure step of the clean up tutorial for step-by-step instructions if needed.
Your agent container will log the destruction plan as well.
[INFO] core: Waiting for next job
[INFO] core: Job received: type=apply id=run-dVh4azMHBwS49QR5
[INFO] terraform: Handling run: id=run-dVh4azMHBwS49QR5 type=apply org=hashicorp-training workspace=learn-terraform-cloud-agents
[INFO] terraform: Extracting Terraform from release archive
[INFO] terraform: Terraform CLI details: version=0.14.6
[INFO] terraform: Recovering filesystem from remote storage
[INFO] terraform: Running terraform init
[INFO] terraform: Running terraform apply
[INFO] terraform: Finished handling run
[INFO] core: Waiting for next job
Note
Destroy your infrastructure resources prior to destroying the agent pool. HCP Terraform will not be able to destroy your local containers without the agent.
Disassociate the agent poolTo delete an agent pool, you must first disassociate it from all workspaces. In your workspace settings, change the "Execution Mode" to "Remote." Be sure to click "Save Settings."
Delete the agent poolOn the "Agent Pool" page in your HCP Terraform organization settings, scroll to the bottom for the "Delete Agent Pool" section. Click "Delete agent pool" on the "Agents" page and confirm the prompt "Yes, delete agent pool."
HCP Terraform's self-hosted agents allow you to manage more of your resources using HCP Terraform or Terraform Enterprise without modifying your network perimeter. Since run execution modes are workspace-specific, you can continue using remote execution for any publicly accessible-resources and use the self-hosted agent for resources that require extra security.
Learn more in the HCP Terraform Agents documentation.
Review the configuration details and differences for using HCP Terraform Agents with Terraform Enterprise.
Explore the HCP Terraform/Enterprise provider.
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4