Nomad is a flexible scheduler that can run on your local machine as a single node cluster or as a multi-node cluster on virtualized or physical infrastructure.
In this tutorial, you will create a cluster locally on your machine or on one of the major cloud providers, verify connectivity to the cluster, and interact with Nomad via the UI.
Nomad supports many task drivers but the example application in this tutorial uses Docker. Docker Desktop provides the Docker engine, the CLI client, and a graphical interface. Other third-party options are available for running Docker on your machine but we recommend Docker Desktop for the simplicity and ease of use.
Nomad uses a socket file to communicate with Docker and by default, this file is /var/run/docker.sock on Linux, Mac, and other Unix platforms, and /./pipe/docker_engine on Windows. Docker Desktop creates a symlink to this socket during installation.
If the Nomad client fails to detect the Docker driver with the error Constraint "missing drivers": 1 nodes excluded by filter, the issue may be this missing symlink. Additional information about this symlink for Mac and Windows is available.
The example repository includes Terraform configurations for starting a cluster on the cloud as well as the jobspec files for the example application.
Clone the code repository.
$ git clone https://github.com/hashicorp-education/learn-nomad-getting-started
Navigate to the repository folder.
$ cd learn-nomad-getting-started
Check out the v1.1 tag of the repository as a local branch named nomad-getting-started.
$ git checkout -b nomad-getting-started v1.1
Switched to a new branch 'nomad-getting-started'
Start a Nomad cluster. You will use the Nomad CLI to access your cluster regardless of its location. Select where you want to create and run your cluster.
Open your terminal and start the development agent. This creates a Nomad cluster of one node that acts as both the server and client.
Warning
This tutorial uses the network interface attached to the default route on your machine and as a result, workloads will be accessible to other machines on the same network. This is for development and illustration purposes. We recommend using a firewall for production environments or when connected to public networks to limit unauthorized workload access.
The -bind flag set to 0.0.0.0 instructs Nomad to listen on all network interfaces present on the machine. The -network-interface flag instructs Nomad to use a network interface other than the default loopback interface (localhost). In this example, Nomad automatically gets the network interface attached to the default route on the machine with the GetDefaultInterfaces function but you can also provide the name of an interface. This flag is necessary for the example application as each service runs in a different container and cannot reach the other services on the loopback address.
Note
You may encounter a permission denied error when creating the allocation directory. If so, run the nomad agent command without sudo.
Leave Nomad running in this terminal session.
$ sudo nomad agent -dev \
-bind 0.0.0.0 \
-network-interface='{{ GetDefaultInterfaces | attr "name" }}'
In a second terminal session, export the cluster address.
$ export NOMAD_ADDR=http://localhost:4646
Each of the following cloud setups use Terraform to create and configure the server and client nodes. During this process, Terraform installs the dependencies and the Nomad binary, and configures Nomad with user startup scripts.
Be sure that you have Terraform installed and configured.
This process will take about 5 minutes depending on the cloud provider.
Warning
We do not recommend this guide for a production deployment of Nomad. It is a method for quickly creating a multi-node cluster for learning, development, and testing purposes. When you are ready to move on to a production deployment, check out the cluster setup collection and the reference architecture guide for best practices.
Choose the cloud provider below that you want to use.
Make sure that you have your AWS access credentials set as environment variables.
Navigate to the aws folder.
Rename the example variables file to terraform.tfvars.
$ mv terraform.tfvars.example terraform.tfvars
Update the region variable in terraform.tfvars with your AWS region preference. Save the file.
Make sure that you have the gcloud CLI tool installed on your machine.
Navigate to the gcp folder.
Rename the example variables file to terraform.tfvars.
$ mv terraform.tfvars.example terraform.tfvars
Log in to GCP with gcloud in your terminal and follow the prompts to complete the login process.
$ gcloud auth login
Your browser has been opened to visit:
https://accounts.google.com/o/oauth2/auth?response_type=code[...]
You are now logged in as [YOUR_GCP_ACCOUNT].
Your current project is [YOUR_CURRENT_PROJECT]. You can change this setting by running:
$ gcloud config set project PROJECT_ID
Set the project, region, and zone configurations in gcloud.
Note
If you already have a project in your GCP account, these configurations will be set for you as part of the login step. If not, first create a project.
Set project to the project ID.
$ gcloud config set project <GCP_PROJECT_ID>
Updated property [core/project].
Then, set region to the associated region.
$ gcloud config set compute/region <GCP_REGION>
Updated property [compute/region].
Finally, set zone to the associated zone. Note that the zone must be in the region set above.
$ gcloud config set compute/zone <GCP_ZONE>
Updated property [compute/zone].
List the configurations with gcloud.
$ gcloud config list
[compute]
region = us-east1
zone = us-east1-b
[core]
account = [GCP_ACCOUNT]
disable_usage_reporting = True
project = hc-3ff63253e6a54756b207e4d4727
Copy the values for project, region, and zone into terraform.tfvars. In this example, those would be hc-3ff63253e6a54756b207e4d4727, us-east1, and us-east1-b. Save the file.
gcp/terraform.tfvars
region = "us-east1"
zone = "us-east1-b"
project = "hc-3ff63253e6a54756b207e4d4727"
Make sure that you have the az CLI tool installed on your machine.
Navigate to the azure folder.
Rename the example variables file to terraform.tfvars.
$ mv terraform.tfvars.example terraform.tfvars
Open your terminal, log in to Azure with az, and follow the prompts to complete the login process.
$ az login
A web browser has been opened at https://login.microsoftonline.com/organizations/oauth2/v2.0/authorize.
Please continue the login in the web browser. If no web browser is available or if the web browser
fails to open, use device code flow with `az login --use-device-code`.
[
{
"cloudName": "AzureCloud",
"homeTenantId": "1e472a2a-7ab3-9bd1-2016-a32fd04dfb29",
"id": "0e3e2e88-47a3-4107-a2b2-f325314dfb67",
"isDefault": true,
"managedByTenants": [
{
"tenantId": "c9ed8610-2016-4bf5-b919-437a07bf2464"
}
],
"name": "[SUBSCRIPTION_NAME]",
"state": "Enabled",
"tenantId": "1e472a2a-7ab3-9bd1-2016-a32fd04dfb29",
"user": {
"name": "[USER_EMAIL]",
"type": "user"
}
}
]
Copy the values for id and tenantId and paste them into the terraform.tfvars file as values for subscription_id and tenant_id. For this example, the value for subscription_id would be 0e3e2e88-47a3-4107-a2b2-f325314dfb67 and tenant_id would be 1e472a2a-7ab3-9bd1-2016-a32fd04dfb29. Save the file.
azure/terraform.tfvars
location = "LOCATION"
subscription_id = "0e3e2e88-47a3-4107-a2b2-f325314dfb67"
tenant_id = "1e472a2a-7ab3-9bd1-2016-a32fd04dfb29"
client_id = "CLIENT_ID"
client_secret = "CLIENT_SECRET"
Next, create an Azure service principal by providing the value of subscription_id to the --scopes argument.
$ az ad sp create-for-rbac \
--role="Contributor" \
--scopes="/subscriptions/0e3e2e88-47a3-4107-a2b2-f325314dfb67"
Creating 'Contributor' role assignment under scope '/subscriptions/0e3e2e88-47a3-4107-a2b2-f325314dfb67'
The output includes credentials that you must protect. Be sure that you do not include these credentials in your code or check the credentials into your source control. For more information, see https://aka.ms/azadsp-cli
{
"appId": "ab3cb7b2-c932-4eb7-89ce-a369de998a37",
"displayName": "azure-cli-2022-12-02-15-40-24",
"password": "UVq8Q~7VPT9hIVYQ6QCtmCfUyNOTLoaIsze8IdwS",
"tenant": "1e472a2a-7ab3-9bd1-2016-a32fd04dfb29"
}
Copy the values for appId and password and paste them into the terraform.tfvars file as values for client_id and client_secret. For this example, the value for client_id would be ab3cb7b2-c932-4eb7-89ce-a369de998a37 and client_secret would be UVq8Q~7VPT9hIVYQ6QCtmCfUyNOTLoaIsze8IdwS. Save the file.
azure/terraform.tfvars
location = "LOCATION"
subscription_id = "0e3e2e88-47a3-4107-a2b2-f325314dfb67"
tenant_id = "1e472a2a-7ab3-9bd1-2016-a32fd04dfb29"
client_id = "ab3cb7b2-c932-4eb7-89ce-a369de998a37"
client_secret = "UVq8Q~7VPT9hIVYQ6QCtmCfUyNOTLoaIsze8IdwS"
Update the location variable with your Azure location preference. Save the file.
azure/terraform.tfvars
location = "eastus"
subscription_id = "0e3e2e88-47a3-4107-a2b2-f325314dfb67"
tenant_id = "1e472a2a-7ab3-9bd1-2016-a32fd04dfb29"
client_id = "ab3cb7b2-c932-4eb7-89ce-a369de998a37"
client_secret = "UVq8Q~7VPT9hIVYQ6QCtmCfUyNOTLoaIsze8IdwS"
Initialize Terraform to download required plugins and set up the workspace.
Provision the resources. Respond yes to the prompt to confirm the operation. The provisioning takes a couple of minutes.
$ terraform apply
# ...
Apply complete!
Outputs:
IP_Addresses = <<EOT
It will take a little bit for setup to complete and the UI to become available.
Once it is, you can access the Nomad UI at:
http://3.14.8.189:4646/ui
Set the Nomad address, run the bootstrap, export the management token, set the token variable, and test connectivity:
export NOMAD_ADDR=http://3.14.8.189:4646 && \
nomad acl bootstrap | grep -i secret | awk -F "=" '{print $2}' | xargs > nomad-management.token && \
export NOMAD_TOKEN=$(cat nomad-management.token) && \
nomad server members
Copy the token value and use it to log in to the UI:
cat nomad-management.token
Once the provisioning is complete, the Nomad web UI will become available in another couple of minutes.
Open the web UI by visiting the URL in the Terraform output. You may see an error from your browser letting you know that the site cannot be reached or a No cluster leader error message from the Nomad UI after refreshing the page. These are both normal during the setup and mean the cluster is not yet ready. Wait a few minutes and refresh your browser again to see the Nomad UI.
The Nomad UI will show a Not Authorized message once setup is complete. This means the ACL system has not been set up yet. Continue to the next section to set it up.
Set up Nomad
Once the cluster is ready and the web UI is available, you can bootstrap the ACL system.
Export the cluster address as the NOMAD_ADDR environment variable.
$ export NOMAD_ADDR=$(terraform output -raw nomad_ip)
Bootstrap the ACLs and save the management token to a file.
$ nomad acl bootstrap | \
grep -i secret | \
awk -F "=" '{print $2}' | \
xargs > nomad-management.token
Export the token as the NOMAD_TOKEN environment variable.
$ export NOMAD_TOKEN=$(cat nomad-management.token)
Your CLI is now set up to interact with the cluster.
Verify connectivity to the cluster by checking the status of the nodes. The output you see may be different than the example output depending on where your cluster is running and how many nodes it has.
$ nomad node status
ID DC Name Class Drain Eligibility Status
13416cb7 dc1 user-C05G17CLKD <none> false eligible ready
$ nomad node status
ID DC Name Class Drain Eligibility Status
83a1527e dc1 ip-172-31-36-72 <none> false eligible ready
4aeff813 dc1 ip-172-31-32-8 <none> false eligible ready
Navigate to the Nomad UI in your web browser by visiting http://localhost:4646/ui(opens in new tab).
Open the Nomad UI from your terminal. This ui command reads the NOMAD_ADDR and NOMAD_TOKEN environment variables, opens the UI in your browser, and logs in.
$ nomad ui -authenticate
Opening URL "http://3.14.8.189:4646/ui" with one-time token
Alternatively, you can navigate to the Nomad UI in your web browser, click the Sign In link in the top right of the page, and log in with the bootstrap token saved in the NOMAD_TOKEN variable or the nomad-management.token file by setting the Secret ID to the token's value.
Visit the Servers page from the left navigation to see the server nodes in the cluster and the Clients page to see the client nodes. These pages will show only one node each if you are running a local development agent.
Visit the Topology page to see an overview of the clients, the resources available on each, and the total resources available to the cluster.
In this tutorial you created a Nomad cluster. Continue on to the next tutorial by clicking on the Next button below and learn how to submit and run a job.
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4