This article explains what compute resources are and how to create and use them.
Compute resources are one type of workload assets. A compute resource is a template that simplifies how workloads are submitted and can be used by AI practitioners when they submit their workloads.
A compute resource asset is a preconfigured building block that encapsulates all the specifications of compute requirements for the workload including:
GPU devices and GPU memory
CPU memory and CPU compute
The Compute resource table can be found under Workload manager in the NVIDIA Run:ai UI.
The Compute resource table provides a list of all the compute resources defined in the platform and allows you to manage them.
The Compute resource table consists of the following columns:
The name of the compute resource
A description of the essence of the compute resource
GPU devices request per pod
The number of requested physical devices per pod of the workload that uses this compute resource
GPU memory request per device
The amount of GPU memory per requested device that is granted to each pod of the workload that uses this compute resource
The minimum amount of CPU memory per pod of the workload that uses this compute resource
The maximum amount of CPU memory per pod of the workload that uses this compute resource
The minimum number of CPU cores per pod of the workload that uses this compute resource
The maximum number of CPU cores per pod of the workload that uses this compute resource
The scope of this compute resource within the organizational tree. Click the name of the scope to view the organizational tree diagram
The list of workloads associated with the compute resource
The list of workload templates that use this compute resource
The name of the user who created the compute resource
The timestamp of when the compute resource was created
The timestamp of when the compute resource was last updated
The cluster that the compute resource is associated with
Workloads Associated with the Compute ResourceClick one of the values in the Workload(s) column to view the list of workloads and their parameters.
The workload that uses the compute resource
Workspace/Training/Inference
Customizing the Table ViewFilter - Click ADD FILTER, select the column to filter by, and enter the filter values
Search - Click SEARCH and type the value to search by
Sort - Click each column header to sort by
Column selection - Click COLUMNS and select the columns to display in the table
To add a new compute resource:
Go to the Compute resource table
Click +NEW COMPUTE RESOURCE
Select under which cluster to create the compute resource
Enter a name for the compute resource. The name must be unique.
Optional: Provide a description of the essence of the compute resource
Set the resource types needed within a single node. The NVIDIA Run:ai Scheduler tries to match a single node that complies with the compute resource for each of the workload’s pods.
GPU devices - The number of devices (physical GPUs) per pod. For example, if you requested 3 devices per pod and the running workload using this compute resource consists of 3 pods, there are 9 physical GPU devices used in total.
Note
When setting it to zero, the workload using this computer resource neither requests nor uses GPU resources while running.
You can set any number of GPU devices and specify the memory requirement to any portion size (1..100), or memory size value using GB or MB units per device.
Enable GPU fractioning to allocate a portion of the GPU memory to each device
Select the memory request format
% (of device) - Fraction of a GPU device’s memory
MB (memory size) - An explicit GPU memory unit
GB (memory size) - An explicit GPU memory unit
Set the memory Request - The minimum amount of GPU memory that is provisioned per device. This means that any pod of a running workload that uses this compute resource, receives this amount of GPU memory for each device(s) the pod utilizes.
Optional: Set the memory Limit - The maximum amount of GPU memory that is provisioned per device. This means that any pod of a running workload that uses this compute resource, receives at most this amount of GPU memory for each device(s) the pod utilizes. The limit value must be equal to or higher than the request.
Note
GPU memory limit is disabled by default. If you cannot see Limit in the compute resource form, then it must be enabled by your Administrator, under General settings → Resources → GPU resource optimization.
When a Limit is set and is bigger than the Request, the Scheduler allows each pod to reach the maximum amount of GPU memory in an opportunistic manner (only upon availability).
If the GPU memory Limit is bigger that the Request, the pod is prone to be killed by the NVIDIA Run:ai toolkit (out of memory signal). The greater the difference between the GPU memory used and the request, the higher the risk of being killed.
If GPU resource optimization is turned off, the minimum and maximum are in fact equal.
CPU resources
Select the units for the CPU compute (Cores / Millicores)
Set the CPU compute Request - The minimum amount of CPU compute that is provisioned per pod. This means that any pod of a running workload that uses this compute resource, receives this amount of CPU compute for each pod.
Optional: Set the CPU compute Limit - The maximum amount of CPU compute that is provisioned per pod. This means that any pod of a running workload that uses this compute resource, receives at most this amount of CPU compute. The limit value must be equal to or higher than the request. By default, the limit is set to “Auto”, which means that the pod may consume up to the node's maximum available CPU compute resources.
Select the units for the CPU memory (MB / GB)
Set the CPU memory Request - The minimum amount of CPU memory that is provisioned per pod. This means that any pod of a running workload that uses this compute resource, receives this amount of CPU memory for each pod.
Optional: Set the CPU memory Limit - The maximum amount of CPU memory that is provisioned per pod. This means that any pod of a running workload that uses this compute resource, receives at most this amount of CPU memory. The limit value must be equal to or higher than the request. By default, the limit is set to “Auto”, which means that the pod may consume up to the node's maximum available CPU memory resources.
Note
If the CPU Memory Limit is bigger that the Request, the pod is prone to be killed by the operating system (out of memory signal). The greater the difference between the CPU memory used and the request, the higher the risk of being killed.
Optional: More settings
Increase shared memory size When enabled, the shared memory size available to the pod is increased from the default 64MB to the node's total available memory or the CPU memory limit, if set above.
Set extended resource(s) Click +EXTENDED RESOURCES to add resource/quantity pairs. For more information on how to set extended resources, see the Extended resources and Quantity guides
Click CREATE COMPUTE RESOURCE
To edit a compute resource:
Select the compute resource you want to edit
Update the compute resource and click SAVE COMPUTE RESOURCE
Note
The already bound workload that is using this asset will not be affected.
Copying a Compute ResourceTo copy an existing compute resource:
Select the compute resource you want to copy
Enter a name for the compute resource. The name must be unique.
Update the compute resource and click CREATE COMPUTE RESOURCE
Select the compute resource you want to delete
On the dialog, click DELETE to confirm
Note
The already bound workload that is using this asset will not be affected.
Go to the Compute resources API reference to view the available actions
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4