Stay organized with collections Save and categorize content based on your preferences.
This document explains concepts for the operating system (OS) environment—the boot disks and OS image—for the Compute Engine virtual machine (VM) instances that run Batch jobs.
A job is executed by running the instructions on the boot disk of each of its VMs. Each boot disk contains a bootloader, file system, and OS. You can optionally configure the VM OS environment for a job by doing one of the following:
The following sections describe when to configure the OS environment, the default OS environment, VM OS image options, and boot disk options.
When to configure the OS environmentReview this section to determine whether to configure the VM OS image and/or boot disks when you create a job or use the default configuration.
Configure the VM OS image for a job in the following cases:
If the trusted images (compute.trustedImageProjects) policy constraint prevents you from creating a job with the default VM OS image.
For more information about the trusted images policy constraint, see Control access to the VM OS images for Batch.
If you prefer not to or cannot use the default VM OS image for your job. For example, if the runnables that you've defined for a job require an application that is not supported by the default image, you might be able to specify a different public VM OS image. Alternatively, if you want to preinstall an application that is required for your job to help reduce runtime, you might want to create and specify a custom image.
Configure the boot disk for a job in the following cases:
If you are still unsure if your job requires you to configure the VM OS image and/or boot disks, you can review the following section to understand the default configuration. Additionally, you might also just try to create and run the job with the default configuration. If the job fails with the default configuration, you can try to determine if the issue was caused by the VM OS image or boot disk by troubleshooting.
Otherwise, if you don't need and don't want to configure the VM OS image and/or boot disk for a job, leave these properties undefined to use the default configuration.
Default OS environmentReview this section to understand the default configuration for a job's VM OS image and boot disks. For example, you might use the default configuration as a reference when determining how you want to configure the OS environment for a job.
When you create a job and don't define the VM OS image, the job defaults to using one of the following VM OS images:
When you create a job and don't define the other properties of the boot disks, the job defaults to using the following options:
Persistent Disk type: the default Persistent Disk type for boot disks is pd-balanced.
size: the default size for each VM's boot disk is determined by the following formula for \({defaultBootDiskSize}\). Additionally, the default size is rounded to be at least 30 GB and at most 10,000 GB.
\[{defaultBootDiskSize}={reservedSize}+({taskSize}*{parallelTasksPerVm})\]
This formula has the following values:
\({reservedSize}\): the reserved size in GB for required software for the VM, such as the VM OS image. For the default boot disk size, Batch reserves 10 GB.
\({taskSize}\): the boot disk size for each task in GB, which is is determined by converting the units of the boot disk size per task (bootDiskMib) field.
\({parallelTasksPerVM}\): the maximum number of tasks that can run on a VM simultaneously. For more information, see the formula for \({parallelTasksPerVM}\).
interface: the interface is automatically selected as either NVMe or SCSI depending on the machine type of your job's VMs. You can optionally specify a machine type when you define the type of resources for a job to use. For the default interface of each machine type, see Storage interface types in the Compute Engine documentation.
Note: Unlike the other properties in this list, you cannot directly choose the interface of a persistent disk. However, the interface determines the required drivers, which you might need to consider if you specify your VM OS image. For more information, see Restrictions for VM OS images in this document.The following sections describe the VM OS image options for Batch, including all the types of VM OS images that Batch jobs can support and the restrictions for the supported VM OS images. When you specify the VM OS image for a job, you can specify an image name or image family for any of the supported VM OS images that are allowed by the restrictions.
Supported VM OS imagesBatch jobs can support the following types of VM OS images:
Public images: images that are provided and maintained by Google, open source communities, and third-party vendors. Specifically, Batch jobs can use public images from Batch or Compute Engine.
Batch images: images that are provided and maintained by Batch based on Compute Engine OSes, stored in the batch-custom-image image project, and are recommended for Batch jobs.
Batch images are available in the following OSes. When you view image families and image names from Batch, the OS is indicated by the corresponding prefix.
Batch Debian (batch-debian prefix): an OS based on Debian images from Compute Engine that is optimized for Batch jobs. This OS is the default for jobs with any script runnables.
Batch Container-Optimized OS (batch-cos prefix): an OS based on Container-Optimized OS images from Compute Engine that is optimized for Batch jobs with containers. This OS is the default for jobs with only container runnables.
Batch High-Performance Computing (HPC) Rocky Linux (batch-hpc-rocky prefix): an OS based on HPC Rocky Linux images from Compute Engine that is optimized for Batch jobs with HPC workloads.
Batch CentOS (batch-centos prefix): an OS based on CentOS Linux images from Compute Engine that is optimized for Batch jobs.
batch-hpc-centos prefix): an OS based on HPC CentOS images from Compute Engine that is optimized for Batch jobs with HPC workloads.To see the specific image names and families from Batch that you can specify for a job, follow the steps to view VM OS images from Batch.
Compute Engine images: public images available on Compute Engine, which are described on the Operating system details page in the Compute Engine documentation.
Batch only supports generally available (GA) Compute Engine images from the following OSes:
Important: Due to a known issue, some Compute Engine images are causing jobs to fail. You might resolve this issue by using a newer image version. For more information, see Jobs might fail when specifying Compute Engine (or custom) VM OS images with outdated kernels.Custom images: images that you create, own, and control access to by using Compute Engine. Learn more about custom images in the Compute Engine documentation.
Important: Custom images can be used with Batch but might not be supported. If you want to use a custom image with Batch, building your custom image on top of a public image that Batch supports is recommended.
To learn how to build an image that is compatible with Batch for various OSes, see the build-custom-image samples on GitHub.
The supported VM OS images that a job can specify are subject to the following restrictions:
Batch images that are marked as deprecated might not be supported. The minimum supported lifetime for Batch images is the earliest of the following dates:
batch-cos-stable-official-20230719-00-p00 image is from July 19, 2023 (20230719).Certain job configurations require external access for the image. Specifically, when a job does not have external access and does not allow access through Cloud NAT or Private Google Access, only the following images are supported:
If the trusted images policy constraint is enabled, you cannot use any VM OS images that the trusted image policy does not allow.
An administrator for your project, folder, or organization can optionally configure the trusted images policy constraint to restrict the VM OS images that can be used to create boot disks. For example, the trusted images policy constraint can be used to require users to create VMs that contain approved software that meets your policy or security requirements. For more information, see Control access to VM OS images for Batch.
The VM OS image for your job must have the drivers for the interface of its boot disks and any persistent disk storage volumes installed. Most public images automatically install both NVMe and SCSI drivers, including all Batch images.
You cannot configure the interface of persistent disks, so the VM OS image requires the drivers for the default interface.
If a job specifies any VM OS image that is neither a Batch image nor a custom image based on a Batch image, Batch installs required packages on the job's VMs after they start. These required packages can vary for different job configurations and change over time, and they are required for Batch to run the job.
This section describes the options for the type and size of boot disks for Batch. When you customize the boot disks for a job, you can specify the VM OS image, type, and/or size. For information about specifying the VM OS image, see VM OS image options in this document instead.
For boot disk type, Batch supports all the Persistent Disk types in the Compute Engine documentation.
For boot disk size, Batch supports all sizes and increments for persistent disks in the Compute Engine documentation that meet the following requirements:
For Batch jobs, the minimum boot disk size is 30 GB.
For a Batch job to run properly, the boot disk size must be big enough to allocate the job's VM OS image and the tasks that run on each VM. For example, see the formula for the default boot disk size in this document.
Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see the Google Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.
Last updated 2025-08-07 UTC.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2025-08-07 UTC."],[[["This document details the operating system (OS) environment, including boot disks and OS images, for Compute Engine virtual machine (VM) instances that run Batch jobs."],["You can configure the VM OS environment for a job by specifying the VM OS image to customize the software, or by customizing the boot disk to adjust the image, disk size, and disk type."],["The VM OS image and boot disk configuration can be defined or left as default, and certain scenarios such as policy constraints, specific application requirements, or the need for more storage space, may dictate a custom configuration."],["Batch supports public images from Batch and Compute Engine, as well as custom images, with Batch images being the recommended choice and certain images having usage restrictions, especially if the job lacks external network access."],["The default boot disk configuration uses `pd-balanced` disk type and sizes based on a formula involving reserved space, task size and number of tasks, with the ability to customize the disk type and size with a minimum of 30 GB."]]],[]]
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4