RetroSearch Browse

Home - News ( United States | United Kingdom | Italy | Germany ) - Football scores

Showing content from http://cloud.google.com/compute/docs/disks/creating-linux-application-consistent-pd-snapshots below:

Create Linux application consistent disk snapshots | Compute Engine Documentation

Create Linux application consistent disk snapshots

Stay organized with collections Save and categorize content based on your preferences.

You can create application consistent snapshots of disks attached to Linux virtual machine (VM) instances. In general, the quality of your disk snapshot depends on how well your applications can recover from snapshots that you create during heavy write workloads. Application consistent snapshots capture the state of application data at the time of backup with all application transactions completed and all pending writes flushed to the disk.

To create snapshots that are application consistent, pause apps or operating system processes that write data to the disk, flush the disk buffers, and sync the file system before you create the snapshot. Depending on your application, these and other steps might be required to ensure that all application transactions are complete and captured in the backup.

To create an application consistent snapshot of your disks, use the following process:

To prepare the guest environment for application consistency, create custom shell scripts to run before and after the snapshot is captured
Configure snapshot settings on your virtual machine (VM) instance.
Create a snapshot with the guest-flush option enabled. The guest-flush option starts your pre and post snapshot scripts.

Before you begin

Create a Linux VM.
Update the guest environment.
If you haven't already, then set up authentication. Authentication is the process by which your identity is verified for access to Google Cloud services and APIs. To run code or samples from a local development environment, you can authenticate to Compute Engine by selecting one of the following options:
Select the tab for how you plan to use the samples on this page:
Console
When you use the Google Cloud console to access Google Cloud services and APIs, you don't need to set up authentication.
gcloud
1. Install the Google Cloud CLI. After installation, initialize the Google Cloud CLI by running the following command:
```
gcloud init
```
  If you're using an external identity provider (IdP), you must first sign in to the gcloud CLI with your federated identity.
  Note: If you installed the gcloud CLI previously, make sure you have the latest version by running gcloud components update.
2. Set a default region and zone.
REST
To use the REST API samples on this page in a local development environment, you use the credentials you provide to the gcloud CLI.
For more information, see Authenticate for using REST in the Google Cloud authentication documentation.

Required roles and permissions

To get the permissions that you need to manage standard snapshots, ask your administrator to grant you the following IAM roles on the project:

Compute Instance Admin (v1) (roles/compute.instanceAdmin.v1)
To connect to a VM that can run as a service account: Service Account User (v1) (roles/iam.serviceAccountUser)

For more information about granting roles, see Manage access to projects, folders, and organizations.

These predefined roles contain the permissions required to manage standard snapshots. To see the exact permissions that are required, expand the Required permissions section:

Required permissions

The following permissions are required to manage standard snapshots:

To create a snapshot of a zonal disk:
- compute.snapshots.create on the project
- compute.disks.createSnapshot on the disk
To create a snapshot of a regional disk using the data on the disk:
- compute.snapshots.create on the project
- compute.instances.useReadOnly on the source VM
- compute.disks.createSnapshot on the disk
To create a snapshot of a regional disk from a replica recovery checkpoint:
- compute.snapshots.create on the project
- compute.disks.createSnapshot on the disk
To create a snapshot schedule: compute.resourcePolicies.create on the project or organization
To attach a snapshot schedule to a disk:
- compute.disks.addResourcePolicies on the disk
- compute.resourcePolicies.use on the resource policy
To delete a snapshot:
- compute.snapshots.delete on the snapshot
- compute.snapshots.list on the project

You might also be able to get these permissions with custom roles or other predefined roles.

Limitations

Creating application consistent snapshots on Linux has the following limitations:

Application consistency is guaranteed only by the behavior of your custom pre and post snapshot scripts, not by the snapshot operation itself.
When using the guest-flush option in your snapshot creation request, the snapshot isn't created if the script returns an error or reaches the timeout limit.

Create pre and post snapshot scripts

Before you proceed, update the guest environment so that you are running the latest software on your Linux VM.

To facilitate application consistency, create pre and post snapshot shell scripts to run before and after the snapshot is captured. Use the pre and post scripts for operations such as:

Pause apps or operating system processes running on the VM that writes data to the disk.
Flush the disk buffers. For example, MySQL has a FLUSH statement. Use whichever tool is available for your application.
Sync your file system.

The following code example shows a pre snapshot script. Note the leading #! characters.

Caution: Running fsfreeze -f blocks any running process that tries to access the file system, so use with caution if your application is latency-sensitive.

#!/bin/bash
sudo fsfreeze -f [example-disk-location]

The following code example shows a post snapshot script. Note the leading #! characters.

#!/bin/bash
sudo fsfreeze -u [example-disk-location]

You must save your scripts on your VM in the directory /etc/google/snapshots/. The full path of your pre script must be /etc/google/snapshots/pre.sh and the full path of your post script must be /etc/google/snapshots/post.sh.

Note: You must copy the scripts to every VM on which you want to enable Linux application consistent snapshots. Referencing specific disks in your scripts

The first argument passed to your pre and post snapshot scripts is a list of disks for which you are creating snapshots. You can use this argument in your scripts for various checks. For example, if your VM has multiple disks attached but you only specified one disk in your snapshot request, you can check which disk the snapshot is being created for.

The argument is formatted as follows:

SCSI-attached disks: a comma-separated list of <target/lun> pairs.
NVME-attached disks: a comma-separated list of <nvme:namespace> pairs.

For example, your SCSI-attached boot disk might appear as 1/0 while an additional disk attached to the VM might appear as 2/0.

Edit your guest environment configuration file

Configure your application consistent snapshot settings by updating a specific configuration file on your VM.

Open or create your guest environment configuration file.
```
edit /etc/default/instance_configs.cfg
```
Add the following section to the configuration file, then save your changes and exit the editor.
```
[Snapshots]
enabled = ENABLED
timeout_in_seconds = TIMEOUT_SECONDS
```
Replace the following:
- ENABLED: Set to true to enable the application consistent snapshot feature. The default value is false.
- TIMEOUT_SECONDS: The number of seconds the pre or post snapshot script can take to finish running before timing out. The integer value must be between 0 and 300. The default value is 60.
  Note: The number of seconds the entire snapshot operation can take to complete before a timeout error is 300 seconds per disk, and can't be changed.
Restart the Guest Agent to use the new configuration settings.
```
$ sudo systemctl restart google-guest-agent.service
```

Create a snapshot with guest-flush enabled

Using the Google Cloud console, the Google Cloud CLI, or REST, create a snapshot with the guest-flush option enabled. This starts running the pre and post snapshot scripts before and after the snapshot is captured.

Console

Go to the Create a Snapshot page in the Google Cloud console.
Go to the Create a Snapshot page
Enter a snapshot Name.
Select a Snapshot type. The default is a STANDARD snapshot, which is the best option for long-term back up and disaster recovery.

Choose Archive snapshot for more cost-efficient data retention.
Optional: Enter a Description of the snapshot.
Under Source disk, select the existing disk that you want to create a snapshot of.
In the Location section, choose your snapshot storage location.

The predefined or customized default location defined in your snapshot settings is automatically selected. Optionally, you can override the snapshot settings and store your snapshots in a custom storage location by doing the following:
1. Choose the type of storage location that you want for your snapshot.
  - Choose Multi-regional for higher availability at a higher cost.
  - Choose Regional snapshots for more control over the physical location of your data at a lower cost.
2. In the Select location field, select the specific region or multi-region that you want to use. To use the region or multi-region that is closest to your source disk, select Based on disk's location.
Check the Enable application consistent snapshot option.
Click Create to create the snapshot.

gcloud

You can create your snapshot in the storage location policy defined by your snapshot settings or using an alternative storage location of your choice. For more information, see Choose your snapshot storage location.

To create a snapshot in the predefined or customized default location configured in your snapshot settings, use the gcloud compute snapshots create command.

gcloud compute snapshots create SNAPSHOT_NAME \
    --source-disk-zone=SOURCE_ZONE \
    --source-disk=SOURCE_DISK_NAME \
    --snapshot-type=SNAPSHOT_TYPE \
    --guest-flush

Alternatively, to override the snapshot settings and create a snapshot in a custom storage location, include the --storage-location flag to indicate where to store your snapshot.

gcloud compute snapshots create SNAPSHOT_NAME \
    --source-disk-zone=SOURCE_ZONE \
    --source-disk=SOURCE_DISK_NAME \
    --snapshot-type=SNAPSHOT_TYPE \
    --storage-location=STORAGE_LOCATION \
    --guest-flush

(Preview) To create a regionally scoped snapshot in an allowed region, include the --region flag to indicate where to create your snapshot.

gcloud beta compute snapshots create SNAPSHOT_NAME \
    --region=SNAPSHOT_SCOPE_REGION
    --source-disk=SOURCE_DISK_NAME \
    --source-disk-zone=SOURCE_ZONE \
    --snapshot-type=SNAPSHOT_TYPE

Replace the following:

SNAPSHOT_NAME: A name for the snapshot.
SOURCE_ZONE: The zone of the source disk.
SOURCE_DISK_NAME: The name of the disk volume from which you want to create a snapshot.
SNAPSHOT_TYPE: The snapshot type, either STANDARD or ARCHIVE. If a snapshot type is not specified, a STANDARD snapshot is created.
STORAGE_LOCATION: Optional: For globally scoped snapshots, the Cloud Storage multi-region or the Cloud Storage region where you want to store your snapshot. You can specify only one storage location.

Use the --storage-location parameter only when you want to override the predefined or customized default storage location configured in your snapshot settings.
SNAPSHOT_SCOPE_REGION: Optional: For regionally scoped snapshots, the region that the snapshot is scoped to. If you include this parameter, you can't use the --storage-location parameter. The STORAGE_LOCATION is automatically set to the SNAPSHOT_SCOPE_REGION.

Note: Google recommends using the gcloud compute snapshots create command instead of the gcloud compute disks snapshot command because it supports more features, such as creating snapshots in a project different from the source disk project. REST

To create a snapshot in the predefined or customized default location configured in your snapshot settings, make a POST request to the snapshots.insert method:

POST https://compute.googleapis.com/compute/beta/projects/DESTINATION_PROJECT_ID/global/snapshots
{
  "name": "SNAPSHOT_NAME",
  "sourceDisk": "projects/SOURCE_PROJECT_ID/zones/SOURCE_ZONE/disks/SOURCE_DISK_NAME",
  "snapshotType": "SNAPSHOT_TYPE",
  "guestFlush": true,
}

Alternatively, to override the snapshot settings and create a snapshot in a custom storage location, make a POST request to the snapshots.insert method and include the storageLocations property in your request:

POST https://compute.googleapis.com/compute/beta/projects/DESTINATION_PROJECT_ID/global/snapshots
{
  "name": "SNAPSHOT_NAME",
  "sourceDisk": "projects/SOURCE_PROJECT_ID/zones/SOURCE_ZONE/disks/SOURCE_DISK_NAME",
  "snapshotType": "SNAPSHOT_TYPE",
  "storageLocations": [
      "STORAGE_LOCATION"
  ],
  "guestFlush": true,
}

(Preview) To create a regionally scoped snapshot in an allowed region, make a POST request to the snapshots.insert method and define the creation region:

POST https://compute.googleapis.com/compute/beta/projects/DESTINATION_PROJECT_ID/regions/SNAPSHOT_SCOPE_REGION/snapshots
{
    "name": "SNAPSHOT_NAME",
    "sourceDisk": "projects/SOURCE_PROJECT_ID/zones/SOURCE_ZONE/disks/SOURCE_DISK_NAME",
    "snapshotType": "SNAPSHOT_TYPE"
}

Replace the following:

DESTINATION_PROJECT_ID: The ID of project in which you want to create the snapshot.
SNAPSHOT_NAME: A name for the snapshot.
SOURCE_PROJECT_ID: The ID of the source disk project.
SOURCE_ZONE: The zone of the source disk.
SOURCE_DISK_NAME: The name of the disk from which you want to create a snapshot.
SNAPSHOT_TYPE: The snapshot type, either STANDARD or ARCHIVE. If a snapshot type is not specified, a STANDARD snapshot is created.
STORAGE_LOCATION: Optional: For globally scoped snapshots, the Cloud Storage multi-region or the Cloud Storage region where you want to store your snapshot. You can specify only one storage location.

Use the storageLocations parameter only when you want to override the predefined or customized default storage location configured in your snapshot settings.
SNAPSHOT_SCOPE_REGION: Optional: For regionally scoped snapshots, the region that the snapshot is scoped to. If you include this parameter, you can't use the storageLocations parameter. The STORAGE_LOCATION is automatically set to the SNAPSHOT_SCOPE_REGION.

Note: Google recommends using the snapshots.insert method instead of the disks.createSnapshot method because it supports more features, such as creating snapshots in a project different from the source disk project. Create a snapshot schedule with guest-flush enabled

Use scheduled snapshots to regularly and automatically backup your zonal and regional Persistent Disk and Google Cloud Hyperdisk. If you want to schedule application consistent snapshots for your backup, use the --guest-flush option when you create the snapshot schedule so that the pre and post snapshot scripts execute before and after each scheduled snapshot.

For example, after configuring your guest environment configuration file and creating custom scripts, the following command creates hourly application consistent snapshots:

gcloud compute resource-policies create snapshot-schedule SCHEDULE_NAME \
  --description "MY HOURLY SNAPSHOT SCHEDULE" \
  --start-time 22:00 \
  --hourly-schedule 4 \
  --guest-flush \
  --max-retention-days SNAPSHOT_RETENTION_AGE

To learn more, see About snapshot schedules for disks.

Troubleshooting

Troubleshoot the snapshot schedule and creation process by reviewing reviewing logs and checking configurations.

Review the logs

Go to the Logs Explorer page in the Google Cloud console:

Go to Logs Explorer

Paste the following query in the Log query pane:

resource.type="gce_disk"
jsonPayload.event_subtype="compute.disks.createSnapshot" OR
protoPayload.methodName="ScheduledSnapshots"

Run the query and investigate the logs:

Check configurations

If no pre or post snapshot scripts are found on the instance to freeze the activity/IOPS on the file system, no snapshot will be created. Make sure you've followed the steps in Create pre and post snapshot scripts.
If there is a script error or timeout, no snapshot is created. Review Preparing for consistent snapshots. Note that the maximum timeout period that you can configure in your settings is 300 seconds. Try repeating the entire process using the sample scripts.
A disk must be attached to a VM to capture a snapshot of the disk with guest-flush enabled. To learn how to create a Linux VM and attach a disk, see Creating a VM.

What's next

Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see the Google Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.

Last updated 2025-08-07 UTC.

[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2025-08-07 UTC."],[[["Application-consistent snapshots of Linux VM disks capture application data with all transactions completed and pending writes flushed, crucial for data integrity during backups."],["Creating application-consistent snapshots involves pausing applications, flushing disk buffers, syncing the file system, and utilizing custom pre and post snapshot scripts."],["The `guest-flush` option must be enabled when creating a snapshot to trigger custom scripts that prepare the guest environment for application consistency."],["Proper configuration of the guest environment and the creation of custom pre and post snapshot scripts, saved in the `/etc/google/snapshots/` directory, are essential for successful application-consistent snapshots."],["Snapshots can be created manually or scheduled, with the option to define a storage location for them, and troubleshooting involves checking logs and ensuring correct script configuration and timeouts."]]],[]]

RetroSearch is an open source project built by @garambo | Open a GitHub Issue

Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo

HTML: 3.2 | Encoding: UTF-8 | Version: 0.7.4