Home - News ( United States | United Kingdom | Italy | Germany ) - Football scores

Showing content from https://cloud.google.com/sql/docs/postgres/replication/managed-import-replication-from-external below:

Use a managed import to set up replication from external databases | Cloud SQL for PostgreSQL

Stay organized with collections Save and categorize content based on your preferences.

This page describes how to set up and use a managed import for data when replicating from an external server to Cloud SQL.

You must complete all the steps on this page. When finished, you can administer and monitor the source representation instance the same way as you would any other Cloud SQL instance.

Before you begin, complete these steps:

1. Configure the external server.

2. Create the source representation instance.

3. Set up the Cloud SQL replica.

After your setup is complete, ensure that the Cloud SQL replica can replicate from the external server.

The following external sync settings must be correct.

• Connectivity between the Cloud SQL replica and external server
• Replication user privileges
• Version compatibility
• The Cloud SQL replica is not already replicating

To verify these settings, open a Cloud Shell terminal and enter the following commands:

gcloud auth login
ACCESS_TOKEN="$(gcloud auth print-access-token)"
curl --header "Authorization: Bearer ${ACCESS_TOKEN}" \
     --header 'Content-Type: application/json' \
     --data '{
         "syncMode": "SYNC_MODE",
         "syncParallelLevel": "SYNC_PARALLEL_LEVEL",
         "selectedObjects": "SELECTED_OBJECTS"
       }' \
     -X POST \
     https://sqladmin.googleapis.com/sql/v1beta4/projects/PROJECT_ID/instances/REPLICA_INSTANCE_ID/verifyExternalSyncSettings

gcloud auth login
ACCESS_TOKEN="$(gcloud auth print-access-token)"
curl --header "Authorization: Bearer ${ACCESS_TOKEN}" \
     --header 'Content-Type: application/json' \
     --data '{
         "syncMode": "online",
         "syncParallelLevel": "optimal",
         "selectedObjects":[{"database":"db1"}, {"database":"db2"}]
       }' \
     -X POST \
     https://sqladmin.googleapis.com/sql/v1beta4/projects/myproject/instances/myreplica/verifyExternalSyncSettings

These calls return a list of type sql#externalSyncSettingErrorList.

If the list is empty, then there are no errors. A response without errors appears like this:

  {
    "kind": "sql#externalSyncSettingErrorList"
  }

Verify the setting that controls the speed at which data from tables of a database are transferred. The following values are available:

• min: Takes the lowest amount of compute resources on the database. This is the slowest speed for transferring data.
• optimal: Provides a balanced performance with an optimal load on the database.
• max: Provides the highest speed for transferring data, but this might cause an increased load on the database.

Note: The default value for this parameter is optimal because this setting provides a good speed to transfer the data and it has a reasonable impact on the database. We recommend that you use this value.

To update the databases that you want to migrate from the source representation instance to the destination Cloud SQL instance, update the list of database names that are associated with the selectedObjects parameter. If you don't use this parameter or provide an empty list as the value for the parameter, then all databases are migrated from the source to the destination.

    {
      "name": "SOURCE_NAME",
      "region": "REGION",
      "databaseVersion": "DATABASE_VERSION",
      "onPremisesConfiguration": {
        "selectedObjects": "SELECTED_OBJECTS",
        "username": "USERNAME",
        "password": "PASSWORD"
      }
    }

// example of source.json for external server that
// - initiates replication from a Cloud SQL managed import
// - doesn't use SSL/TLS

{
  "name": "cloudsql-source-instance",
  "region": "us-central1",
  "databaseVersion": "POSTGRES_9_6",
  "onPremisesConfiguration": {
    "selectedObjects":[{"database":"db1"}, {"database":"db3"}],
    "username": "newReplicationUser",
    "password": "525#@%*@"
  }
}

Then, to modify the source representation instance in Cloud SQL, open a Cloud Shell terminal and enter the following commands:

gcloud auth login
ACCESS_TOKEN="$(gcloud auth print-access-token)"
curl --header "Authorization: Bearer ${ACCESS_TOKEN}" \
     --header 'Content-Type: application/json' \
     --data @JSON_PATH \
     -X PATCH \
     https://sqladmin.googleapis.com/sql/v1beta4/projects/PROJECT_ID/instances/SOURCE_NAME

gcloud auth login
ACCESS_TOKEN="$(gcloud auth print-access-token)"
curl --header "Authorization: Bearer ${ACCESS_TOKEN}" \
     --header 'Content-Type: application/json' \
     --data @./source.json \
     -X PATCH \
     https://sqladmin.googleapis.com/sql/v1beta4/projects/MyProject/instances/cloudsql-source-instance

After verifying that you can replicate from the external server, start the replication. The speed for performing the replication for the initial import process is up to 500 GB per hour. However, this speed can vary based on the machine tier, data disk size, network throughput, and nature of your database.

gcloud auth login
ACCESS_TOKEN="$(gcloud auth print-access-token)"
curl --header "Authorization: Bearer ${ACCESS_TOKEN}" \
     --header 'Content-Type: application/json' \
     --data '{
         "syncMode": "SYNC_MODE",
         "skipVerification": "SKIP_VERIFICATION",
         "syncParallelLevel": "SYNC_PARALLEL_LEVEL"
       }' \
     -X POST \
     https://sqladmin.googleapis.com/sql/v1beta4/projects/PROJECT_ID/instances/REPLICA_INSTANCE_ID/startExternalSync

gcloud auth login
ACCESS_TOKEN="$(gcloud auth print-access-token)"
curl --header "Authorization: Bearer ${ACCESS_TOKEN}" \
     --header 'Content-Type: application/json' \
     --data '{
         "syncMode": "online",
         "syncParallelLevel": "optimal"
       }' \
     -X POST \
     https://sqladmin.googleapis.com/sql/v1beta4/projects/MyProject/instances/replica-instance/startExternalSync

Provide a setting that controls the speed at which data from tables of a database are transferred. The following values are available:

• min: Takes the lowest amount of compute resources on the database. This is the slowest speed for transferring data.
• optimal: Provides a balanced performance with an optimal load on the database.
• max: Provides the highest speed for transferring data, but this might cause an increased load on the database.

Note: The default value for this parameter is optimal because this setting provides a good speed to transfer the data and it has a reasonable impact on the database. We recommend that you use this value.

Once you start replication from the external server, you need to monitor replication. To learn more, see Monitoring replication. You can then complete your migration.

Consider the following troubleshooting options:

First, check that the value of the max_connections flag is greater than or equal to the value on the primary.

If the max_connections flag is set appropriately, inspect the logs in Cloud Logging to find the actual error.

If the error is: set Service Networking service account as servicenetworking.serviceAgent role on consumer project, then disable and re-enable the Service Networking API. This action creates the service account necessary to continue with the process.

Restart the replica instance to reclaim the temporary memory space.

Edit the instance to enable automatic storage increase.

• Slow queries on the replica. Find and fix them.
• All tables must have a unique/primary key. Every update on such a table without a unique/primary key causes full table scans on th replica.
• Queries like DELETE ... WHERE field < 50000000 cause replication lag with row-based replication since a huge number of updates are piled up on the replica.

Some possible solutions include:

• Edit the instance to increase the size of the replica.
• Reduce the load on the database.
• Send read traffic to the read replica.
• Index the tables.
• Identify and fix slow write queries.
• Recreate the replica.

If you must use hash indexes, upgrade to PostgreSQL 10+. Otherwise, if you also want to use replicas, don't use hash indexes in PostgreSQL 9.6.

Recreate the replica after stopping all running queries.

To resolve this issue, complete the following steps:

1. Turn on the log_duration flag and set the log_statement parameter to ddl. This provides you with both the queries and the run time on the database. However, depending on your workload, this might cause performance issues.
2. On both the primary instance and the read replica, run explain analyze for the queries.
3. Compare the query plan and check for differences.

If this is a specific query, then modify the query. For example, you can change the order of the joins to see if you get better performance.

When you verify your replication settings, logs are produced.

You can view these logs by following these steps:

1. Go to the Logs Viewer in the Google Cloud console.

   Go to the Logs Viewer

2. Select the Cloud SQL replica from the Instance dropdown.
3. Select the replication-setup.log log file.

If the Cloud SQL replica is unable to connect to the external server, confirm the following:

• Any firewall on the external server is configured to allow connections from the Cloud SQL replica's outgoing IP address.
• Your SSL/TLS configuration is correct.
• Your replication user, host, and password are correct.

• Learn about updating an instance.
• Learn about managing replicas.
• Learn about monitoring instances.
• Learn about promoting your Cloud SQL replica.

Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see the Google Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.

Last updated 2025-07-02 UTC.

[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2025-07-02 UTC."],[],[]]

RetroSearch is an open source project built by @garambo | Open a GitHub Issue

Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo

HTML: 3.2 | Encoding: UTF-8 | Version: 0.7.4