Home - News ( United States | United Kingdom | Italy | Germany ) - Football scores

Showing content from http://cloud.google.com/sql/docs/mysql/replication/dump-file-replication-from-external below:

Use a dump file to set up replication from external databases | Cloud SQL for MySQL

This page describes the process for setting up replication when you have a dump file that you created from your external server.

You must complete all the steps on this page. When finished, you can administer and monitor the source representation instance the same way as you would any other Cloud SQL instance.

Before you start, you should have configured the external server, created the source representation instance, and set up the Cloud SQL replica.

The replication user on the external server is configured to accept connections from any host (%). You should update this user account so that it can only be used with the Cloud SQL replica. Open a terminal on the external server and enter these commands:

    UPDATE mysql.user
      SET Host='NEW_HOST'
      WHERE Host='OLD_HOST'
      AND User='USERNAME';
      GRANT REPLICATION SLAVE, EXECUTE
      ON *.* TO 'GCP_USERNAME'@'HOST';
    FLUSH PRIVILEGES;

    UPDATE mysql.user
      SET Host='192.0.2.0'
      WHERE Host='%'
      AND User='replicationUser';
      GRANT REPLICATION SLAVE, EXECUTE
      ON *.* TO 'gcp_user'@'gmail.com';
    FLUSH PRIVILEGES;

After your setup is complete, ensure that the Cloud SQL replica can replicate from the external server.

First, ensure that your external sync settings are correct. To do this, use the commands below to verify:

• Connectivity between the Cloud SQL replica and external server
• Replication user privileges
• Version compatibility
• The Cloud SQL replica is not already replicating
• Binlogs are enabled on the external server
• A global transaction identifier (GTID) is enabled

Open a terminal and enter these commands to verify external sync settings are correct:

gcloud auth login
ACCESS_TOKEN="$(gcloud auth print-access-token)"
curl --header "Authorization: Bearer ${ACCESS_TOKEN}" \
     --header 'Content-Type: application/json' \
     --data '{
         "syncMode": "SYNC_MODE",
         "skipVerification": "SKIP_VERIFICATION"
       }' \
     -X POST \
     https://sqladmin.googleapis.com/sql/v1beta4/projects/PROJECT_ID/instances/REPLICA_INSTANCE/verifyExternalSyncSettings

gcloud auth login
ACCESS_TOKEN="$(gcloud auth print-access-token)"
curl --header "Authorization: Bearer ${ACCESS_TOKEN}" \
     --header 'Content-Type: application/json' \
     --data '{
         "syncMode": "online",
       }' \
     -X POST \
     https://sqladmin.googleapis.com/sql/v1beta4/projects/myproject/instances/myreplica/verifyExternalSyncSettings

You can populate a Cloud SQL replica with a mysqldump file located in a Cloud Storage bucket. These conditions apply:

• You must use the mysqldump utility bundled with MySQL.
• While mysqldump is running, do not perform any DDL operations on the external server. Doing so could cause inconsistencies in the export file.

To export your database to a Cloud Storage bucket, follow these steps:

1. In Google Cloud, create a Cloud Storage bucket.
2. Open a terminal using a client that connects to the external database server and run the following command.

    mysqldump \
        --host=EXTERNAL_HOST \
        --port=EXTERNAL_PORT \
        --user=USERNAME\
        --password=PASSWORD \
        --databases=DATABASE_LIST  \
        --hex-blob \
        SOURCE_DATA  \
        --no-autocommit \
        --default-character-set=utf8mb4 \
        --single-transaction \
        --set-gtid-purged=on \
        ADD_DROP_TABLE \
        ROUTINES \
        COMPRESS \
        GZIP \
        | gcloud storage cp - gs://BUCKET/DUMP_FILENAME

    mysqldump \
        --host=192.0.2.1 \
        --port=3306 \
        --user=replicationUser \
        --password \
        --databases guestbook journal \
        --hex-blob \
        --master-data=1 \
        --no-autocommit \
        --default-character-set=utf8mb4 \
        --single-transaction \
        --compress \
        | gzip \
        | gcloud storage cp - gs://replica-bucket/external-database.sql.gz

The source representation instance is a Cloud SQL instance that represents the source database server to the Cloud SQL replica. It's visible in the Google Cloud console and appears the same as a regular Cloud SQL instance, but it contains no data, requires no configuration or maintenance, and doesn't affect billing.

The source.json file contains information about the source representation instance.

{
  "name": "PRIMARY_INSTANCE_NAME",
  "region": "REGION_NAME",
  "databaseVersion": "DB_NAME_AND_VERSION",
  "onPremisesConfiguration": {
    "hostPort": "IP_ADDRESS_AND_PORT",
    "username": "USERNAME",
    "password": "PASSWORD"
  },
  "dumpFilePath" :"DUMP_FILE_PATH"
}

{
  "name": "cloudsql-source-instance",
  "region": "us-central1",
  "databaseVersion": "MYSQL_5_7",
  "onPremisesConfiguration": {
    "hostPort": "192.0.2.0:3306",
    "username": "replicationUser",
    "password": "486#@%*@"
  },
  "dumpFilePath" :"gs://replica-bucket/source-database.sql.gz"
}

For a list of the full set of parameters that you can use with the source.json file, see REST Resource: instances.

The onPremisesConfiguration parameter contains information that Cloud SQL needs to connect to the on-premises source. For more information about the values of this parameter, see OnPremisesConfiguration.

After you set up the Cloud SQL replica, you need to update your source representation instance with the file path of the Cloud Storage bucket.

    gcloud auth login
    ACCESS_TOKEN="$(gcloud auth print-access-token)"
    curl --header "Authorization: Bearer ${ACCESS_TOKEN}" \
         --header 'Content-Type: application/json' \
         --data @JSON_PATH \
         -X PATCH \
         https://sqladmin.googleapis.com/sql/v1beta4/projects/PROJECT-ID/instances/SOURCE_REPRESENTATION_INSTANCE

    gcloud auth login
    ACCESS_TOKEN="$(gcloud auth print-access-token)"
    curl --header "Authorization: Bearer ${ACCESS_TOKEN}" \
         --header 'Content-Type: application/json' \
         --data @./source.json \
         -X PATCH \
         https://sqladmin.googleapis.com/sql/v1beta4/projects/MyProject/instances/cloudsql-source-instance

After you have verified that you can replicate from the external server, you are ready to perform the replication.

During the initial import process, do not perform any DDL operations on the external server. Doing so could cause inconsistencies during the import. After the import process completes, the replica uses the binary logs on the external server to catch up to the current state of the external server.

Open a terminal, log in using gcloud, then enter the curl command to replicate from the external server.

  gcloud auth login
  ACCESS_TOKEN="$(gcloud auth print-access-token)"
  curl --header "Authorization: Bearer ${ACCESS_TOKEN}" \
       --header 'Content-Type: application/json' \
       --data '{
           "syncMode": "SYNC_MODE",
           "skipVerification": "SKIP_VERIFICATION"
         }' \
       -X POST \
       https://sqladmin.googleapis.com/sql/v1beta4/projects/PROJECT_ID/instances/REPLICA_INSTANCE/startExternalSync

    gcloud auth login
    ACCESS_TOKEN="$(gcloud auth print-access-token)"
    curl --header "Authorization: Bearer ${ACCESS_TOKEN}" \
         --header 'Content-Type: application/json' \
         --data '{
             "syncMode": "online",
             "skipVerification": false
           }' \
         -X POST \
         https://sqladmin.googleapis.com/sql/v1beta4/projects/MyProject/instances/replica-instance/startExternalSync

If you replicated from a file in a bucket, you can remove this file and bucket. See the Cloud Storage documentation for Deleting Objects and Deleting Buckets.

Once you start replication from the external server, you need to monitor replication and then complete your migration. To learn more, see Monitoring replication.

Make sure the external primary is available to connect. You can also modify the values of the net_read_timeout and net_write_timeout flags on the source instance to stop the error. For more information on the allowable values for these flags, see Configure database flags.

To learn more about using mysqldump flags for managed import migration, see Allowed and default initial sync flags

Make sure the replication flags such as binlog-do-db, binlog-ignore-db, replicate-do-db or replicate-ignore-db are not set in a conflicting way.

Run the command show master status on the primary instance to see the current settings.

• Check the replication metrics for your replica instance in the Cloud Monitoring section of the Google Cloud console.
• The errors from the MySQL IO thread or SQL thread can be found in Cloud Logging in the mysql.err log files.
• The error can also be found when connecting to the replica instance. Run the command SHOW SLAVE STATUS, and check for the following fields in the output:
  • Slave_IO_Running
  • Slave_SQL_Running
  • Last_IO_Error
  • Last_SQL_Error

Increase the disk size of the replica instance. You can either manually increase the disk size or enable auto storage increase.

When you verify your replication settings, logs are produced.

You can view these logs by following these steps:

1. Go to the Logs Viewer in the Google Cloud console.

   Go to the Logs Viewer

2. Select the Cloud SQL replica from the Instance dropdown.
3. Select the replication-setup.log log file.

If the Cloud SQL replica is unable to connect to the external server, confirm the following:

• Any firewall on the external server is configured to allow connections from the Cloud SQL replica's outgoing IP address.
• Your SSL/TLS configuration is correct.
• Your replication user, host, and password are correct.

• Learn about updating an instance.
• Learn about managing replicas.
• Learn about monitoring instances.
• Learn about promoting your Cloud SQL replica to promote the replica to a standalone instance and stop replicating from the external server.

RetroSearch is an open source project built by @garambo | Open a GitHub Issue

Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo

HTML: 3.2 | Encoding: UTF-8 | Version: 0.7.4