This document shows you how to create an external Application Load Balancer to route requests for static content to Cloud Storage buckets. After you configure a load balancer with the backend buckets, requests to URL paths that begin with /love-to-fetch
are sent to the us-east1
Cloud Storage bucket, and all other requests are sent to the europe-north1
Cloud Storage bucket, regardless of the user's region.
If your backends serve dynamic content over HTTP(S), consider using backend services instead of backend buckets.
If you are an existing user of the classic Application Load Balancer, make sure that you review
Migration overviewwhen you plan a new deployment with the global external Application Load Balancer.
Cloud Storage buckets as load balancer backendsAn external Application Load Balancer uses a URL map to direct traffic from specified URL paths to your backends.
In the following diagram, the load balancer sends traffic with a path of /love-to-fetch/
to a Cloud Storage bucket in the us-east1
region. All other requests go to a Cloud Storage bucket in the europe-north1
region.
By default, Cloud Storage uses the same cache that Cloud CDN uses. If you enable Cloud CDN on the backend bucket, you can use Cloud CDN controls on your content. Cloud CDN controls include, for example, cache modes, signed URLs, and invalidation. Cloud CDN also lets you cache large content (> 10 MB). If you don't enable Cloud CDN on your backend bucket, you can only use origin Cache-Control
headers to control caching for smaller content, as set by the Cloud Storage metadata.
Make sure that your setup meets the following prerequisites. If you are using the gcloud storage
utility, you can install it by using instructions in Discover object storage with the gcloud tool.
In the Google Cloud console, on the project selector page, select or create a Google Cloud project.
Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.Verify that billing is enabled for your Google Cloud project.
In the Google Cloud console, on the project selector page, select or create a Google Cloud project.
Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.Verify that billing is enabled for your Google Cloud project.
gcloud config set project PROJECT_ID
Replace PROJECT_ID
with the project that you are using for this guide.
export GOOGLE_CLOUD_PROJECT=PROJECT_IDPermissions
To follow this guide, you need to create Cloud Storage buckets and a load balancer in a project. You should be either a project owner or editor, or you should have the following Compute Engine IAM roles:
For more information, see the following guides:
Set up an SSL certificate resourceFor an HTTPS load balancer, create an SSL certificate resource as described in the following documentation:
We recommend using a Google-managed certificate.
This example assumes that you already have an SSL certificate resource named www-ssl-cert
.
The process for preparing your Cloud Storage buckets is as follows:
Create the buckets.
Copy content to the buckets.
Provide public access to the buckets.
In this example, you create two Cloud Storage buckets for the load balancer to access. For production deployments, we recommend that you choose a multi-region bucket, which automatically replicates objects across multiple Google Cloud regions. This can improve the availability of your content and improve failure tolerance across your application.
Note the names of the Cloud Storage buckets you create, as they're used later. In this guide, they're referred to as BUCKET_1_NAME and BUCKET_2_NAME.
ConsoleIn the Google Cloud console, go to the Cloud Storage Buckets page.
Click Create bucket.
In the Name your bucket box, enter a globally unique name that follows the naming guidelines.
Click Choose where to store your data.
Set Location type to Region.
Set Location to europe-north1. This is BUCKET_1_NAME in this guide.
Click Create.
Click Buckets to return to the Cloud Storage Buckets page. Use these instructions to create a second bucket, but set the Location to us-east1. This is BUCKET_2_NAME in this guide.
gcloud storage buckets create gs://BUCKET_1_NAME --project=PROJECT_ID --default-storage-class=standard --location=europe-north1 --uniform-bucket-level-access
gcloud storage buckets create gs://BUCKET_2_NAME --project=PROJECT_ID --default-storage-class=standard --location=us-east1 --uniform-bucket-level-access
Replace BUCKET_1_NAME
and BUCKET_2_NAME
with the names of the buckets that you want to create.
To create the buckets, use the google_storage_bucket
resource.
To learn how to apply or remove a Terraform configuration, see Basic Terraform commands.
Transfer content to your Cloud Storage bucketsSo you can test the setup later, copy the following images from a public Cloud Storage bucket to your own Cloud Storage buckets.
gcloudClick terminal Activate Cloud Shell.
Run the following commands in Cloud Shell, replacing the bucket name variables with your Cloud Storage bucket names:
gcloud storage cp gs://gcp-external-http-lb-with-bucket/three-cats.jpg gs://BUCKET_1_NAME/never-fetch/
gcloud storage cp gs://gcp-external-http-lb-with-bucket/two-dogs.jpg gs://BUCKET_2_NAME/love-to-fetch/Terraform
To copy items into the bucket, you can use the google_storage_bucket_object
resource.
Alternatively, use the null_resource
resource.
resource "null_resource" "upload_cat_image" { provisioner "local-exec" { command = "gcloud storage cp gs://gcp-external-http-lb-with-bucket/three-cats.jpg gs://${google_storage_bucket.bucket_1.name}/never-fetch/" } } resource "null_resource" "upload_dog_image" { provisioner "local-exec" { command = "gcloud storage cp gs://gcp-external-http-lb-with-bucket/two-dogs.jpg gs://${google_storage_bucket.bucket_2.name}/love-to-fetch/" } }
In the Google Cloud console, click Refresh on each bucket's details page to verify that the file has copied successfully.
Make your Cloud Storage buckets publicly readableWhen you make Cloud Storage buckets publicly readable, anyone on the internet can list and view their objects, and view their metadata (excluding ACLs). Don't include sensitive information in your public buckets.
To reduce the likelihood of accidental exposure of sensitive information, don't store public objects and sensitive data in the same bucket.
ConsoleTo grant all users access to view objects in your buckets, repeat the following procedure for each bucket:
In the Google Cloud console, go to the Cloud Storage Buckets page.
Click the bucket name, followed by the Permissions tab.
Click Add.
In the New principals box, enter allUsers
.
In the Select a role box, select Cloud Storage > Storage Object Viewer.
Click Save.
Click Allow public access.
To grant all users access to view objects in your buckets, run the following commands:
gcloud storage buckets add-iam-policy-binding gs://BUCKET_1_NAME --member=allUsers --role=roles/storage.objectViewer
gcloud storage buckets add-iam-policy-binding gs://BUCKET_2_NAME --member=allUsers --role=roles/storage.objectViewerTerraform
To grant all users access to view objects in your buckets, use the google_storage_bucket_iam_member
resource and specify the allUsers
member.
After you've set up your Cloud Storage buckets, you can reserve a global static external IP address that your audience uses to reach your load balancer.
This step is optional but recommended, as a static external IP address provides a single address to point your domain at.
Note: You can skip this step and have Google Cloud associate an ephemeral IP address with your load balancer's forwarding rule. An ephemeral IP address remains constant while the forwarding rule exists. If you need to delete the forwarding rule and re-add it, the forwarding rule might receive a new IP address. If needed, you can make an ephemeral IP address static. ConsoleIn the Google Cloud console, go to the External IP addresses page.
Click Reserve static address.
In the Name box, enter example-ip
.
Set the Network Service Tier to Premium.
Set the IP version to IPv4.
Set the Type to Global.
Click Reserve.
gcloud compute addresses create example-ip \ --network-tier=PREMIUM \ --ip-version=IPV4 \ --global
Note the IPv4 address that was reserved:
gcloud compute addresses describe example-ip \ --format="get(address)" \ --globalTerraform
To reserve an external IP address, use the google_compute_global_address
resource.
These instructions cover creating either an HTTP or HTTPS load balancer. To create an HTTPS load balancer you must add an SSL certificate resource to the load balancer's frontend. For more information, see the SSL certificates overview.
Console Select the load balancer typeIn the Google Cloud console, go to the Load balancing page.
http-lb
.Click Backend configuration.
Click the Backend services and backend buckets box, and then click Create a backend bucket.
In the Backend bucket name box, enter cats
.
In the Cloud Storage bucket box, click Browse.
Select BUCKET_1_NAME, and then click Select. Creating the cats
backend bucket first makes it the default, where all unmatched traffic requests are directed. You can't change a default backend bucket's redirect rules in the load balancer.
Click Create.
Use the same process to create a backend bucket named dogs
, and select BUCKET_2_NAME.
Click OK.
Routing rules determine how your traffic is directed. To configure routing, you'll set up host rules and path matchers, which are configuration components of an external Application Load Balancer's URL map. To set up the rules for this example:
dogs
, enter *
in the Hosts field, and /love-to-fetch/*
in the Paths field.Click Frontend configuration.
Verify that the following options are configured with these values:
Property Value (type a value or select an option as specified) Protocol HTTP Network Service Tier Premium IP version IPv4 IP address example-ip Port 80 Optional: HTTP keepalive timeout Enter a timeout value from 5 to 1200 seconds. The default value is 610 seconds.If you want to create an HTTPS load balancer instead of an HTTP load balancer, you must have an SSL certificate (gcloud compute ssl-certificates list
), and you must fill in the fields as follows:
www-ssl-cert
certificate you created in the Set up an SSL certificate resource section, or create a new certificate. Optional: Enable HTTP to HTTPS Redirect Use this checkbox to enable redirects.
Enabling this checkbox creates an additional partial HTTP load balancer that uses the same IP address as your HTTPS load balancer and redirects HTTP requests to your load balancer's HTTPS frontend.
This checkbox can only be selected when the HTTPS protocol is selected and a reserved IP address is used.
Click Done.
Click Review and finalize.
Review the Frontend, Host and path rules, and Backend buckets.
Click Create and wait for the load balancer to be created.
Click the name of the load balancer (http-lb).
Note the IP address of the load balancer for the next task. In this guide, it's referred to as IP_ADDRESS.
gcloud compute backend-buckets create cats \ --gcs-bucket-name=BUCKET_1_NAME
gcloud compute backend-buckets create dogs \ --gcs-bucket-name=BUCKET_2_NAMEConfigure the URL map
gcloud compute url-maps create http-lb \ --default-backend-bucket=cats
gcloud compute url-maps add-path-matcher http-lb \ --path-matcher-name=path-matcher-2 \ --new-hosts=* \ --backend-bucket-path-rules="/love-to-fetch/*=dogs" \ --default-backend-bucket=catsConfigure the target proxy
gcloud compute target-http-proxies create http-lb-proxy \ --http-keep-alive-timeout-sec=HTTP_KEEP_ALIVE_TIMEOUT_SEC \ --url-map=http-lb
Replace HTTP_KEEP_ALIVE_TIMEOUT_SEC
with the client HTTP keepalive timeout value from 5 to 1200 seconds. The default value is 610 seconds. This field is optional.
gcloud compute forwarding-rules create http-lb-forwarding-rule \ --load-balancing-scheme=EXTERNAL_MANAGED \ --network-tier=PREMIUM \ --address=example-ip \ --global \ --target-http-proxy=http-lb-proxy \ --ports=80Terraform
To create the load balancer, use the following Terraform resources.
Configure the backendTo create the backend, use the google_compute_backend_bucket
resource.
To create the URL map, use the google_compute_url_map
resource.
To create the target HTTP proxy, use the google_compute_target_http_proxy
resource.
To create the forwarding rule, use the google_compute_global_forwarding_rule
resource.
NOTE: To change the mode to classic Application Load Balancer, set the load_balancing_scheme
attribute to "EXTERNAL"
instead of "EXTERNAL_MANAGED"
.
To learn how to apply or remove a Terraform configuration, see Basic Terraform commands.
Send traffic to your load balancerSeveral minutes after you have configured your load balancer, you can start sending traffic to the load balancer's IP address.
ConsoleThe Google Cloud console isn't supported.
gcloudUse the curl
command to test the response from the following URLs. Replace IP_ADDRESS
with the load balancer's IPv4 address:
curl http://IP_ADDRESS/love-to-fetch/two-dogs.jpg
curl http://IP_ADDRESS/never-fetch/three-cats.jpgAdditional configuration
This section expands on the configuration example to provide alternative and additional configuration options. All of the tasks are optional. You can perform them in any order.
Update client HTTP keepalive timeoutThe load balancer created in the previous steps has been configured with a default value for the
client HTTP keepalive timeout.
To update the client HTTP keepalive timeout, use the following instructions.
ConsoleIn the Google Cloud console, go to the Load balancing page.
For an HTTP load balancer, update the target HTTP proxy by using the gcloud compute target-http-proxies update
command:
gcloud compute target-http-proxies update TARGET_HTTP_PROXY_NAME \ --http-keep-alive-timeout-sec=HTTP_KEEP_ALIVE_TIMEOUT_SEC \ --global
For an HTTPS load balancer, update the target HTTPS proxy by using the gcloud compute target-https-proxies update
command:
gcloud compute target-https-proxies update TARGET_HTTPS_PROXY_NAME \ --http-keep-alive-timeout-sec=HTTP_KEEP_ALIVE_TIMEOUT_SEC \ --global
Replace the following:
TARGET_HTTP_PROXY_NAME
: the name of the target HTTP proxy.TARGET_HTTPS_PROXY_NAME
: the name of the target HTTPS proxy.HTTP_KEEP_ALIVE_TIMEOUT_SEC
: the HTTP keepalive timeout value from 5 to 600 seconds.When certain query string parameters are included in requests sent to a backend bucket through an Application Load Balancer, the client receives an HTTP 404 response with an "Unsupported query parameter" error. This happens because the Cloud Storage XML API doesn't support these parameters when the request comes from an Application Load Balancer.
The following table summarizes how the Cloud Storage XML API responds to various query parameters when requests are routed through an Application Load Balancer. Parameters are grouped by observed behavior to help identify which are supported, ignored, or rejected in this context.
Parameter type Parameters Observed behavior Supported parametersgeneration
, prefix
, marker
, max-keys
Adding these parameters (with appropriate values) works as described in the Cloud Storage XML API documentation. The API returns a standard HTTP response. Ignored parameters acl
, billing
, compose
, delimiter
, encryption
, encryptionConfig
, response-content-disposition
, response-content-type
, tagging
, versions
, websiteConfig
Adding these parameters has no effect.
If the load balancer passes these parameters to Cloud Storage, the Cloud Storage XML API ignores them, and responds as though the parameters don't exist.
Rejected parameterscors
, lifecycle
, location
, logging
, storageClass
, versioning
The Cloud Storage XML API returns an "Unsupported query parameter" error. Limitations
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4