Stay organized with collections Save and categorize content based on your preferences.
This page describes how to create and customize a push queue, and how to examine the contents of a queue.
This API is supported for first-generation runtimes and can be used when upgrading to corresponding second-generation runtimes. If you are updating to the App Engine runtime, refer to the Upgrade to second-generation runtimes page to learn about your migration options for legacy bundled services. Using a queue configuration file to create queuesTo process a task, you must add it to a push queue. App Engine provides a default push queue, named default
, which is configured and ready to use with default settings. If you want, you can just add all your tasks to the default queue, without having to create and configure other queues.
To add queues or change the default configuration, edit the queue configuration file for your application, which you upload to App Engine. You can create up to 100 queues. Queues cannot be created dynamically.
This queue configuration file defines two queues:
queue:
- name: queue-blue
target: v2.task-module
rate: 5/s
- name: queue-red
rate: 1/s
To upload the file:
gcloud app deploy queue.yaml
Note: Your app should not assume App Engine processes tasks in a queue in the order in which they were enqueued.
All tasks added to queue-blue
are sent to the target module v2.task-module
. The refresh rate of queue-red
is changed from 5/s to 1/s. Tasks will be dequeued and sent to their targets at the rate of 1 task per second.
If you delete a queue, you must wait approximately 7 days before creating a new queue with the same name.
There are many other parameters that can be added to the configuration file to customize the behavior of a push queue. For more information, see the queue configuration file reference.
Note: If you are using a service account to manage authentication for deploying your queue configuration file, it needs to have theserviceusage.services.list
permission. This permission can be added either using the serviceusage.serviceUsageViewer
role or by creating a custom role with that permission. Defining the push queue processing rate
You can control the rate at which tasks are processed in each of your queues by defining other directives, such as rate
, bucket_size
, and max_concurrent_requests
.
The task queue uses token buckets to control the rate of task execution. Each named queue has a token bucket that holds tokens, up to the maximum specified by the bucket_size
, or a maximum of 5 tokens if you don't specify the bucket size.
Each time your application executes a task, a token is removed from the bucket. Your app continues processing tasks in the queue until the queue's bucket runs out of tokens. App Engine refills the bucket with new tokens continuously based on the rate
that you specified for the queue.
If your queue contains tasks to process, and the queue's bucket contains tokens, App Engine simultaneously processes as many tasks as there are tokens. This can lead to bursts of processing, consuming system resources and competing with user-serving requests.
If you want to prevent too many tasks from running at once or to prevent datastore contention, use max_concurrent_requests
.
The following sample shows how to set max_concurrent_requests
to limit tasks and also shows how to adjust the bucket size and rate based on your application's needs and available resources:
queue:
- name: queue-blue
rate: 20/s
bucket_size: 40
max_concurrent_requests: 10
Setting storage limits for all queues
You can use your queue configuration file to define the total amount of storage that task data can consume over all queues. To define the total storage limit, include an element named total_storage_limit
at the top level:
# Set the total storage limit for all queues to 120MB
total_storage_limit: 120M
queue:
- name: queue-blue
rate: 35/s
The value is a number followed by a unit: B
for bytes, K
for kilobytes, M
for megabytes, G
for gigabytes, T
for terabytes. For example, 100K
specifies a limit of 100 kilobytes. If adding a task would cause the queue to exceed its storage limit, the call to add the task will fail. The default limit is 500M
(500 megabytes) for free apps. For billed apps there is no limit until you explicitly set one. You can use this limit to protect your app from a fork bomb programming error in which each task adds multiple other tasks during its execution.
If your app is receiving errors for insufficient quota when adding tasks, increasing the total storage limit can help. If you are using this feature, we strongly recommend setting a limit that corresponds to the storage required for several days' worth of tasks. This allows the queues to be temporarily backed up and continue to accept new tasks while working through the backlog while still being protected from a fork bomb programming error.
Monitoring queues in the Google Cloud consoleIn the Google Cloud console, go to the Cloud Tasks page.
Note that if you go to the App Engine Task queue page, there will be instructions that guide you to the Cloud Tasks page. This update in the Google Cloud console does not change how Task queues function.
Enable the Cloud Tasks API.
Once you're in the Cloud Tasks page, you will see a list of all of the queues in the application. Clicking on a the name of a queue brings up the Queue Details page, which shows all of the tasks in the selected queue.
Learn about creating tasks.
Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see the Google Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.
Last updated 2025-08-07 UTC.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2025-08-07 UTC."],[[["App Engine utilizes push queues, including a default queue named `default`, to manage and process tasks, but users can create up to 100 custom queues via a queue configuration file."],["The processing rate of tasks in each queue can be customized using directives like `rate`, `bucket_size`, and `max_concurrent_requests`, allowing control over task execution and resource consumption."],["A `total_storage_limit` can be defined in the queue configuration file to restrict the cumulative storage used by task data across all queues, preventing issues like fork bombs and allowing better handling of task backlogs."],["The Google Cloud console's Cloud Tasks page provides a centralized location to view and monitor all push queues, offering detailed information about each queue and its tasks."],["If a service account is being used to manage authentication for deploying the queue configuration file, it must have the `serviceusage.services.list` permission, or a role that includes that permission."]]],[]]
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4