This page describes composite objects, which you create from existing objects without transferring additional object data. Composite objects are useful for making appends to an existing object, as well as for recreating objects that you uploaded as multiple components in parallel.
Compose operationThe compose operation concatenates the data in a given sequence of source objects to create a new object called a composite object. The source objects all must:
When you perform a composition:
The composite object that results from a composition:
When using gcloud storage to perform object composition, the object that results has a Content-Type set to match the Content-Type of the first source object.
There are several differences between the metadata of a composite object and the metadata of other objects:
Composite objects do not have an MD5 hash metadata field.
The ETag value of a composite object is not based on an MD5 hash, and client code should make no assumptions about composite object ETags except that they change whenever the underlying object changes per the IETF specification for HTTP/1.1.
Caution: Exercise caution when first using composite objects, since any clients expecting to find an MD5 digest within the ETag header may conclude that object data has been corrupted, which could trigger endless data retransmission attempts.Each composite object has a component count metadata field, which counts the number of non-composite objects contained within the composite object.
Cloud Storage uses CRC32C for integrity checking each source object at upload time and for allowing the caller to perform an integrity check of the resulting composite object when it is downloaded. CRC32C is an error detecting code that can be efficiently calculated from the CRC32C values of its components. Your application should use CRC32C as follows:
You can use the compose operation to perform limited object appends and edits.
You accomplish appending by uploading data to a temporary new object, composing the object you wish to append with this temporary object, optionally naming the output of the compose operation the same as the original object, and deleting the temporary object.
For example, in the gcloud CLI, the series of commands to append the string new data to an existing Cloud Storage object is the following:
$ echo 'new data' | gcloud storage cp - gs://bucket/temporary_object
$ gcloud storage objects compose gs://bucket/object_to_append gs://bucket/temporary_object \
gs://bucket/object_to_append
$ gcloud storage rm gs://bucket/temporary_object
You can also use composition to support a basic flavor of object editing. For example, you could compose an object X from the sequence {Y1, Y2, Y3}, replace the contents of Y2, and recompose X from those same components. Note that this requires that Y1, Y2, and Y3 be left undeleted, so you will be billed for those components as well as for the composite.
Caution: Compose operations create a new version of an object. When performing appends in a bucket with Object Versioning enabled, be sure that you properly manage the noncurrent version of the object that each append generates. Composite object contextsDuring a compose object operation, Cloud Storage merges all contexts (preview) from the source objects and attaches these contexts to the destination object. The contexts are merged to handle both unique and duplicate context keys, as described in the following sections.
Unique context keysIf source objects have unique context keys, Cloud Storage directly attaches these keys and their corresponding values to the destination object.
Consider the following example:
Source object A contexts: Department: Engineering, Status: Active
Source object B contexts: Owner: m_jones, Version: 1.1
After the compose operation, the destination object has the following combined contexts:
{
"contexts": {
"custom": {
"Department": {
"value": "Engineering",
"createTime": "2023-10-26T10:00:00.000Z",
"updateTime": "2023-10-26T10:00:00.000Z"
},
"Status": {
"value": "Active",
"createTime": "2023-10-26T10:00:00.000Z",
"updateTime": "2023-10-26T10:00:00.000Z"
},
"Owner": {
"value": "m_jones",
"createTime": "2023-10-26T10:00:00.000Z",
"updateTime": "2023-10-26T10:00:00.000Z"
},
"Version": {
"value": "1.1",
"createTime": "2023-10-26T10:00:00.000Z",
"updateTime": "2023-10-26T10:00:00.000Z"
}
}
}
}
Duplicate context keys
When multiple source objects have the same context key, the value from the last object processed by Cloud Storage overrides the values from any objects processed earlier.
For example, consider source objects processed in the following order:
Source object A
Source object B
Source object A contexts: Version: 1.0, ReleaseDate: 2024-01-15
Source object B contexts: Version: 1.1, Owner: m_jones
Both source objects have a Version key, but object A has Version: 1.0 and object B has Version: 1.1. Because Cloud Storage processes source object B after source object A, the Version value from source object B takes precedence and the final value is 1.1.
The destination object combines these contexts as follows:
{
"contexts": {
"custom": {
"Version": {
"value": "1.1",
"createTime": "2025-01-01T00:00:00.000Z",
"updateTime": "2025-01-01T00:00:00.000Z"
},
"ReleaseDate": {
"value": "2024-01-15",
"createTime": "2025-01-01T00:00:00.000Z",
"updateTime": "2025-01-01T00:00:00.000Z"
},
"Owner": {
"value": "m_jones",
"createTime": "2025-01-01T00:00:00.000Z",
"updateTime": "2025-01-01T00:00:00.000Z"
}
}
}
}
What's next
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.5