RetroSearch Browse

The ordered elements in the array.

Cell

Specifies (some of) the contents of a single row/column/timestamp of a table.

Fields timestamp_micros

int64

The cell's stored timestamp, which also uniquely identifies it within its column. Values are always expressed in microseconds, but individual tables may set a coarser granularity to further restrict the allowed values. For example, a table which specifies millisecond granularity will only allow values of timestamp_micros which are multiples of 1000.

value

bytes

The value stored in the cell. May contain any byte string, including the empty string, up to 100MiB in length.

labels[]

string

Labels applied to the cell by a RowFilter.

CheckAndMutateRowRequest

Request message for Bigtable.CheckAndMutateRow.

Fields table_name

string

Optional. The unique name of the table to which the conditional mutation should be applied.

Values are of the form projects/<project>/instances/<instance>/tables/<table>.

Authorization requires one or more of the following IAM permissions on the specified resource tableName:

bigtable.tables.mutateRows
bigtable.tables.readRows

authorized_view_name

string

Optional. The unique name of the AuthorizedView to which the conditional mutation should be applied.

Values are of the form projects/<project>/instances/<instance>/tables/<table>/authorizedViews/<authorized_view>.

Authorization requires one or more of the following IAM permissions on the specified resource authorizedViewName:

bigtable.tables.mutateRows
bigtable.authorizedViews.readRows

app_profile_id

string

This value specifies routing for replication. If not specified, the "default" application profile will be used.

row_key

bytes

Required. The key of the row to which the conditional mutation should be applied.

predicate_filter

The filter to be applied to the contents of the specified row. Depending on whether or not any results are yielded, either true_mutations or false_mutations will be executed. If unset, checks that the row contains any values at all.

true_mutations[]

Changes to be atomically applied to the specified row if predicate_filter yields at least one cell when applied to row_key. Entries are applied in order, meaning that earlier mutations can be masked by later ones. Must contain at least one entry if false_mutations is empty, and at most 100000.

false_mutations[]

Changes to be atomically applied to the specified row if predicate_filter does not yield any cells when applied to row_key. Entries are applied in order, meaning that earlier mutations can be masked by later ones. Must contain at least one entry if true_mutations is empty, and at most 100000.

CheckAndMutateRowResponse

Response message for Bigtable.CheckAndMutateRow.

Fields predicate_matched

bool

Whether or not the request's predicate_filter yielded any results for the specified row.

Column

Specifies (some of) the contents of a single row/column intersection of a table.

Fields qualifier

bytes

The unique key which identifies this column within its family. This is the same key that's used to identify the column in, for example, a RowFilter which sets its column_qualifier_regex_filter field. May contain any byte string, including the empty string, up to 16kiB in length.

cells[]

Cell

Must not be empty. Sorted in order of decreasing "timestamp_micros".

Describes a column in a Bigtable Query Language result set.

ColumnRange

Specifies a contiguous range of columns within a single column family. The range spans from <column_family>:<start_qualifier> to <column_family>:<end_qualifier>, where both bounds can be either inclusive or exclusive.

Fields family_name

string

The name of the column family within which this range falls.

Union field start_qualifier. The column qualifier at which to start the range (within column_family). If neither field is set, interpreted as the empty string, inclusive. start_qualifier can be only one of the following: start_qualifier_closed

bytes

Used when giving an inclusive lower bound for the range.

start_qualifier_open

bytes

Used when giving an exclusive lower bound for the range.

Union field end_qualifier. The column qualifier at which to end the range (within column_family). If neither field is set, interpreted as the infinite string, exclusive. end_qualifier can be only one of the following: end_qualifier_closed

bytes

Used when giving an inclusive upper bound for the range.

end_qualifier_open

bytes

Used when giving an exclusive upper bound for the range.

ExecuteQueryRequest

Request message for Bigtable.ExecuteQuery

Fields instance_name

string

Required. The unique name of the instance against which the query should be executed. Values are of the form projects/<project>/instances/<instance>

Authorization requires the following IAM permission on the specified resource instanceName:

bigtable.instances.executeQuery

app_profile_id

string

Optional. This value specifies routing for replication. If not specified, the default application profile will be used.

query
(deprecated)

string

This item is deprecated!

Required. The query string.

Exactly one of query and prepared_query is required. Setting both or neither is an INVALID_ARGUMENT.

prepared_query

bytes

A prepared query that was returned from PrepareQueryResponse.

Exactly one of query and prepared_query is required. Setting both or neither is an INVALID_ARGUMENT.

Setting this field also places restrictions on several other fields: - data_format must be empty. - validate_only must be false. - params must match the param_types set in the PrepareQueryRequest.

resume_token

bytes

Optional. If this request is resuming a previously interrupted query execution, resume_token should be copied from the last PartialResultSet yielded before the interruption. Doing this enables the query execution to resume where the last one left off. The rest of the request parameters must exactly match the request that yielded this token. Otherwise the request will fail.

params

map<string, Value>

Required. params contains string type keys and Bigtable type values that bind to placeholders in the query string. In query string, a parameter placeholder consists of the @ character followed by the parameter name (for example, @firstName) in the query string.

For example, if params["firstName"] = bytes_value: "foo" type {bytes_type {}} then @firstName will be replaced with googlesql bytes value "foo" in the query string during query evaluation.

If Value.kind is not set, the value is treated as a NULL value of the given type. For example, if params["firstName"] = type {string_type {}} then @firstName will be replaced with googlesql null string.

If query is set, any empty Value.type in the map will be rejected with INVALID_ARGUMENT.

If prepared_query is set, any empty Value.type in the map will be inferred from the param_types in the PrepareQueryRequest. Any non-empty Value.type must match the corresponding param_types entry, or be rejected with INVALID_ARGUMENT.

Union field data_format. Requested data format for the response.

If prepared_query is set, then the data_format is fixed by the PrepareQueryRequest, and a non-empty data_format in the ExecuteQueryRequest will be rejected with INVALID_ARGUMENT. data_format can be only one of the following:

proto_format
(deprecated)

ProtoFormat

This item is deprecated!

Protocol buffer format as described by ProtoSchema and ProtoRows messages.

ExecuteQueryResponse

Response message for Bigtable.ExecuteQuery

Fields Union field response. The first response streamed from the server is of type ResultSetMetadata and includes information about the columns and types of the result set. From there on, we stream PartialResultSet messages with no additional information. PartialResultSet will contain resume_token to restart the response if query interrupts. In case of resumption with resume_token, the server will not resend the ResultSetMetadata. response can be only one of the following: metadata

ResultSetMetadata

Structure of rows in this response stream. The first (and only the first) response streamed from the server will be of this type.

results

PartialResultSet

A partial result set with row data potentially including additional instructions on how recent past and future partial responses should be interpreted.

Family

Specifies (some of) the contents of a single row/column family intersection of a table.

Fields name

string

The unique key which identifies this family within its row. This is the same key that's used to identify the family in, for example, a RowFilter which sets its "family_name_regex_filter" field. Must match [-_.a-zA-Z0-9]+, except that AggregatingRowProcessors may produce cells in a sentinel family with an empty name. Must be no greater than 64 characters in length.

columns[]

Column

Must not be empty. Sorted in order of increasing "qualifier".

FullReadStatsView

FullReadStatsView captures all known information about a read.

Fields read_iteration_stats

ReadIterationStats

Iteration stats describe how efficient the read is, e.g. comparing rows seen vs. rows returned or cells seen vs cells returned can provide an indication of read efficiency (the higher the ratio of seen to retuned the better).

request_latency_stats

RequestLatencyStats

Request latency stats describe the time taken to complete a request, from the server side.

GenerateInitialChangeStreamPartitionsRequest

NOTE: This API is intended to be used by Apache Beam BigtableIO. Request message for Bigtable.GenerateInitialChangeStreamPartitions.

Fields table_name

string

Required. The unique name of the table from which to get change stream partitions. Values are of the form projects/<project>/instances/<instance>/tables/<table>. Change streaming must be enabled on the table.

Authorization requires the following IAM permission on the specified resource tableName:

bigtable.tables.readRows

app_profile_id

string

This value specifies routing for replication. If not specified, the "default" application profile will be used. Single cluster routing must be configured on the profile.

GenerateInitialChangeStreamPartitionsResponse

NOTE: This API is intended to be used by Apache Beam BigtableIO. Response message for Bigtable.GenerateInitialChangeStreamPartitions.

MutateRowRequest

Request message for Bigtable.MutateRow.

Fields table_name

string

Optional. The unique name of the table to which the mutation should be applied.

Values are of the form projects/<project>/instances/<instance>/tables/<table>.

Authorization requires the following IAM permission on the specified resource tableName:

bigtable.tables.mutateRows

authorized_view_name

string

Optional. The unique name of the AuthorizedView to which the mutation should be applied.

Values are of the form projects/<project>/instances/<instance>/tables/<table>/authorizedViews/<authorized_view>.

Authorization requires the following IAM permission on the specified resource authorizedViewName:

bigtable.authorizedViews.mutateRows

app_profile_id

string

This value specifies routing for replication. If not specified, the "default" application profile will be used.

row_key

bytes

Required. The key of the row to which the mutation should be applied.

mutations[]

Required. Changes to be atomically applied to the specified row. Entries are applied in order, meaning that earlier mutations can be masked by later ones. Must contain at least one entry and at most 100000.

MutateRowResponse

This type has no fields.

Response message for Bigtable.MutateRow.

MutateRowsRequest

Request message for BigtableService.MutateRows.

Fields table_name

string

Optional. The unique name of the table to which the mutations should be applied.

Values are of the form projects/<project>/instances/<instance>/tables/<table>.

Authorization requires the following IAM permission on the specified resource tableName:

bigtable.tables.mutateRows

authorized_view_name

string

Optional. The unique name of the AuthorizedView to which the mutations should be applied.

Values are of the form projects/<project>/instances/<instance>/tables/<table>/authorizedViews/<authorized_view>.

Authorization requires the following IAM permission on the specified resource authorizedViewName:

bigtable.authorizedViews.mutateRows

app_profile_id

string

This value specifies routing for replication. If not specified, the "default" application profile will be used.

entries[]

Entry

Required. The row keys and corresponding mutations to be applied in bulk. Each entry is applied as an atomic mutation, but the entries may be applied in arbitrary order (even between entries for the same row). At least one entry must be specified, and in total the entries can contain at most 100000 mutations.

Entry

A mutation for a given row.

Fields row_key

bytes

The key of the row to which the mutations should be applied.

mutations[]

Required. Changes to be atomically applied to the specified row. Mutations are applied in order, meaning that earlier mutations can be masked by later ones. You must specify at least one mutation.

MutateRowsResponse

Response message for BigtableService.MutateRows.

Fields entries[]

Entry

One or more results for Entries from the batch request.

rate_limit_info

RateLimitInfo

Information about how client should limit the rate (QPS). Primirily used by supported official Cloud Bigtable clients. If unset, the rate limit info is not provided by the server.

Entry

The result of applying a passed mutation in the original request.

Fields index

int64

The index into the original request's entries list of the Entry for which a result is being reported.

status

Status

The result of the request Entry identified by index. Depending on how requests are batched during execution, it is possible for one Entry to fail due to an error with another Entry. In the event that this occurs, the same error will be reported for both entries.

Mutation

Specifies a particular change to be made to the contents of a row.

Fields Union field mutation. Which of the possible Mutation types to apply. mutation can be only one of the following: set_cell

SetCell

Set a cell's value.

add_to_cell

AddToCell

Incrementally updates an Aggregate cell.

merge_to_cell

MergeToCell

Merges accumulated state to an Aggregate cell.

delete_from_column

DeleteFromColumn

Deletes cells from a column.

delete_from_family

DeleteFromFamily

Deletes cells from a column family.

delete_from_row

DeleteFromRow

Deletes cells from the entire row.

AddToCell

A Mutation which incrementally updates a cell in an Aggregate family.

Fields family_name

string

The name of the Aggregate family into which new data should be added. This must be a family with a value_type of Aggregate. Format: [-_.a-zA-Z0-9]+

column_qualifier

The qualifier of the column into which new data should be added. This must be a raw_value.

timestamp

The timestamp of the cell to which new data should be added. This must be a raw_timestamp_micros that matches the table's granularity.

input

The input value to be accumulated into the specified cell. This must be compatible with the family's value_type.input_type.

DeleteFromColumn

A Mutation which deletes cells from the specified column, optionally restricting the deletions to a given timestamp range.

Fields family_name

string

The name of the family from which cells should be deleted. Must match [-_.a-zA-Z0-9]+

column_qualifier

bytes

The qualifier of the column from which cells should be deleted. Can be any byte string, including the empty string.

time_range

TimestampRange

The range of timestamps within which cells should be deleted.

DeleteFromFamily

A Mutation which deletes all cells from the specified column family.

Fields family_name

string

The name of the family from which cells should be deleted. Must match [-_.a-zA-Z0-9]+

DeleteFromRow

This type has no fields.

A Mutation which deletes all cells from the containing row.

MergeToCell

A Mutation which merges accumulated state into a cell in an Aggregate family.

Fields family_name

string

The name of the Aggregate family into which new data should be added. This must be a family with a value_type of Aggregate. Format: [-_.a-zA-Z0-9]+

column_qualifier

The qualifier of the column into which new data should be added. This must be a raw_value.

timestamp

The timestamp of the cell to which new data should be added. This must be a raw_timestamp_micros that matches the table's granularity.

input

The input value to be merged into the specified cell. This must be compatible with the family's value_type.state_type. Merging NULL is allowed, but has no effect.

SetCell

A Mutation which sets the value of the specified cell.

Fields family_name

string

The name of the family into which new data should be written. Must match [-_.a-zA-Z0-9]+

column_qualifier

bytes

The qualifier of the column into which new data should be written. Can be any byte string, including the empty string.

timestamp_micros

int64

The timestamp of the cell into which new data should be written. Use -1 for current Bigtable server time. Otherwise, the client should set this value itself, noting that the default value is a timestamp of zero if the field is left unspecified. Values must match the granularity of the table (e.g. micros, millis).

value

bytes

The value to be written into the specified cell.

PartialResultSet

A partial result set from the streaming query API. Cloud Bigtable clients buffer partial results received in this message until a resume_token is received.

The pseudocode below describes how to buffer and parse a stream of PartialResultSet messages.

Having: - queue of row results waiting to be returned queue - extensible buffer of bytes buffer - a place to keep track of the most recent resume_token for each PartialResultSet p received { if p.reset { ensure queue is empty ensure buffer is empty } if p.estimated_batch_size != 0 { (optional) ensure buffer is sized to at least p.estimated_batch_size } if p.proto_rows_batch is set { append p.proto_rows_batch.bytes to buffer } if p.batch_checksum is set and buffer is not empty { validate the checksum matches the contents of buffer (see comments on batch_checksum) parse buffer as ProtoRows message, clearing buffer add parsed rows to end of queue } if p.resume_token is set { release results in queue save p.resume_token in resume_token } }

Fields resume_token

bytes

An opaque token sent by the server to allow query resumption and signal that the buffered values constructed from received partial_rows can be yielded to the caller. Clients can provide this token in a subsequent request to resume the result stream from the current point.

When resume_token is non-empty, the buffered values received from partial_rows since the last non-empty resume_token can be yielded to the callers, provided that the client keeps the value of resume_token and uses it on subsequent retries.

A resume_token may be sent without information in partial_rows to checkpoint the progress of a sparse query. Any previous partial_rows data should still be yielded in this case, and the new resume_token should be saved for future retries as normal.

A resume_token will only be sent on a boundary where there is either no ongoing result batch, or batch_checksum is also populated.

The server will also send a sentinel resume_token when last batch of partial_rows is sent. If the client retries the ExecuteQueryRequest with the sentinel resume_token, the server will emit it again without any data in partial_rows, then return OK.

reset

bool

If true, any data buffered since the last non-empty resume_token must be discarded before the other parts of this message, if any, are handled.

estimated_batch_size

int32

Estimated size of the buffer required to hold the next batch of results.

This value will be sent with the first partial_rows of a batch. That is, on the first partial_rows received in a stream, on the first message after a batch_checksum message, and any time reset is true.

The client can use this estimate to allocate a buffer for the next batch of results. This helps minimize the number of allocations required, though the buffer size may still need to be increased if the estimate is too low.

Union field partial_rows. Some rows of the result set in one of the supported formats.

Multiple PartialResultSet messages may be sent to represent a complete response. The client should buffer data constructed from the fields in partial_rows until a non-empty resume_token is received. Each sub-message documents the appropriate way to combine results. partial_rows can be only one of the following:

proto_rows_batch

ProtoRowsBatch

Partial rows in serialized ProtoRows format.

batch_checksum

uint32

CRC32C checksum of concatenated partial_rows data for the current batch.

When present, the buffered data from partial_rows forms a complete parseable message of the appropriate type.

The client should mark the end of a parseable message and prepare to receive a new one starting from the next PartialResultSet message. Clients must verify the checksum of the serialized batch before yielding it to the caller.

This does NOT mean the values can be yielded to the callers since a resume_token is required to safely do so.

If resume_token is non-empty and any data has been received since the last one, this field is guaranteed to be non-empty. In other words, clients may assume that a batch will never cross a resume_token boundary.

PingAndWarmRequest

Request message for client connection keep-alive and warming.

Fields name

string

Required. The unique name of the instance to check permissions for as well as respond. Values are of the form projects/<project>/instances/<instance>.

Authorization requires the following IAM permission on the specified resource name:

bigtable.instances.ping

app_profile_id

string

This value specifies routing for replication. If not specified, the "default" application profile will be used.

PingAndWarmResponse

This type has no fields.

Response message for Bigtable.PingAndWarm connection keepalive and warming.

PrepareQueryRequest

Request message for Bigtable.PrepareQuery

Fields instance_name

string

Required. The unique name of the instance against which the query should be executed. Values are of the form projects/<project>/instances/<instance>

Authorization requires the following IAM permission on the specified resource instanceName:

bigtable.instances.executeQuery

app_profile_id

string

Optional. This value specifies routing for preparing the query. Note that this app_profile_id is only used for preparing the query. The actual query execution will use the app profile specified in the ExecuteQueryRequest. If not specified, the default application profile will be used.

query

string

Required. The query string.

param_types

map<string, Type>

Required. param_types is a map of parameter identifier strings to their Types.

In query string, a parameter placeholder consists of the @ character followed by the parameter name (for example, @firstName) in the query string.

For example, if param_types["firstName"] = Bytes then @firstName will be a query parameter of type Bytes. The specific Value to be used for the query execution must be sent in ExecuteQueryRequest in the params map.

Union field data_format. Required. Requested data format for the response. Note that the selected data format is binding for all ExecuteQuery rpcs that use the prepared query. data_format can be only one of the following: proto_format

ProtoFormat

Protocol buffer format as described by ProtoSchema and ProtoRows messages.

PrepareQueryResponse

Response message for Bigtable.PrepareQueryResponse

Fields metadata

ResultSetMetadata

Structure of rows in the response stream of ExecuteQueryResponse for the returned prepared_query.

prepared_query

bytes

A serialized prepared query. Clients should treat this as an opaque blob of bytes to send in ExecuteQueryRequest.

valid_until

The time at which the prepared query token becomes invalid. A token may become invalid early due to changes in the data being read, but it provides a guideline to refresh query plans asynchronously.

ProtoFormat

This type has no fields.

Protocol buffers format descriptor, as described by Messages ProtoSchema and ProtoRows

ProtoRows

Rows represented in proto format.

This should be constructed by concatenating the batch_data from each of the relevant ProtoRowsBatch messages and parsing the result as a ProtoRows message.

Fields values[]

A proto rows message consists of a list of values. Every N complete values defines a row, where N is equal to the number of entries in the metadata.proto_schema.columns value received in the first response.

ProtoRowsBatch

A part of a serialized ProtoRows message.

Fields batch_data

bytes

Part of a serialized ProtoRows message. A complete, parseable ProtoRows message is constructed by concatenating batch_data from multiple ProtoRowsBatch messages. The PartialResultSet that contains the last part has complete_batch set to true.

ProtoSchema

ResultSet schema in proto format

RateLimitInfo

Information about how client should adjust the load to Bigtable.

Fields period

Duration

Time that clients should wait before adjusting the target rate again. If clients adjust rate too frequently, the impact of the previous adjustment may not have been taken into account and may over-throttle or under-throttle. If clients adjust rate too slowly, they will not be responsive to load changes on server side, and may over-throttle or under-throttle.

factor

double

If it has been at least one period since the last load adjustment, the client should multiply the current load by this value to get the new target load. For example, if the current load is 100 and factor is 0.8, the new target load should be 80. After adjusting, the client should ignore factor until another period has passed.

The client can measure its load using any unit that's comparable over time. For example, QPS can be used as long as each request involves a similar amount of work.

ReadChangeStreamRequest

NOTE: This API is intended to be used by Apache Beam BigtableIO. Request message for Bigtable.ReadChangeStream.

Fields table_name

string

Required. The unique name of the table from which to read a change stream. Values are of the form projects/<project>/instances/<instance>/tables/<table>. Change streaming must be enabled on the table.

Authorization requires the following IAM permission on the specified resource tableName:

bigtable.tables.readRows

app_profile_id

string

This value specifies routing for replication. If not specified, the "default" application profile will be used. Single cluster routing must be configured on the profile.

partition

StreamPartition

The partition to read changes from.

end_time

If specified, OK will be returned when the stream advances beyond this time. Otherwise, changes will be continuously delivered on the stream. This value is inclusive and will be truncated to microsecond granularity.

heartbeat_duration

Duration

If specified, the duration between Heartbeat messages on the stream. Otherwise, defaults to 5 seconds.

Union field start_from. Options for describing where we want to start reading from the stream. start_from can be only one of the following: start_time

StreamContinuationTokens

Start reading the stream at the specified timestamp. This timestamp must be within the change stream retention period, less than or equal to the current time, and after change stream creation, whichever is greater. This value is inclusive and will be truncated to microsecond granularity.

continuation_tokens

Tokens that describe how to resume reading a stream where reading previously left off. If specified, changes will be read starting at the the position. Tokens are delivered on the stream as part of Heartbeat and CloseStream messages.

If a single token is provided, the token's partition must exactly match the request's partition. If multiple tokens are provided, as in the case of a partition merge, the union of the token partitions must exactly cover the request's partition. Otherwise, INVALID_ARGUMENT will be returned.

ReadChangeStreamResponse

NOTE: This API is intended to be used by Apache Beam BigtableIO. Response message for Bigtable.ReadChangeStream.

Fields Union field stream_record. The data or control message on the stream. stream_record can be only one of the following: data_change

DataChange

A mutation to the partition.

heartbeat

Heartbeat

A periodic heartbeat message.

close_stream

CloseStream

An indication that the stream should be closed.

CloseStream

A message indicating that the client should stop reading from the stream. If status is OK and continuation_tokens & new_partitions are empty, the stream has finished (for example if there was an end_time specified). If continuation_tokens & new_partitions are present, then a change in partitioning requires the client to open a new stream for each token to resume reading. Example:

                                 [B,      D) ends
                                      |
                                      v
              new_partitions:  [A,  C) [C,  E)
continuation_tokens.partitions:  [B,C) [C,D)
                                 ^---^ ^---^
                                 ^     ^
                                 |     |
                                 |     StreamContinuationToken 2
                                 |
                                 StreamContinuationToken 1

To read the new partition [A,C), supply the continuation tokens whose ranges cover the new partition, for example ContinuationToken[A,B) & ContinuationToken[B,C).

Fields status

Status

The status of the stream.

continuation_tokens[]

StreamContinuationToken

If non-empty, contains the information needed to resume reading their associated partitions.

new_partitions[]

StreamPartition

If non-empty, contains the new partitions to start reading from, which are related to but not necessarily identical to the partitions for the above continuation_tokens.

DataChange

A message corresponding to one or more mutations to the partition being streamed. A single logical DataChange message may also be split across a sequence of multiple individual messages. Messages other than the first in a sequence will only have the type and chunks fields populated, with the final message in the sequence also containing done set to true.

Fields type

Type

The type of the mutation.

source_cluster_id

string

The cluster where the mutation was applied. Not set when type is GARBAGE_COLLECTION.

row_key

bytes

The row key for all mutations that are part of this DataChange. If the DataChange is chunked across multiple messages, then this field will only be set for the first message.

commit_timestamp

The timestamp at which the mutation was applied on the Bigtable server.

tiebreaker

int32

A value that lets stream consumers reconstruct Bigtable's conflict resolution semantics. https://cloud.google.com/bigtable/docs/writes#conflict-resolution In the event that the same row key, column family, column qualifier, timestamp are modified on different clusters at the same commit_timestamp, the mutation with the larger tiebreaker will be the one chosen for the eventually consistent state of the system.

chunks[]

MutationChunk

The mutations associated with this change to the partition. May contain complete mutations or chunks of a multi-message chunked DataChange record.

done

bool

When true, indicates that the entire DataChange has been read and the client can safely process the message.

token

string

An encoded position for this stream's partition to restart reading from. This token is for the StreamPartition from the request.

estimated_low_watermark

StreamContinuationToken

An estimate of the commit timestamp that is usually lower than or equal to any timestamp for a record that will be delivered in the future on the stream. It is possible that, under particular circumstances that a future record has a timestamp is is lower than a previously seen timestamp. For an example usage see https://beam.apache.org/documentation/basics/#watermarks

Type

The type of mutation.

Heartbeat

A periodic message with information that can be used to checkpoint the state of a stream.

Fields continuation_token

A token that can be provided to a subsequent ReadChangeStream call to pick up reading at the current stream position.

estimated_low_watermark

An estimate of the commit timestamp that is usually lower than or equal to any timestamp for a record that will be delivered in the future on the stream. It is possible that, under particular circumstances that a future record has a timestamp that is lower than a previously seen timestamp. For an example usage see https://beam.apache.org/documentation/basics/#watermarks

MutationChunk

A partial or complete mutation.

Fields chunk_info

ChunkInfo

If set, then the mutation is a SetCell with a chunked value across multiple messages.

mutation

If this is a continuation of a chunked message (chunked_value_offset > 0), ignore all fields except the SetCell's value and merge it with the previous message by concatenating the value fields.

ChunkInfo

Information about the chunking of this mutation. Only SetCell mutations can be chunked, and all chunks for a SetCell will be delivered contiguously with no other mutation types interleaved.

Fields chunked_value_size

int32

The total value size of all the chunks that make up the SetCell.

chunked_value_offset

int32

The byte offset of this chunk into the total value size of the mutation.

last_chunk

bool

When true, this is the last chunk of a chunked SetCell.

ReadIterationStats

ReadIterationStats captures information about the iteration of rows or cells over the course of a read, e.g. how many results were scanned in a read operation versus the results returned.

Fields rows_seen_count

int64

The rows seen (scanned) as part of the request. This includes the count of rows returned, as captured below.

rows_returned_count

int64

The rows returned as part of the request.

cells_seen_count

int64

The cells seen (scanned) as part of the request. This includes the count of cells returned, as captured below.

cells_returned_count

int64

The cells returned as part of the request.

ReadModifyWriteRowRequest

Request message for Bigtable.ReadModifyWriteRow.

Fields table_name

string

Optional. The unique name of the table to which the read/modify/write rules should be applied.

Values are of the form projects/<project>/instances/<instance>/tables/<table>.

Authorization requires the following IAM permission on the specified resource tableName:

bigtable.tables.mutateRows

authorized_view_name

string

Optional. The unique name of the AuthorizedView to which the read/modify/write rules should be applied.

Values are of the form projects/<project>/instances/<instance>/tables/<table>/authorizedViews/<authorized_view>.

Authorization requires the following IAM permission on the specified resource authorizedViewName:

bigtable.authorizedViews.mutateRows

app_profile_id

string

This value specifies routing for replication. If not specified, the "default" application profile will be used.

row_key

bytes

Required. The key of the row to which the read/modify/write rules should be applied.

rules[]

ReadModifyWriteRule

Required. Rules specifying how the specified row's contents are to be transformed into writes. Entries are applied in order, meaning that earlier rules will affect the results of later ones. At least one entry must be specified, and there can be at most 100000 rules.

ReadModifyWriteRowResponse

Response message for Bigtable.ReadModifyWriteRow.

Fields row

Row

A Row containing the new contents of all cells modified by the request.

ReadModifyWriteRule

Specifies an atomic read/modify/write operation on the latest value of the specified column.

Fields family_name

string

The name of the family to which the read/modify/write should be applied. Must match [-_.a-zA-Z0-9]+

column_qualifier

bytes

The qualifier of the column to which the read/modify/write should be applied. Can be any byte string, including the empty string.

Union field rule. The rule used to determine the column's new latest value from its current latest value. rule can be only one of the following: append_value

bytes

Rule specifying that append_value be appended to the existing value. If the targeted cell is unset, it will be treated as containing the empty string.

increment_amount

int64

Rule specifying that increment_amount be added to the existing value. If the targeted cell is unset, it will be treated as containing a zero. Otherwise, the targeted cell must contain an 8-byte value (interpreted as a 64-bit big-endian signed integer), or the entire request will fail.

ReadRowsRequest

Request message for Bigtable.ReadRows.

Fields table_name

string

Optional. The unique name of the table from which to read.

Values are of the form projects/<project>/instances/<instance>/tables/<table>.

Authorization requires the following IAM permission on the specified resource tableName:

bigtable.tables.readRows

authorized_view_name

string

Optional. The unique name of the AuthorizedView from which to read.

Values are of the form projects/<project>/instances/<instance>/tables/<table>/authorizedViews/<authorized_view>.

Authorization requires the following IAM permission on the specified resource authorizedViewName:

bigtable.authorizedViews.readRows

materialized_view_name

string

Optional. The unique name of the MaterializedView from which to read.

Values are of the form projects/<project>/instances/<instance>/materializedViews/<materialized_view>.

Authorization requires the following IAM permission on the specified resource materializedViewName:

bigtable.materializedViews.readRows

app_profile_id

string

This value specifies routing for replication. If not specified, the "default" application profile will be used.

rows

RowSet

The row keys and/or ranges to read sequentially. If not specified, reads from all rows.

filter

The filter to apply to the contents of the specified row(s). If unset, reads the entirety of each row.

rows_limit

int64

The read will stop after committing to N rows' worth of results. The default (zero) is to return all results.

request_stats_view

RequestStatsView

The view into RequestStats, as described above.

reversed

bool

Experimental API - Please note that this API is currently experimental and can change in the future.

Return rows in lexiographical descending order of the row keys. The row contents will not be affected by this flag.

Example result set:

[
  {key: "k2", "f:col1": "v1", "f:col2": "v1"},
  {key: "k1", "f:col1": "v2", "f:col2": "v2"}
]

RequestStatsView

The desired view into RequestStats that should be returned in the response.

See also: RequestStats message.

Enums REQUEST_STATS_VIEW_UNSPECIFIED The default / unset value. The API will default to the NONE option below. REQUEST_STATS_NONE Do not include any RequestStats in the response. This will leave the RequestStats embedded message unset in the response. REQUEST_STATS_FULL Include the full set of available RequestStats in the response, applicable to this read. ReadRowsResponse

Response message for Bigtable.ReadRows.

Fields chunks[]

CellChunk

A collection of a row's contents as part of the read request.

last_scanned_row_key

bytes

Optionally the server might return the row key of the last row it has scanned. The client can use this to construct a more efficient retry request if needed: any row keys or portions of ranges less than this row key can be dropped from the request. This is primarily useful for cases where the server has read a lot of data that was filtered out since the last committed row key, allowing the client to skip that work on a retry.

request_stats

RequestStats

If requested, return enhanced query performance statistics. The field request_stats is empty in a streamed response unless the ReadRowsResponse message contains request_stats in the last message of the stream. Always returned when requested, even when the read request returns an empty response.

CellChunk

Specifies a piece of a row's contents returned as part of the read response stream.

Fields row_key

bytes

The row key for this chunk of data. If the row key is empty, this CellChunk is a continuation of the same row as the previous CellChunk in the response stream, even if that CellChunk was in a previous ReadRowsResponse message.

family_name

StringValue

The column family name for this chunk of data. If this message is not present this CellChunk is a continuation of the same column family as the previous CellChunk. The empty string can occur as a column family name in a response so clients must check explicitly for the presence of this message, not just for family_name.value being non-empty.

qualifier

BytesValue

The column qualifier for this chunk of data. If this message is not present, this CellChunk is a continuation of the same column as the previous CellChunk. Column qualifiers may be empty so clients must check for the presence of this message, not just for qualifier.value being non-empty.

timestamp_micros

int64

labels[]

string

Labels applied to the cell by a RowFilter. Labels are only set on the first CellChunk per cell.

value

bytes

The value stored in the cell. Cell values can be split across multiple CellChunks. In that case only the value field will be set in CellChunks after the first: the timestamp and labels will only be present in the first CellChunk, even if the first CellChunk came in a previous ReadRowsResponse.

value_size

int32

If this CellChunk is part of a chunked cell value and this is not the final chunk of that cell, value_size will be set to the total length of the cell value. The client can use this size to pre-allocate memory to hold the full cell value.

Union field row_status. Signals to the client concerning previous CellChunks received. row_status can be only one of the following: reset_row

bool

Indicates that the client should drop all previous chunks for row_key, as it will be re-read from the beginning.

commit_row

bool

Indicates that the client can safely process all previous chunks for row_key, as its data has been fully read.

RequestLatencyStats

RequestLatencyStats provides a measurement of the latency of the request as it interacts with different systems over its lifetime, e.g. how long the request took to execute within a frontend server.

Fields frontend_server_latency

Duration

The latency measured by the frontend server handling this request, from when the request was received, to when this value is sent back in the response. For more context on the component that is measuring this latency, see: https://cloud.google.com/bigtable/docs/overview

Note: This value may be slightly shorter than the value reported into aggregate latency metrics in Monitoring for this request (https://cloud.google.com/bigtable/docs/monitoring-instance) as this value needs to be sent in the response before the latency measurement including that transmission is finalized.

Note: This value includes the end-to-end latency of contacting nodes in the targeted cluster, e.g. measuring from when the first byte arrives at the frontend server, to when this value is sent back as the last value in the response, including any latency incurred by contacting nodes, waiting for results from nodes, and finally sending results from nodes back to the caller.

RequestStats

RequestStats is the container for additional information pertaining to a single request, helpful for evaluating the performance of the sent request. Currently, the following method is supported: google.bigtable.v2.ReadRows

Fields

Union field stats_view. Information pertaining to each request type received. The type is chosen based on the requested view.

See the messages above for additional context. stats_view can be only one of the following:

full_read_stats_view

FullReadStatsView

Available with the ReadRowsRequest.RequestStatsView.REQUEST_STATS_FULL view, see package google.bigtable.v2.

ResponseParams

Response metadata proto

Fields zone_id

string

The cloud bigtable zone associated with the cluster.

cluster_id

string

Identifier for a cluster that represents set of bigtable resources.

Describes the structure of a Bigtable result set.

Row

Specifies the complete (requested) contents of a single row of a table. Rows which exceed 256MiB in size cannot be read in full.

Fields key

bytes

The unique key which identifies this row within its table. This is the same key that's used to identify the row in, for example, a MutateRowRequest. May contain any non-empty byte string up to 4KiB in length.

families[]

Family

May be empty, but only if the entire row is empty. The mutual ordering of column families is not specified.

RowFilter

Takes a row as input and produces an alternate view of the row based on specified rules. For example, a RowFilter might trim down a row to include just the cells from columns matching a given regular expression, or might return all the cells of a row but not their values. More complicated filters can be composed out of these components to express requests such as, "within every column of a particular family, give just the two most recent cells which are older than timestamp X."

There are two broad categories of RowFilters (true filters and transformers), as well as two ways to compose simple filters into more complex ones (chains and interleaves). They work as follows:

True filters alter the input row by excluding some of its cells wholesale from the output row. An example of a true filter is the value_regex_filter, which excludes cells whose values don't match the specified pattern. All regex true filters use RE2 syntax (https://github.com/google/re2/wiki/Syntax) in raw byte mode (RE2::Latin1), and are evaluated as full matches. An important point to keep in mind is that RE2(.) is equivalent by default to RE2([^\n]), meaning that it does not match newlines. When attempting to match an arbitrary byte, you should therefore use the escape sequence \C, which may need to be further escaped as \\C in your client language.
Transformers alter the input row by changing the values of some of its cells in the output, without excluding them completely. Currently, the only supported transformer is the strip_value_transformer, which replaces every cell's value with the empty string.
Chains and interleaves are described in more detail in the RowFilter.Chain and RowFilter.Interleave documentation.

The total serialized size of a RowFilter message must not exceed 20480 bytes, and RowFilters may not be nested within each other (in Chains or Interleaves) to a depth of more than 20.

Fields Union field filter. Which of the possible RowFilter types to apply. If none are set, this RowFilter returns all cells in the input row. filter can be only one of the following: chain

Chain

Applies several RowFilters to the data in sequence, progressively narrowing the results.

interleave

Interleave

Applies several RowFilters to the data in parallel and combines the results.

condition

Condition

Applies one of two possible RowFilters to the data based on the output of a predicate RowFilter.

sink

bool

ADVANCED USE ONLY. Hook for introspection into the RowFilter. Outputs all cells directly to the output of the read rather than to any parent filter. Consider the following example:

Chain(
  FamilyRegex("A"),
  Interleave(
    All(),
    Chain(Label("foo"), Sink())
  ),
  QualifierRegex("B")
)

                    A,A,1,w
                    A,B,2,x
                    B,B,4,z
                       |
                FamilyRegex("A")
                       |
                    A,A,1,w
                    A,B,2,x
                       |
          +------------+-------------+
          |                          |
        All()                    Label(foo)
          |                          |
       A,A,1,w              A,A,1,w,labels:[foo]
       A,B,2,x              A,B,2,x,labels:[foo]
          |                          |
          |                        Sink() --------------+
          |                          |                  |
          +------------+      x------+          A,A,1,w,labels:[foo]
                       |                        A,B,2,x,labels:[foo]
                    A,A,1,w                             |
                    A,B,2,x                             |
                       |                                |
               QualifierRegex("B")                      |
                       |                                |
                    A,B,2,x                             |
                       |                                |
                       +--------------------------------+
                       |
                    A,A,1,w,labels:[foo]
                    A,B,2,x,labels:[foo]  // could be switched
                    A,B,2,x               // could be switched

Despite being excluded by the qualifier filter, a copy of every cell that reaches the sink is present in the final result.

As with an Interleave, duplicate cells are possible, and appear in an unspecified mutual order. In this case we have a duplicate with column "A:B" and timestamp 2, because one copy passed through the all filter while the other was passed through the label and sink. Note that one copy has label "foo", while the other does not.

Cannot be used within the predicate_filter, true_filter, or false_filter of a Condition.

pass_all_filter

bool

Matches all cells, regardless of input. Functionally equivalent to leaving filter unset, but included for completeness.

block_all_filter

bool

Does not match any cells, regardless of input. Useful for temporarily disabling just part of a filter.

row_key_regex_filter

bytes

Matches only cells from rows whose keys satisfy the given RE2 regex. In other words, passes through the entire row when the key matches, and otherwise produces an empty row. Note that, since row keys can contain arbitrary bytes, the \C escape sequence must be used if a true wildcard is desired. The . character will not match the new line character \n, which may be present in a binary key.

row_sample_filter

double

Matches all cells from a row with probability p, and matches no cells from the row with probability 1-p.

family_name_regex_filter

string

Matches only cells from columns whose families satisfy the given RE2 regex. For technical reasons, the regex must not contain the : character, even if it is not being used as a literal. Note that, since column families cannot contain the new line character \n, it is sufficient to use . as a full wildcard when matching column family names.

column_qualifier_regex_filter

bytes

Matches only cells from columns whose qualifiers satisfy the given RE2 regex. Note that, since column qualifiers can contain arbitrary bytes, the \C escape sequence must be used if a true wildcard is desired. The . character will not match the new line character \n, which may be present in a binary qualifier.

column_range_filter

ColumnRange

Matches only cells from columns within the given range.

timestamp_range_filter

TimestampRange

Matches only cells with timestamps within the given range.

value_regex_filter

bytes

Matches only cells with values that satisfy the given regular expression. Note that, since cell values can contain arbitrary bytes, the \C escape sequence must be used if a true wildcard is desired. The . character will not match the new line character \n, which may be present in a binary value.

value_range_filter

ValueRange

Matches only cells with values that fall within the given range.

cells_per_row_offset_filter

int32

Skips the first N cells of each row, matching all subsequent cells. If duplicate cells are present, as is possible when using an Interleave, each copy of the cell is counted separately.

cells_per_row_limit_filter

int32

Matches only the first N cells of each row. If duplicate cells are present, as is possible when using an Interleave, each copy of the cell is counted separately.

cells_per_column_limit_filter

int32

Matches only the most recent N cells within each column. For example, if N=2, this filter would match column foo:bar at timestamps 10 and 9, skip all earlier cells in foo:bar, and then begin matching again in column foo:bar2. If duplicate cells are present, as is possible when using an Interleave, each copy of the cell is counted separately.

strip_value_transformer

bool

Replaces each cell's value with the empty string.

apply_label_transformer

string

Applies the given label to all cells in the output row. This allows the client to determine which results were produced from which part of the filter.

Values must be at most 15 characters in length, and match the RE2 pattern [a-z0-9\\-]+

Due to a technical limitation, it is not currently possible to apply multiple labels to a cell. As a result, a Chain may have no more than one sub-filter which contains a apply_label_transformer. It is okay for an Interleave to contain multiple apply_label_transformers, as they will be applied to separate copies of the input. This may be relaxed in the future.

Chain

A RowFilter which sends rows through several RowFilters in sequence.

Fields filters[]

The elements of "filters" are chained together to process the input row: in row -> f(0) -> intermediate row -> f(1) -> ... -> f(N) -> out row The full chain is executed atomically.

Condition

A RowFilter which evaluates one of two possible RowFilters, depending on whether or not a predicate RowFilter outputs any cells from the input row.

IMPORTANT NOTE: The predicate filter does not execute atomically with the true and false filters, which may lead to inconsistent or unexpected results. Additionally, Condition filters have poor performance, especially when filters are set for the false condition.

Fields predicate_filter

If predicate_filter outputs any cells, then true_filter will be evaluated on the input row. Otherwise, false_filter will be evaluated.

true_filter

The filter to apply to the input row if predicate_filter returns any results. If not provided, no results will be returned in the true case.

false_filter

The filter to apply to the input row if predicate_filter does not return any results. If not provided, no results will be returned in the false case.

Interleave

A RowFilter which sends each row to each of several component RowFilters and interleaves the results.

Fields filters[]

The elements of "filters" all process a copy of the input row, and the results are pooled, sorted, and combined into a single output row. If multiple cells are produced with the same column and timestamp, they will all appear in the output row in an unspecified mutual order. Consider the following example, with three filters:

                             input row
                                 |
       -----------------------------------------------------
       |                         |                         |
      f(0)                      f(1)                      f(2)
       |                         |                         |
1: foo,bar,10,x             foo,bar,10,z              far,bar,7,a
2: foo,blah,11,z            far,blah,5,x              far,blah,5,x
       |                         |                         |
       -----------------------------------------------------
                                 |
1:                      foo,bar,10,z   // could have switched with #2
2:                      foo,bar,10,x   // could have switched with #1
3:                      foo,blah,11,z
4:                      far,bar,7,a
5:                      far,blah,5,x   // identical to #6
6:                      far,blah,5,x   // identical to #5

All interleaved filters are executed atomically.

RowRange

Specifies a contiguous range of rows.

Fields Union field start_key. The row key at which to start the range. If neither field is set, interpreted as the empty string, inclusive. start_key can be only one of the following: start_key_closed

bytes

Used when giving an inclusive lower bound for the range.

start_key_open

bytes

Used when giving an exclusive lower bound for the range.

Union field end_key. The row key at which to end the range. If neither field is set, interpreted as the infinite row key, exclusive. end_key can be only one of the following: end_key_open

bytes

Used when giving an exclusive upper bound for the range.

end_key_closed

bytes

Used when giving an inclusive upper bound for the range.

RowSet

Specifies a non-contiguous set of rows.

Fields row_keys[]

bytes

Single rows included in the set.

row_ranges[]

RowRange

Contiguous row ranges included in the set.

SampleRowKeysRequest

Request message for Bigtable.SampleRowKeys.

Fields table_name

string

Optional. The unique name of the table from which to sample row keys.

Values are of the form projects/<project>/instances/<instance>/tables/<table>.

Authorization requires the following IAM permission on the specified resource tableName:

bigtable.tables.sampleRowKeys

authorized_view_name

string

Optional. The unique name of the AuthorizedView from which to sample row keys.

Values are of the form projects/<project>/instances/<instance>/tables/<table>/authorizedViews/<authorized_view>.

Authorization requires the following IAM permission on the specified resource authorizedViewName:

bigtable.authorizedViews.sampleRowKeys

materialized_view_name

string

Optional. The unique name of the MaterializedView from which to read.

Values are of the form projects/<project>/instances/<instance>/materializedViews/<materialized_view>.

Authorization requires the following IAM permission on the specified resource materializedViewName:

bigtable.materializedViews.sampleRowKeys

app_profile_id

string

This value specifies routing for replication. If not specified, the "default" application profile will be used.

SampleRowKeysResponse

Response message for Bigtable.SampleRowKeys.

Fields row_key

bytes

Sorted streamed sequence of sample row keys in the table. The table might have contents before the first row key in the list and after the last one, but a key containing the empty string indicates "end of table" and will be the last response given, if present. Note that row keys in this list may not have ever been written to or read from, and users should therefore not make any assumptions about the row key structure that are specific to their use case.

offset_bytes

int64

Approximate total storage space used by all rows in the table which precede row_key. Buffering the contents of all rows between two subsequent samples would require space roughly equal to the difference in their offset_bytes fields.

StreamContinuationToken

NOTE: This API is intended to be used by Apache Beam BigtableIO. The information required to continue reading the data from a StreamPartition from where a previous read left off.

Fields partition

StreamPartition

The partition that this token applies to.

token

string

An encoded position in the stream to restart reading from.

StreamContinuationTokens

NOTE: This API is intended to be used by Apache Beam BigtableIO. The information required to continue reading the data from multiple StreamPartitions from where a previous read left off.

StreamPartition

NOTE: This API is intended to be used by Apache Beam BigtableIO. A partition of a change stream.

Fields row_range

RowRange

The row range covered by this partition and is specified by [start_key_closed, end_key_open).

TimestampRange

Specified a contiguous range of microsecond timestamps.

Fields start_timestamp_micros

int64

Inclusive lower bound. If left empty, interpreted as 0.

end_timestamp_micros

int64

Exclusive upper bound. If left empty, interpreted as infinity.

Type

Type represents the type of data that is written to, read from, or stored in Bigtable. It is heavily based on the GoogleSQL standard to help maintain familiarity and consistency across products and features.

For compatibility with Bigtable's existing untyped APIs, each Type includes an Encoding which describes how to convert to or from the underlying data.

Each encoding can operate in one of two modes:

Sorted: In this mode, Bigtable guarantees that Encode(X) <= Encode(Y) if and only if X <= Y. This is useful anywhere sort order is important, for example when encoding keys.
Distinct: In this mode, Bigtable guarantees that if X != Y then Encode(X) != Encode(Y). However, the converse is not guaranteed. For example, both "{'foo': '1', 'bar': '2'}" and "{'bar': '2', 'foo': '1'}" are valid encodings of the same JSON value.

The API clearly documents which mode is used wherever an encoding can be configured. Each encoding also documents which values are supported in which modes. For example, when encoding INT64 as a numeric STRING, negative numbers cannot be encoded in sorted mode. This is because INT64(1) > INT64(-1), but STRING("-00001") > STRING("00001").

Fields Union field kind. The kind of type that this represents. kind can be only one of the following: bytes_type

Bytes

Bytes

string_type

String

String

int64_type

Int64

Int64

float32_type

Float32

Float32

float64_type

Float64

Float64

bool_type

Bool

Bool

timestamp_type

HyperLogLogPlusPlusUniqueCount

Timestamp

date_type

Date

Date

aggregate_type

Aggregate

Aggregate

struct_type

Struct

Struct

array_type

Array

Array

map_type

Map

Map

Aggregate

A value that combines incremental updates into a summarized value.

Data is never directly written or read using type Aggregate. Writes provide either the input_type or state_type, and reads always return the state_type .

Fields input_type

Type

Type of the inputs that are accumulated by this Aggregate. Use AddInput mutations to accumulate new inputs.

state_type

Type

Output only. Type that holds the internal accumulator state for the Aggregate. This is a function of the input_type and aggregator chosen.

Union field aggregator. Which aggregator function to use. The configured types must match. aggregator can be only one of the following: sum

Sum

Sum aggregator.

hllpp_unique_count

HyperLogLogPlusPlusUniqueCount aggregator.

max

Max

Max aggregator.

min

Min

Min aggregator.

HyperLogLogPlusPlusUniqueCount

This type has no fields.

Computes an approximate unique count over the input values. When using raw data as input, be careful to use a consistent encoding. Otherwise the same value encoded differently could count more than once, or two distinct values could count as identical. Input: Any, or omit for Raw State: TBD Special state conversions: Int64 (the unique count estimate)

Max

This type has no fields.

Computes the max of the input values. Allowed input: Int64 State: same as input

Min

This type has no fields.

Computes the min of the input values. Allowed input: Int64 State: same as input

Sum

This type has no fields.

Computes the sum of the input values. Allowed input: Int64 State: same as input

Array

An ordered list of elements of a given type. Values of type Array are stored in Value.array_value.

Fields element_type

Type

The type of the elements in the array. This must not be Array.

Bool

This type has no fields.

bool Values of type Bool are stored in Value.bool_value.

Bytes

Bytes Values of type Bytes are stored in Value.bytes_value.

Fields encoding

The encoding to use when converting to or from lower level types.

Encoding

Rules used to convert to or from lower level types.

Fields Union field encoding. Which encoding to use. encoding can be only one of the following: raw

Raw

Use Raw encoding.

Raw

Leaves the value as-is.

Sorted mode: all values are supported.

Distinct mode: all values are supported.

Fields escape_nulls

bool

If set, allows NULL values to be encoded as the empty string "".

The actual empty string, or any value which only contains the null byte 0x00, has one more null byte appended.

Date

This type has no fields.

Date Values of type Date are stored in Value.date_value.

Float32

This type has no fields.

Float32 Values of type Float32 are stored in Value.float_value.

Float64

This type has no fields.

Float64 Values of type Float64 are stored in Value.float_value.

Int64

Int64 Values of type Int64 are stored in Value.int_value.

Fields encoding

The encoding to use when converting to or from lower level types.

Encoding

Rules used to convert to or from lower level types.

Fields Union field encoding. Which encoding to use. encoding can be only one of the following: big_endian_bytes

BigEndianBytes

Use BigEndianBytes encoding.

ordered_code_bytes

OrderedCodeBytes

Use OrderedCodeBytes encoding.

BigEndianBytes

Encodes the value as an 8-byte big-endian two's complement value.

Sorted mode: non-negative values are supported.

Distinct mode: all values are supported.

Compatible with:

BigQuery BINARY encoding
HBase Bytes.toBytes
Java ByteBuffer.putLong() with ByteOrder.BIG_ENDIAN

Fields

bytes_type
(deprecated)

Bytes

This item is deprecated!

Deprecated: ignored if set.

OrderedCodeBytes

This type has no fields.

Encodes the value in a variable length binary format of up to 10 bytes. Values that are closer to zero use fewer bytes.

Sorted mode: all values are supported.

Distinct mode: all values are supported.

Map

A mapping of keys to values of a given type. Values of type Map are stored in a Value.array_value where each entry is another Value.array_value with two elements (the key and the value, in that order). Normally encoded Map values won't have repeated keys, however, clients are expected to handle the case in which they do. If the same key appears multiple times, the last value takes precedence.

Fields key_type

Type

The type of a map key. Only Bytes, String, and Int64 are allowed as key types.

value_type

Type

The type of the values in a map.

String

String Values of type String are stored in Value.string_value.

Fields encoding

The encoding to use when converting to or from lower level types.

Encoding

Rules used to convert to or from lower level types.

Fields Union field encoding. Which encoding to use. encoding can be only one of the following:

utf8_raw
(deprecated)

Utf8Raw

This item is deprecated!

Deprecated: if set, converts to an empty utf8_bytes.

utf8_bytes

Utf8Bytes

Use Utf8Bytes encoding.

Utf8Bytes

UTF-8 encoding.

Sorted mode: - All values are supported. - Code point order is preserved.

Distinct mode: all values are supported.

Compatible with:

BigQuery TEXT encoding
HBase Bytes.toBytes
Java String#getBytes(StandardCharsets.UTF_8)

Fields null_escape_char

string

Single-character escape sequence used to support NULL values.

If set, allows NULL values to be encoded as the empty string "".

The actual empty string, or any value where every character equals null_escape_char, has one more null_escape_char appended.

If null_escape_char is set and does not equal the ASCII null character 0x00, then the encoding will not support sorted mode.

Utf8Raw

This type has no fields.

This item is deprecated!

Deprecated: prefer the equivalent Utf8Bytes.

Struct

A structured data value, consisting of fields which map to dynamically typed values. Values of type Struct are stored in Value.array_value where entries are in the same order and number as field_types.

Fields fields[]

Field

The names and types of the fields in this struct.

encoding

The encoding to use when converting to or from lower level types.

Encoding

Rules used to convert to or from lower level types.

Fields Union field encoding. Which encoding to use. encoding can be only one of the following: singleton

Singleton

Use Singleton encoding.

delimited_bytes

DelimitedBytes

Use DelimitedBytes encoding.

ordered_code_bytes

OrderedCodeBytes

User OrderedCodeBytes encoding.

DelimitedBytes

Fields are encoded independently and concatenated with a configurable delimiter in between.

A struct with no fields defined is encoded as a single delimiter.

Sorted mode:

Fields are encoded in sorted mode.
Encoded field values must not contain any bytes <= delimiter[0]
Element-wise order is preserved: A < B if A[0] < B[0], or if A[0] == B[0] && A[1] < B[1], etc. Strict prefixes sort first.

Distinct mode:

Fields are encoded in distinct mode.
Encoded field values must not contain delimiter[0].

Fields delimiter

bytes

Byte sequence used to delimit concatenated fields. The delimiter must contain at least 1 character and at most 50 characters.

OrderedCodeBytes

This type has no fields.

Fields are encoded independently and concatenated with the fixed byte pair {0x00, 0x01} in between.

Any null (0x00) byte in an encoded field is replaced by the fixed byte pair {0x00, 0xFF}.

Fields that encode to the empty string "" have special handling:

If every field encodes to "", or if the STRUCT has no fields defined, then the STRUCT is encoded as the fixed byte pair {0x00, 0x00}.
Otherwise, the STRUCT only encodes until the last non-empty field, omitting any trailing empty fields. Any empty fields that aren't omitted are replaced with the fixed byte pair {0x00, 0x00}.

Examples:

STRUCT() -> "\00\00"
STRUCT("") -> "\00\00"
STRUCT("", "") -> "\00\00"
STRUCT("", "B") -> "\00\00" + "\00\01" + "B"
STRUCT("A", "") -> "A"
STRUCT("", "B", "") -> "\00\00" + "\00\01" + "B"
STRUCT("A", "", "C") -> "A" + "\00\01" + "\00\00" + "\00\01" + "C"

Since null bytes are always escaped, this encoding can cause size blowup for encodings like Int64.BigEndianBytes that are likely to produce many such bytes.

Sorted mode:

Fields are encoded in sorted mode.
All values supported by the field encodings are allowed
Element-wise order is preserved: A < B if A[0] < B[0], or if A[0] == B[0] && A[1] < B[1], etc. Strict prefixes sort first.

Distinct mode:

Fields are encoded in distinct mode.
All values supported by the field encodings are allowed.

Singleton

This type has no fields.

Uses the encoding of fields[0].type as-is. Only valid if fields.size == 1.

Field

A struct field and its type.

Fields field_name

string

The field name (optional). Fields without a field_name are considered anonymous and cannot be referenced by name.

type

Type

The type of values in this field.

Timestamp

Timestamp Values of type Timestamp are stored in Value.timestamp_value.

Fields encoding

The encoding to use when converting to or from lower level types.

Encoding

Rules used to convert to or from lower level types.

Fields Union field encoding. Which encoding to use. encoding can be only one of the following: unix_micros_int64

Encodes the number of microseconds since the Unix epoch using the given Int64 encoding. Values must be microsecond-aligned.

Compatible with:

Java Instant.truncatedTo() with ChronoUnit.MICROS

Value

Value represents a dynamically typed value. The typed fields in Value are used as a transport encoding for the actual value (which may be of a more complex type). See the documentation of the Type message for more details.

Fields type

Type

The verified Type of this Value, if it cannot be inferred.

Read results will never specify the encoding for type since the value will already have been decoded by the server. Furthermore, the type will be omitted entirely if it can be inferred from a previous response. The exact semantics for inferring type will vary, and are therefore documented separately for each read method.

When using composite types (Struct, Array, Map) only the outermost Value will specify the type. This top-level type will define the types for any nested Struct' fields,Arrayelements, orMapkey/value pairs. If a nestedValueprovides atype` on write, the request will be rejected with INVALID_ARGUMENT.

Union field kind. Options for transporting values within the protobuf type system. A given kind may support more than one type and vice versa. On write, this is roughly analogous to a GoogleSQL literal.

The value is NULL if none of the fields in kind is set. If type is also omitted on write, we will infer it based on the schema. kind can be only one of the following:

raw_value

bytes

Represents a raw byte sequence with no type information. The type field must be omitted.

raw_timestamp_micros

int64

Represents a raw cell timestamp with no type information. The type field must be omitted.

bytes_value

bytes

Represents a typed value transported as a byte sequence.

string_value

string

Represents a typed value transported as a string.

int_value

int64

Represents a typed value transported as an integer.

bool_value

bool

Represents a typed value transported as a boolean.

float_value

double

Represents a typed value transported as a floating point number.

timestamp_value