Stay organized with collections Save and categorize content based on your preferences.
DataFrame(
data=None,
index: vendored_pandas_typing.Axes | None = None,
columns: vendored_pandas_typing.Axes | None = None,
dtype: typing.Optional[
bigframes.dtypes.DtypeString | bigframes.dtypes.Dtype
] = None,
copy: typing.Optional[bool] = None,
*,
session: typing.Optional[bigframes.session.Session] = None
)
Two-dimensional, size-mutable, potentially heterogeneous tabular data.
Data structure also contains labeled axes (rows and columns). Arithmetic operations align on both row and column labels. Can be thought of as a dict-like container for Series objects. The primary pandas data structure.
Properties atAccess a single value for a row/column label pair.
axesReturn a list representing the axes of the DataFrame.
It has the row axis labels and column axis labels as the only members. They are returned in that order.
Examples:
>>> import bigframes.pandas as bpd
>>> bpd.options.display.progress_bar = None
>>> df = bpd.DataFrame({'col1': [1, 2], 'col2': [3, 4]})
>>> df.axes[1:]
[Index(['col1', 'col2'], dtype='object')]
columns
The column labels of the DataFrame.
dtypesReturn the dtypes in the DataFrame.
This returns a Series with the data type of each column. The result's index is the original DataFrame's columns. Columns with mixed types aren't supported yet in BigQuery DataFrames.
emptyIndicates whether Series/DataFrame is empty.
True if Series/DataFrame is entirely empty (no items), meaning any of the axes are of length 0.
Note: If Series/DataFrame contains only NA values, it is still not considered empty. Returns Type Descriptionbool
If Series/DataFrame is empty, return True, if not return False. iat
Access a single value for a row/column pair by integer position.
ilocPurely integer-location based indexing for selection by position.
indexThe index (row labels) of the DataFrame.
The index of a DataFrame is a series of labels that identify each row. The labels can be integers, strings, or any other hashable type. The index is used for label-based access and alignment, and can be accessed or modified using this attribute.
locAccess a group of rows and columns by label(s) or a boolean array.
.loc[]
is primarily label based, but may also be used with a boolean array.
Allowed inputs are:
5
or 'a'
, (note that 5
is interpreted as a label of the index, and never as an integer position along the index).['a', 'b', 'c']
.[True, False, True]
.'a':'f'
. Note: contrary to usual python slices, both the start and the stop are included.callable
function with one argument (the calling Series or DataFrame) that returns valid output for indexing (one of the above).NotImplementError
if the inputs are not supported. ndim
Return an int representing the number of axes / array dimensions.
Returns Type Descriptionint
Return 1 if Series. Otherwise return 2 if DataFrame. query_job
BigQuery job metadata for the most recent query.
shapeReturn a tuple representing the dimensionality of the DataFrame.
Examples:
>>> import bigframes.pandas as bpd
>>> bpd.options.display.progress_bar = None
>>> df = bpd.DataFrame({'col1': [1, 2, 3],
... 'col2': [4, 5, 6]})
>>> df.shape
(3, 2)
size
Return an int representing the number of elements in this object.
Returns Type Descriptionint
Return the number of rows if Series. Otherwise return the number of rows times number of columns if DataFrame. sql
Compiles this DataFrame's expression tree to SQL.
valuesReturn the values of DataFrame in the form of a NumPy array.
Examples:
>>> import bigframes.pandas as bpd
>>> bpd.options.display.progress_bar = None
>>> df = bpd.DataFrame({'col1': [1, 2], 'col2': [3, 4]})
>>> df.values
array([[1, 3],
[2, 4]], dtype=object)
Methods __array_ufunc__
__array_ufunc__(
ufunc: numpy.ufunc, method: str, *inputs, **kwargs
) -> bigframes.dataframe.DataFrame
Used to support numpy ufuncs. See: https://numpy.org/doc/stable/reference/ufuncs.html
__getitem____getitem__(
key: typing.Union[
typing.Hashable,
typing.Sequence[typing.Hashable],
pandas.core.indexes.base.Index,
bigframes.series.Series,
]
)
Gets the specified column(s) from the DataFrame.
__repr__Converts a DataFrame to a string. Calls to_pandas.
Only represents the first <xref uid="bigframes.options">bigframes.options</xref>.display.max_rows
.
__setitem__(
key: str, value: typing.Union[bigframes.series.Series, int, float, typing.Callable]
)
Modify or insert a column into the DataFrame.
Note: This does not modify the original table the DataFrame was derived from.
absabs() -> bigframes.dataframe.DataFrame
Return a Series/DataFrame with absolute numeric value of each element.
This function only applies to elements that are all numeric.
addadd(
other: float | int | bigframes.series.Series | DataFrame,
axis: str | int = "columns",
) -> DataFrame
Get addition of DataFrame and other, element-wise (binary operator +
).
Equivalent to dataframe + other
. With reverse version, radd
.
Among flexible wrappers (add
, sub
, mul
, div
, mod
, pow
) to arithmetic operators: +
, -
, *
, /
, //
, %
, **
.
>>> import bigframes.pandas as bpd
>>> bpd.options.display.progress_bar = None
>>> df = bpd.DataFrame({
... 'A': [1, 2, 3],
... 'B': [4, 5, 6],
... })
You can use method name:
>>> df['A'].add(df['B'])
0 5
1 7
2 9
dtype: Int64
You can also use arithmetic operator +
:
>>> df['A'] + (df['B'])
0 5
1 7
2 9
dtype: Int64
Parameters Name Description other
float, int, or Series
Any single or multiple element data structure, or list-like object.
axis
{0 or 'index', 1 or 'columns'}
Whether to compare by the index (0 or 'index') or columns. (1 or 'columns'). For Series input, axis to match Series index on.
Returns Type DescriptionDataFrame
DataFrame result of the arithmetic operation. add_prefix
add_prefix(prefix: str, axis: int | str | None = None) -> DataFrame
Prefix labels with string prefix
.
For Series, the row labels are prefixed. For DataFrame, the column labels are prefixed.
Parameters Name Descriptionprefix
str
The string to add before each label.
axis
int or str or None, default None
{{0 or 'index', 1 or 'columns', None}}
, default None. Axis to add prefix on.
add_suffix(suffix: str, axis: int | str | None = None) -> DataFrame
Suffix labels with string suffix
.
For Series, the row labels are suffixed. For DataFrame, the column labels are suffixed.
aggagg(func: str | typing.Sequence[str]) -> DataFrame | bigframes.series.Series
Aggregate using one or more operations over the specified axis.
Parameter Name Descriptionfunc
function
Function to use for aggregating the data. Accepted combinations are: string function name, list of function names, e.g. ['sum', 'mean']
.
aggregate(func: str | typing.Sequence[str]) -> DataFrame | bigframes.series.Series
Aggregate using one or more operations over the specified axis.
Parameter Name Descriptionfunc
function
Function to use for aggregating the data. Accepted combinations are: string function name, list of function names, e.g. ['sum', 'mean']
.
align(
other: typing.Union[bigframes.dataframe.DataFrame, bigframes.series.Series],
join: str = "outer",
axis: typing.Optional[typing.Union[str, int]] = None,
) -> typing.Tuple[
typing.Union[bigframes.dataframe.DataFrame, bigframes.series.Series],
typing.Union[bigframes.dataframe.DataFrame, bigframes.series.Series],
]
Align two objects on their axes with the specified join method.
Join method is specified for each axis Index.
Parameters Name Descriptionjoin
{{'outer', 'inner', 'left', 'right'}}, default 'outer'
Type of alignment to be performed. left: use only keys from left frame, preserve key order. right: use only keys from right frame, preserve key order. outer: use union of keys from both frames, sort keys lexicographically. inner: use intersection of keys from both frames, preserve the order of the left keys.
axis
allowed axis of the other object, default None
Align on index (0), columns (1), or both (None).
Returns Type Descriptiontuple of (DataFrame, type of other)
Aligned objects. all
all(
axis: typing.Union[str, int] = 0, *, bool_only: bool = False
) -> bigframes.series.Series
Return whether all elements are True, potentially over an axis.
Returns True unless there at least one element within a Series or along a DataFrame axis that is False or equivalent (e.g. zero or empty).
Parameters Name Descriptionaxis
{index (0), columns (1)}
Axis for the function to be applied on. For Series this parameter is unused and defaults to 0.
bool_only
bool. default False
Include only boolean columns.
anyany(
*, axis: typing.Union[str, int] = 0, bool_only: bool = False
) -> bigframes.series.Series
Return whether any element is True, potentially over an axis.
Returns False unless there is at least one element within a series or along a Dataframe axis that is True or equivalent (e.g. non-zero or non-empty).
Parameters Name Descriptionaxis
{index (0), columns (1)}
Axis for the function to be applied on. For Series this parameter is unused and defaults to 0.
bool_only
bool. default False
Include only boolean columns.
applyapply(func, *, args: typing.Tuple = (), **kwargs)
Apply a function along an axis of the DataFrame.
Objects passed to the function are Series objects whose index is the DataFrame's index (axis=0
) the final return type is inferred from the return type of the applied function.
func
function
Function to apply to each column or row.
args
tuple
Positional arguments to pass to func
in addition to the array/series.
pandas.Series or bigframes.DataFrame
Result of applying func
along the given axis of the DataFrame. applymap
applymap(
func, na_action: typing.Optional[str] = None
) -> bigframes.dataframe.DataFrame
Apply a function to a Dataframe elementwise.
This method applies a function that accepts and returns a scalar to every element of a DataFrame.
Note: In pandas 2.1.0, DataFrame.applymap is deprecated and renamed to DataFrame.map. Parameter Name Descriptionna_action
Optional[str], default None
{None, 'ignore'}
, default None. If ‘ignore’, propagate NaN values, without passing them to func.
bigframes.dataframe.DataFrame
Transformed DataFrame. assign
assign(**kwargs) -> bigframes.dataframe.DataFrame
Assign new columns to a DataFrame.
Returns a new object with all original columns in addition to new ones. Existing columns that are re-assigned will be overwritten.
Note: Assigning multiple columns within the sameassign
is possible. Later items in '**kwargs' may refer to newly created or modified columns in 'df'; items are computed and assigned into 'df' in order. Returns Type Description bigframes.dataframe.DataFrame
A new DataFrame with the new columns in addition to all the existing columns. astype
astype(
dtype: typing.Union[
typing.Literal[
"boolean",
"Float64",
"Int64",
"string",
"string[pyarrow]",
"timestamp[us, tz=UTC][pyarrow]",
"timestamp[us][pyarrow]",
"date32[day][pyarrow]",
"time64[us][pyarrow]",
],
pandas.core.arrays.boolean.BooleanDtype,
pandas.core.arrays.floating.Float64Dtype,
pandas.core.arrays.integer.Int64Dtype,
pandas.core.arrays.string_.StringDtype,
pandas.core.dtypes.dtypes.ArrowDtype,
]
) -> bigframes.dataframe.DataFrame
Cast a pandas object to a specified dtype dtype
.
dtype
str or pandas.ExtensionDtype
A dtype supported by BigQuery DataFrame include 'boolean','Float64','Int64', 'string', 'tring[pyarrow]','timestamp[us, tz=UTC][pyarrow]', 'timestampus][pyarrow]
','date32day][pyarrow]
','time64us][pyarrow]
' A pandas.ExtensionDtype include pandas.BooleanDtype(), pandas.Float64Dtype(), pandas.Int64Dtype(), pandas.StringDtype(storage="pyarrow"), pd.ArrowDtype(pa.date32()), pd.ArrowDtype(pa.time64("us")), pd.ArrowDtype(pa.timestamp("us")), pd.ArrowDtype(pa.timestamp("us", tz="UTC")).
bfill(*, limit: typing.Optional[int] = None) -> bigframes.dataframe.DataFrame
Fill NA/NaN values by using the next valid observation to fill the gap.
Returns Type DescriptionSeries/DataFrame or None
Object with missing values filled. combine
combine(
other: bigframes.dataframe.DataFrame,
func: typing.Callable[
[bigframes.series.Series, bigframes.series.Series], bigframes.series.Series
],
fill_value=None,
overwrite: bool = True,
*,
how: str = "outer"
) -> bigframes.dataframe.DataFrame
Perform column-wise combine with another DataFrame.
Combines a DataFrame with other
DataFrame using func
to element-wise combine columns. The row and column indexes of the resulting DataFrame will be the union of the two.
Examples:
>>> import bigframes.pandas as bpd
>>> bpd.options.display.progress_bar = None
>>> df1 = bpd.DataFrame({'A': [0, 0], 'B': [4, 4]})
>>> df2 = bpd.DataFrame({'A': [1, 1], 'B': [3, 3]})
>>> take_smaller = lambda s1, s2: s1 if s1.sum() < s2.sum() else s2
>>> df1.combine(df2, take_smaller)
A B
0 0 3
1 0 3
<BLANKLINE>
[2 rows x 2 columns]
Parameters Name Description other
DataFrame
The DataFrame to merge column-wise.
func
function
Function that takes two series as inputs and return a Series or a scalar. Used to merge the two dataframes column by columns.
fill_value
scalar value, default None
The value to fill NaNs with prior to passing any column to the merge func.
overwrite
bool, default True
If True, columns in self
that do not exist in other
will be overwritten with NaNs.
DataFrame
Combination of the provided DataFrames. combine_first
combine_first(other: bigframes.dataframe.DataFrame)
Update null elements with value in the same location in other
.
Combine two DataFrame objects by filling null values in one DataFrame with non-null values from other DataFrame. The row and column indexes of the resulting DataFrame will be the union of the two. The resulting dataframe contains the 'first' dataframe values and overrides the second one values where both first.loc[index, col] and second.loc[index, col] are not missing values, upon calling first.combine_first(second).
Examples:
>>> import bigframes.pandas as bpd
>>> bpd.options.display.progress_bar = None
>>> df1 = bpd.DataFrame({'A': [None, 0], 'B': [None, 4]})
>>> df2 = bpd.DataFrame({'A': [1, 1], 'B': [3, 3]})
>>> df1.combine_first(df2)
A B
0 1.0 3.0
1 0.0 4.0
<BLANKLINE>
[2 rows x 2 columns]
Parameter Name Description other
DataFrame
Provided DataFrame to use to fill null values.
Returns Type DescriptionDataFrame
The result of combining the provided DataFrame with the other object. copy
copy() -> bigframes.dataframe.DataFrame
Make a copy of this object's indices and data.
A new object will be created with a copy of the calling object's data and indices. Modifications to the data or indices of the copy will not be reflected in the original object.
countcount(*, numeric_only: bool = False) -> bigframes.series.Series
Count non-NA cells for each column or row.
The values None
, NaN
, NaT
, and optionally numpy.inf
(depending on pandas.options.mode.use_inf_as_na
) are considered NA.
numeric_only
bool, default False
Include only float
, int
or boolean
data.
bigframes.series.Series
For each column/row the number of non-NA/null entries. If level
is specified returns a DataFrame
. cummax
cummax() -> bigframes.dataframe.DataFrame
Return cumulative maximum over a DataFrame axis.
Returns a DataFrame of the same size containing the cumulative maximum.
Returns Type Descriptionbigframes.dataframe.DataFrame
Return cumulative maximum of DataFrame. cummin
cummin() -> bigframes.dataframe.DataFrame
Return cumulative minimum over a DataFrame axis.
Returns a DataFrame of the same size containing the cumulative minimum.
Returns Type Descriptionbigframes.dataframe.DataFrame
Return cumulative minimum of DataFrame. cumprod
cumprod() -> bigframes.dataframe.DataFrame
Return cumulative product over a DataFrame axis.
Returns a DataFrame of the same size containing the cumulative product.
Returns Type Descriptionbigframes.dataframe.DataFrame
Return cumulative product of DataFrame. cumsum
Return cumulative sum over a DataFrame axis.
Returns a DataFrame of the same size containing the cumulative sum.
Returns Type Descriptionbigframes.dataframe.DataFrame
Return cumulative sum of DataFrame. describe
describe() -> bigframes.dataframe.DataFrame
Generate descriptive statistics.
Descriptive statistics include those that summarize the central tendency, dispersion and shape of a dataset's distribution, excluding NaN
values.
Only supports numeric columns.
Note: Percentile values are approximates only. Note: For numeric data, the result's index will includecount
, mean
, std
, min
, max
as well as lower, 50
and upper percentiles. By default the lower percentile is 25
and the upper percentile is 75
. The 50
percentile is the same as the median. Returns Type Description bigframes.dataframe.DataFrame
Summary statistics of the Series or Dataframe provided. diff
diff(periods: int = 1) -> bigframes.dataframe.DataFrame
First discrete difference of element.
Calculates the difference of a DataFrame element compared with another element in the DataFrame (default is element in previous row).
Parameter Name Descriptionperiods
int, default 1
Periods to shift for calculating difference, accepts negative values.
Returns Type Descriptionbigframes.dataframe.DataFrame
First differences of the Series. div
div(
other: float | int | bigframes.series.Series | DataFrame,
axis: str | int = "columns",
) -> DataFrame
Get floating division of DataFrame and other, element-wise (binary operator /
).
Equivalent to dataframe / other
. With reverse version, rtruediv
.
Among flexible wrappers (add
, sub
, mul
, div
, mod
, pow
) to arithmetic operators: +
, -
, *
, /
, //
, %
, **
.
>>> import bigframes.pandas as bpd
>>> bpd.options.display.progress_bar = None
>>> df = bpd.DataFrame({
... 'A': [1, 2, 3],
... 'B': [4, 5, 6],
... })
You can use method name:
>>> df['A'].truediv(df['B'])
0 0.25
1 0.4
2 0.5
dtype: Float64
You can also use arithmetic operator /
:
>>> df['A'] / (df['B'])
0 0.25
1 0.4
2 0.5
dtype: Float64
Parameters Name Description other
float, int, or Series
Any single or multiple element data structure, or list-like object.
axis
{0 or 'index', 1 or 'columns'}
Whether to compare by the index (0 or 'index') or columns. (1 or 'columns'). For Series input, axis to match Series index on.
Returns Type DescriptionDataFrame
DataFrame result of the arithmetic operation. divide
divide(
other: float | int | bigframes.series.Series | DataFrame,
axis: str | int = "columns",
) -> DataFrame
Get floating division of DataFrame and other, element-wise (binary operator /
).
Equivalent to dataframe / other
. With reverse version, rtruediv
.
Among flexible wrappers (add
, sub
, mul
, div
, mod
, pow
) to arithmetic operators: +
, -
, *
, /
, //
, %
, **
.
>>> import bigframes.pandas as bpd
>>> bpd.options.display.progress_bar = None
>>> df = bpd.DataFrame({
... 'A': [1, 2, 3],
... 'B': [4, 5, 6],
... })
You can use method name:
>>> df['A'].truediv(df['B'])
0 0.25
1 0.4
2 0.5
dtype: Float64
You can also use arithmetic operator /
:
>>> df['A'] / (df['B'])
0 0.25
1 0.4
2 0.5
dtype: Float64
Parameters Name Description other
float, int, or Series
Any single or multiple element data structure, or list-like object.
axis
{0 or 'index', 1 or 'columns'}
Whether to compare by the index (0 or 'index') or columns. (1 or 'columns'). For Series input, axis to match Series index on.
Returns Type DescriptionDataFrame
DataFrame result of the arithmetic operation. dot
dot(other: _DataFrameOrSeries) -> _DataFrameOrSeries
Compute the matrix multiplication between the DataFrame and other.
This method computes the matrix product between the DataFrame and the values of an other Series or DataFrame.
It can also be called using self @ other
.
The dot method for Series computes the inner product, instead of the matrix product here.
Parameter Name Descriptionother
Series or DataFrame
The other object to compute the matrix product with.
dropdrop(
labels: typing.Optional[typing.Any] = None,
*,
axis: typing.Union[int, str] = 0,
index: typing.Optional[typing.Any] = None,
columns: typing.Optional[
typing.Union[typing.Hashable, typing.Sequence[typing.Hashable]]
] = None,
level: typing.Optional[typing.Union[str, int]] = None
) -> bigframes.dataframe.DataFrame
Drop specified labels from columns.
Remove columns by directly specifying column names.
Exceptions Type DescriptionKeyError
If any of the labels is not found in the selected axis. Returns Type Description bigframes.dataframe.DataFrame
DataFrame without the removed column labels. drop_duplicates
drop_duplicates(
subset: typing.Optional[
typing.Union[typing.Hashable, typing.Sequence[typing.Hashable]]
] = None,
*,
keep: str = "first"
) -> bigframes.dataframe.DataFrame
Return DataFrame with duplicate rows removed.
Considering certain columns is optional. Indexes, including time indexes are ignored.
Parameters Name Descriptionsubset
column label or sequence of labels, optional
Only consider certain columns for identifying duplicates, by default use all of the columns.
keep
{'first', 'last', False
}, default 'first'
Determines which duplicates (if any) to keep. - 'first' : Drop duplicates except for the first occurrence. - 'last' : Drop duplicates except for the last occurrence. - False
: Drop all duplicates.
bigframes.dataframe.DataFrame
DataFrame with duplicates removed droplevel
droplevel(level: LevelsType, axis: int | str = 0)
Return DataFrame with requested index / column level(s) removed.
Parameters Name Descriptionlevel
int, str, or list-like
If a string is given, must be the name of a level If list-like, elements must be names or positional indexes of levels.
axis
{0 or 'index', 1 or 'columns'}, default 0
Axis along which the level(s) is removed: * 0 or 'index': remove level(s) in column. * 1 or 'columns': remove level(s) in row.
Returns Type DescriptionDataFrame
DataFrame with requested index / column level(s) removed. dropna
dropna(
*, axis: int | str = 0, inplace: bool = False, how: str = "any", ignore_index=False
) -> DataFrame
Remove missing values.
Parameters Name Descriptionaxis
{0 or 'index', 1 or 'columns'}, default 'columns'
Determine if rows or columns which contain missing values are removed. * 0, or 'index' : Drop rows which contain missing values. * 1, or 'columns' : Drop columns which contain missing value.
how
{'any', 'all'}, default 'any'
Determine if row or column is removed from DataFrame, when we have at least one NA or all NA. * 'any' : If any NA values are present, drop that row or column. * 'all' : If all values are NA, drop that row or column.
ignore_index
bool, default False
If True
, the resulting axis will be labeled 0, 1, …, n - 1.
bigframes.dataframe.DataFrame
DataFrame with NA entries dropped from it. duplicated
duplicated(subset=None, keep: str = "first") -> bigframes.series.Series
Return boolean Series denoting duplicate rows.
Considering certain columns is optional.
Parameters Name Descriptionsubset
column label or sequence of labels, optional
Only consider certain columns for identifying duplicates, by default use all of the columns.
keep
{'first', 'last', False}, default 'first'
Determines which duplicates (if any) to mark. - first
: Mark duplicates as True
except for the first occurrence. - last
: Mark duplicates as True
except for the last occurrence. - False : Mark all duplicates as True
.
eq(other: typing.Any, axis: str | int = "columns") -> DataFrame
Get equal to of DataFrame and other, element-wise (binary operator eq
).
Among flexible wrappers (eq
, ne
, le
, lt
, ge
, gt
) to comparison operators.
Equivalent to ==
, !=
, <=
, <
, >=
, >
with support to choose axis (rows or columns) and level for comparison.
Examples:
>>> import bigframes.pandas as bpd
>>> bpd.options.display.progress_bar = None
You can use method name:
>>> df = bpd.DataFrame({'angles': [0, 3, 4],
... 'degrees': [360, 180, 360]},
... index=['circle', 'triangle', 'rectangle'])
>>> df["degrees"].eq(360)
circle True
triangle False
rectangle True
Name: degrees, dtype: boolean
You can also use arithmetic operator ==
:
Parameters Name Descriptiondf["degrees"] == 360 circle True triangle False rectangle True Name: degrees, dtype: boolean
other
scalar, sequence, Series, or DataFrame
Any single or multiple element data structure, or list-like object.
axis
{0 or 'index', 1 or 'columns'}, default 'columns'
Whether to compare by the index (0 or 'index') or columns (1 or 'columns').
equalsequals(
other: typing.Union[bigframes.series.Series, bigframes.dataframe.DataFrame]
) -> bool
Test whether two objects contain the same elements.
This function allows two Series or DataFrames to be compared against each other to see if they have the same shape and elements. NaNs in the same location are considered equal.
The row/column index do not need to have the same type, as long as the values are considered equal. Corresponding columns must be of the same dtype.
Parameter Name Descriptionother
Series or DataFrame
The other Series or DataFrame to be compared with the first.
Returns Type Descriptionbool
True if all elements are the same in both objects, False otherwise. expanding
expanding(min_periods: int = 1) -> bigframes.core.window.Window
Provide expanding window calculations.
Parameter Name Descriptionmin_periods
int, default 1
Minimum number of observations in window required to have a value; otherwise, result is np.nan
.
ffill(*, limit: typing.Optional[int] = None) -> bigframes.dataframe.DataFrame
Fill NA/NaN values by propagating the last valid observation to next valid.
Returns Type DescriptionSeries/DataFrame or None
Object with missing values filled. fillna
fillna(value=None) -> bigframes.dataframe.DataFrame
Fill NA/NaN values using the specified method.
Parameter Name Descriptionvalue
scalar, Series
Value to use to fill holes (e.g. 0), alternately a Series of values specifying which value to use for each index (for a Series) or column (for a DataFrame). Values not in the Series will not be filled. This value cannot be a list.
Returns Type DescriptionDataFrame
Object with missing values filled filter
filter(
items: typing.Optional[typing.Iterable] = None,
like: typing.Optional[str] = None,
regex: typing.Optional[str] = None,
axis: int | str | None = None,
) -> DataFrame
Subset the dataframe rows or columns according to the specified index labels.
Note that this routine does not filter a dataframe on its contents. The filter is applied to the labels of the index.
Parameters Name Descriptionitems
list-like
Keep labels from axis which are in items.
like
str
Keep labels from axis for which "like in label == True".
regex
str (regular expression)
Keep labels from axis for which re.search(regex, label) == True.
axis
{0 or 'index', 1 or 'columns', None}, default None
The axis to filter on, expressed either as an index (int) or axis name (str). By default this is the info axis, 'columns' for DataFrame. For Series
this parameter is unused and defaults to None
.
API documentation for first_valid_index
method.
floordiv(
other: float | int | bigframes.series.Series | DataFrame,
axis: str | int = "columns",
) -> DataFrame
Get integer division of DataFrame and other, element-wise (binary operator //
).
Equivalent to dataframe // other
. With reverse version, rfloordiv
.
Among flexible wrappers (add
, sub
, mul
, div
, mod
, pow
) to arithmetic operators: +
, -
, *
, /
, //
, %
, **
.
>>> import bigframes.pandas as bpd
>>> bpd.options.display.progress_bar = None
>>> df = bpd.DataFrame({
... 'A': [1, 2, 3],
... 'B': [4, 5, 6],
... })
You can use method name:
>>> df['A'].floordiv(df['B'])
0 0
1 0
2 0
dtype: Int64
You can also use arithmetic operator //
:
>>> df['A'] // (df['B'])
0 0
1 0
2 0
dtype: Int64
Parameters Name Description other
float, int, or Series
Any single or multiple element data structure, or list-like object.
axis
{0 or 'index', 1 or 'columns'}
Whether to compare by the index (0 or 'index') or columns. (1 or 'columns'). For Series input, axis to match Series index on.
Returns Type DescriptionDataFrame
DataFrame result of the arithmetic operation. ge
ge(other: typing.Any, axis: str | int = "columns") -> DataFrame
Get 'greater than or equal to' of DataFrame and other, element-wise (binary operator >=
).
Among flexible wrappers (eq
, ne
, le
, lt
, ge
, gt
) to comparison operators.
Equivalent to ==
, !=
, <=
, <
, >=
, >
with support to choose axis (rows or columns) and level for comparison.
NaN
values in floating point columns are considered different (i.e. NaN
!= NaN
). Examples:
>>> import bigframes.pandas as bpd
>>> bpd.options.display.progress_bar = None
You can use method name:
>>> df = bpd.DataFrame({'angles': [0, 3, 4],
... 'degrees': [360, 180, 360]},
... index=['circle', 'triangle', 'rectangle'])
>>> df["degrees"].ge(360)
circle True
triangle False
rectangle True
Name: degrees, dtype: boolean
You can also use arithmetic operator >=
:
>>> df["degrees"] >= 360
circle True
triangle False
rectangle True
Name: degrees, dtype: boolean
Parameters Name Description other
scalar, sequence, Series, or DataFrame
Any single or multiple element data structure, or list-like object.
axis
{0 or 'index', 1 or 'columns'}, default 'columns'
Whether to compare by the index (0 or 'index') or columns (1 or 'columns').
Returns Type DescriptionDataFrame
DataFrame of bool. The result of the comparison. get
Get item from object for given key (ex: DataFrame column).
Returns default value if not found.
groupbygroupby(
by: typing.Optional[
typing.Union[
typing.Hashable,
bigframes.series.Series,
typing.Sequence[typing.Union[typing.Hashable, bigframes.series.Series]],
]
] = None,
*,
level: typing.Optional[
typing.Union[str, int, typing.Sequence[typing.Union[str, int]]]
] = None,
as_index: bool = True,
dropna: bool = True
) -> bigframes.core.groupby.DataFrameGroupBy
Group DataFrame by columns.
A groupby operation involves some combination of splitting the object, applying a function, and combining the results. This can be used to group large amounts of data and compute operations on these groups.
Parameters Name Descriptionby
str, Sequence[str]
A label or list of labels may be passed to group by the columns in self
. Notice that a tuple is interpreted as a (single) key.
level
int, level name, or sequence of such, default None
If the axis is a MultiIndex (hierarchical), group by a particular level or levels. Do not specify both by
and level
.
as_index
bool, default True
Default True. Return object with group labels as the index. Only relevant for DataFrame input. as_index=False
is effectively "SQL-style" grouped output. This argument has no effect on filtrations such as head()
, tail()
, nth()
and in transformations.
dropna
bool, default True
Default True. If True, and if group keys contain NA values, NA values together with row/column will be dropped. If False, NA values will also be treated as the key in groups.
gtgt(other: typing.Any, axis: str | int = "columns") -> DataFrame
Get 'greater than' of DataFrame and other, element-wise (binary operator >
).
Among flexible wrappers (eq
, ne
, le
, lt
, ge
, gt
) to comparison operators.
Equivalent to ==
, !=
, <=
, <
, >=
, >
with support to choose axis (rows or columns) and level for comparison.
NaN
values in floating point columns are considered different (i.e. NaN
!= NaN
). Examples:
>>> import bigframes.pandas as bpd
>>> bpd.options.display.progress_bar = None
>>> df = bpd.DataFrame({'angles': [0, 3, 4],
... 'degrees': [360, 180, 360]},
... index=['circle', 'triangle', 'rectangle'])
>>> df["degrees"].gt(360)
circle False
triangle False
rectangle False
Name: degrees, dtype: boolean
You can also use arithmetic operator >
:
>>> df["degrees"] > 360
circle False
triangle False
rectangle False
Name: degrees, dtype: boolean
Parameters Name Description other
scalar, sequence, Series, or DataFrame
Any single or multiple element data structure, or list-like object.
axis
{0 or 'index', 1 or 'columns'}, default 'columns'
Whether to compare by the index (0 or 'index') or columns (1 or 'columns').
Returns Type DescriptionDataFrame
DataFrame of bool: The result of the comparison. head
head(n: int = 5) -> bigframes.dataframe.DataFrame
Return the first n
rows.
This function returns the first n
rows for the object based on position. It is useful for quickly testing if your object has the right type of data in it.
Not yet supported For negative values of n
, this function returns all rows except the last |n|
rows, equivalent to df[:n]
.
If n is larger than the number of rows, this function returns all rows.
Parameter Name Descriptionn
int, default 5
Default 5. Number of rows to select.
idxmaxidxmax() -> bigframes.series.Series
Return index of first occurrence of maximum over requested axis.
NA/null values are excluded.
Returns Type DescriptionSeries
Indexes of maxima along the specified axis. idxmin
idxmin() -> bigframes.series.Series
Return index of first occurrence of minimum over requested axis.
NA/null values are excluded.
Returns Type DescriptionSeries
Indexes of minima along the specified axis. isin
isin(values) -> bigframes.dataframe.DataFrame
Whether each element in the DataFrame is contained in values.
Parameter Name Descriptionvalues
iterable, or dict
The result will only be true at a location if all the labels match. If values
is a dict, the keys must be the column names, which must match.
DataFrame
DataFrame of booleans showing whether each element in the DataFrame is contained in values. isna
isna() -> bigframes.dataframe.DataFrame
Detect missing values.
Return a boolean same-sized object indicating if the values are NA. NA values get mapped to True values. Everything else gets mapped to False values. Characters such as empty strings ''
or numpy.inf
are not considered NA values.
isnull() -> bigframes.dataframe.DataFrame
Detect missing values.
Return a boolean same-sized object indicating if the values are NA. NA values get mapped to True values. Everything else gets mapped to False values. Characters such as empty strings ''
or numpy.inf
are not considered NA values.
Iterate over (column name, Series) pairs.
Iterates over the DataFrame columns, returning a tuple with the column name and the content as a Series.
Returns Type DescriptionIterator
Iterator of label, Series for each column. join
join(
other: bigframes.dataframe.DataFrame,
*,
on: typing.Optional[str] = None,
how: str = "left"
) -> bigframes.dataframe.DataFrame
Join columns of another DataFrame.
Join columns with other
DataFrame on index
how
{'left', 'right', 'outer', 'inner'}, default 'left'`
How to handle the operation of the two objects. left
: use calling frame's index (or column if on is specified) right
: use other
's index. outer
: form union of calling frame's index (or column if on is specified) with other
's index, and sort it lexicographically. inner
: form intersection of calling frame's index (or column if on is specified) with other
's index, preserving the order of the calling's one.
bigframes.dataframe.DataFrame
A dataframe containing columns from both the caller and other
. kurt
kurt(*, numeric_only: bool = False)
Return unbiased kurtosis over requested axis.
Kurtosis obtained using Fisher's definition of kurtosis (kurtosis of normal == 0.0). Normalized by N-1.
Parameter Name Descriptionnumeric_only
bool, default False
Include only float, int, boolean columns.
kurtosiskurtosis(*, numeric_only: bool = False)
Return unbiased kurtosis over requested axis.
Kurtosis obtained using Fisher's definition of kurtosis (kurtosis of normal == 0.0). Normalized by N-1.
Parameter Name Descriptionnumeric_only
bool, default False
Include only float, int, boolean columns.
lele(other: typing.Any, axis: str | int = "columns") -> DataFrame
Get 'less than or equal to' of dataframe and other, element-wise (binary operator <=
).
Among flexible wrappers (eq
, ne
, le
, lt
, ge
, gt
) to comparison operators.
Equivalent to ==
, !=
, <=
, <
, >=
, >
with support to choose axis (rows or columns) and level for comparison.
NaN
values in floating point columns are considered different (i.e. NaN
!= NaN
). Examples:
>>> import bigframes.pandas as bpd
>>> bpd.options.display.progress_bar = None
You can use method name:
>>> df = bpd.DataFrame({'angles': [0, 3, 4],
... 'degrees': [360, 180, 360]},
... index=['circle', 'triangle', 'rectangle'])
>>> df["degrees"].le(180)
circle False
triangle True
rectangle False
Name: degrees, dtype: boolean
You can also use arithmetic operator <=
:
>>> df["degrees"] <= 180
circle False
triangle True
rectangle False
Name: degrees, dtype: boolean
Parameters Name Description other
scalar, sequence, Series, or DataFrame
Any single or multiple element data structure, or list-like object.
axis
{0 or 'index', 1 or 'columns'}, default 'columns'
Whether to compare by the index (0 or 'index') or columns (1 or 'columns').
Returns Type DescriptionDataFrame
DataFrame of bool. The result of the comparison. lt
lt(other: typing.Any, axis: str | int = "columns") -> DataFrame
Get 'less than' of DataFrame and other, element-wise (binary operator <
).
Among flexible wrappers (eq
, ne
, le
, lt
, ge
, gt
) to comparison operators.
Equivalent to ==
, !=
, <=
, <
, >=
, >
with support to choose axis (rows or columns) and level for comparison.
NaN
values in floating point columns are considered different (i.e. NaN
!= NaN
). Examples:
>>> import bigframes.pandas as bpd
>>> bpd.options.display.progress_bar = None
You can use method name:
>>> df = bpd.DataFrame({'angles': [0, 3, 4],
... 'degrees': [360, 180, 360]},
... index=['circle', 'triangle', 'rectangle'])
>>> df["degrees"].lt(180)
circle False
triangle False
rectangle False
Name: degrees, dtype: boolean
You can also use arithmetic operator <
:
>>> df["degrees"] < 180
circle False
triangle False
rectangle False
Name: degrees, dtype: boolean
Parameters Name Description other
scalar, sequence, Series, or DataFrame
Any single or multiple element data structure, or list-like object.
axis
{0 or 'index', 1 or 'columns'}, default 'columns'
Whether to compare by the index (0 or 'index') or columns (1 or 'columns').
Returns Type DescriptionDataFrame
DataFrame of bool. The result of the comparison. map
map(func, na_action: typing.Optional[str] = None) -> bigframes.dataframe.DataFrame
Apply a function to a Dataframe elementwise.
This method applies a function that accepts and returns a scalar to every element of a DataFrame.
Note: In pandas 2.1.0, DataFrame.applymap is deprecated and renamed to DataFrame.map. Parameter Name Descriptionna_action
Optional[str], default None
{None, 'ignore'}
, default None. If ‘ignore’, propagate NaN values, without passing them to func.
bigframes.dataframe.DataFrame
Transformed DataFrame. max
max(
axis: typing.Union[str, int] = 0, *, numeric_only: bool = False
) -> bigframes.series.Series
Return the maximum of the values over the requested axis.
If you want the index of the maximum, use idxmax
. This is the equivalent of the numpy.ndarray
method argmax
.
axis
{index (0), columns (1)}
Axis for the function to be applied on. For Series this parameter is unused and defaults to 0.
numeric_only
bool. default False
Default False. Include only float, int, boolean columns.
meanmean(
axis: typing.Union[str, int] = 0, *, numeric_only: bool = False
) -> bigframes.series.Series
Return the mean of the values over the requested axis.
Parameters Name Descriptionaxis
{index (0), columns (1)}
Axis for the function to be applied on. For Series this parameter is unused and defaults to 0.
numeric_only
bool. default False
Default False. Include only float, int, boolean columns.
medianmedian(
*, numeric_only: bool = False, exact: bool = False
) -> bigframes.series.Series
Return the median of the values over the requested axis.
Parameters Name Descriptionnumeric_only
bool. default False
Default False. Include only float, int, boolean columns.
exact
bool. default False
Default False. Get the exact median instead of an approximate one. Note: exact=True
not yet supported.
melt(
id_vars: typing.Optional[typing.Iterable[typing.Hashable]] = None,
value_vars: typing.Optional[typing.Iterable[typing.Hashable]] = None,
var_name: typing.Optional[
typing.Union[typing.Hashable, typing.Sequence[typing.Hashable]]
] = None,
value_name: typing.Hashable = "value",
)
Unpivot a DataFrame from wide to long format, optionally leaving identifiers set.
This function is useful to massage a DataFrame into a format where one or more columns are identifier variables (id_vars
), while all other columns, considered measured variables (value_vars
), are "unpivoted" to the row axis, leaving just two non-identifier columns, 'variable' and 'value'.
merge(
right: bigframes.dataframe.DataFrame,
how: typing.Literal["inner", "left", "outer", "right"] = "inner",
on: typing.Optional[
typing.Union[typing.Hashable, typing.Sequence[typing.Hashable]]
] = None,
*,
left_on: typing.Optional[
typing.Union[typing.Hashable, typing.Sequence[typing.Hashable]]
] = None,
right_on: typing.Optional[
typing.Union[typing.Hashable, typing.Sequence[typing.Hashable]]
] = None,
sort: bool = False,
suffixes: tuple[str, str] = ("_x", "_y")
) -> bigframes.dataframe.DataFrame
Merge DataFrame objects with a database-style join.
The join is done on columns or indexes. If joining columns on columns, the DataFrame indexes will be ignored. Otherwise if joining indexes on indexes or indexes on a column or columns, the index will be passed on. When performing a cross merge, no column specifications to merge on are allowed.
Warning: If both key columns contain rows where the key is a null value, those rows will be matched against each other. This is different from usual SQL join behaviour and can lead to unexpected results. Parameters Name Descriptionon
label or list of labels
Columns to join on. It must be found in both DataFrames. Either on or left_on + right_on must be passed in.
left_on
label or list of labels
Columns to join on in the left DataFrame. Either on or left_on + right_on must be passed in.
right_on
label or list of labels
Columns to join on in the right DataFrame. Either on or left_on + right_on must be passed in.
Returns Type Descriptionbigframes.dataframe.DataFrame
A DataFrame of the two merged objects. min
min(
axis: typing.Union[str, int] = 0, *, numeric_only: bool = False
) -> bigframes.series.Series
Return the minimum of the values over the requested axis.
If you want the index of the minimum, use idxmin
. This is the equivalent of the numpy.ndarray
method argmin
.
axis
{index (0), columns (1)}
Axis for the function to be applied on. For Series this parameter is unused and defaults to 0.
numeric_only
bool, default False
Default False. Include only float, int, boolean columns.
modmod(
other: int | bigframes.series.Series | DataFrame, axis: str | int = "columns"
) -> DataFrame
Get modulo of DataFrame and other, element-wise (binary operator %
).
Equivalent to dataframe % other
. With reverse version, rmod
.
Among flexible wrappers (add
, sub
, mul
, div
, mod
, pow
) to arithmetic operators: +
, -
, *
, /
, //
, %
, **
.
>>> import bigframes.pandas as bpd
>>> bpd.options.display.progress_bar = None
>>> df = bpd.DataFrame({
... 'A': [1, 2, 3],
... 'B': [4, 5, 6],
... })
You can use method name:
>>> df['A'].mod(df['B'])
0 1
1 2
2 3
dtype: Int64
You can also use arithmetic operator %
:
>>> df['A'] % (df['B'])
0 1
1 2
2 3
dtype: Int64
Parameter Name Description axis
{0 or 'index', 1 or 'columns'}
Whether to compare by the index (0 or 'index') or columns. (1 or 'columns'). For Series input, axis to match Series index on.
Returns Type DescriptionDataFrame
DataFrame result of the arithmetic operation. mul
mul(
other: float | int | bigframes.series.Series | DataFrame,
axis: str | int = "columns",
) -> DataFrame
Get multiplication of DataFrame and other, element-wise (binary operator *
).
Equivalent to dataframe * other
. With reverse version, rmul
.
Among flexible wrappers (add
, sub
, mul
, div
, mod
, pow
) to arithmetic operators: +
, -
, *
, /
, //
, %
, **
.
>>> import bigframes.pandas as bpd
>>> bpd.options.display.progress_bar = None
>>> df = bpd.DataFrame({
... 'A': [1, 2, 3],
... 'B': [4, 5, 6],
... })
You can use method name:
>>> df['A'].mul(df['B'])
0 4
1 10
2 18
dtype: Int64
You can also use arithmetic operator *
:
>>> df['A'] * (df['B'])
0 4
1 10
2 18
dtype: Int64
Parameters Name Description other
float, int, or Series
Any single or multiple element data structure, or list-like object.
axis
{0 or 'index', 1 or 'columns'}
Whether to compare by the index (0 or 'index') or columns. (1 or 'columns'). For Series input, axis to match Series index on.
Returns Type DescriptionDataFrame
DataFrame result of the arithmetic operation. multiply
multiply(
other: float | int | bigframes.series.Series | DataFrame,
axis: str | int = "columns",
) -> DataFrame
Get multiplication of DataFrame and other, element-wise (binary operator *
).
Equivalent to dataframe * other
. With reverse version, rmul
.
Among flexible wrappers (add
, sub
, mul
, div
, mod
, pow
) to arithmetic operators: +
, -
, *
, /
, //
, %
, **
.
>>> import bigframes.pandas as bpd
>>> bpd.options.display.progress_bar = None
>>> df = bpd.DataFrame({
... 'A': [1, 2, 3],
... 'B': [4, 5, 6],
... })
You can use method name:
>>> df['A'].mul(df['B'])
0 4
1 10
2 18
dtype: Int64
You can also use arithmetic operator *
:
>>> df['A'] * (df['B'])
0 4
1 10
2 18
dtype: Int64
Parameters Name Description other
float, int, or Series
Any single or multiple element data structure, or list-like object.
axis
{0 or 'index', 1 or 'columns'}
Whether to compare by the index (0 or 'index') or columns. (1 or 'columns'). For Series input, axis to match Series index on.
Returns Type DescriptionDataFrame
DataFrame result of the arithmetic operation. ne
ne(other: typing.Any, axis: str | int = "columns") -> DataFrame
Get not equal to of DataFrame and other, element-wise (binary operator ne
).
Among flexible wrappers (eq
, ne
, le
, lt
, ge
, gt
) to comparison operators.
Equivalent to ==
, !=
, <=
, <
, >=
, >
with support to choose axis (rows or columns) and level for comparison.
Examples:
>>> import bigframes.pandas as bpd
>>> bpd.options.display.progress_bar = None
You can use method name:
>>> df = bpd.DataFrame({'angles': [0, 3, 4],
... 'degrees': [360, 180, 360]},
... index=['circle', 'triangle', 'rectangle'])
>>> df["degrees"].ne(360)
circle False
triangle True
rectangle False
Name: degrees, dtype: boolean
You can also use arithmetic operator !=
:
>>> df["degrees"] != 360
circle False
triangle True
rectangle False
Name: degrees, dtype: boolean
Parameters Name Description other
scalar, sequence, Series, or DataFrame
Any single or multiple element data structure, or list-like object.
axis
{0 or 'index', 1 or 'columns'}, default 'columns'
Whether to compare by the index (0 or 'index') or columns (1 or 'columns').
Returns Type DescriptionDataFrame
Result of the comparison. nlargest
nlargest(
n: int,
columns: typing.Union[typing.Hashable, typing.Sequence[typing.Hashable]],
keep: str = "first",
) -> bigframes.dataframe.DataFrame
Return the first n
rows ordered by columns
in descending order.
Return the first n
rows with the largest values in columns
, in descending order. The columns that are not specified are returned as well, but not used for ordering.
This method is equivalent to df.sort_values(columns, ascending=False).head(n)
, but more performant.
n
int
Number of rows to return.
columns
label or list of labels
Column label(s) to order by.
keep
{'first', 'last', 'all'}, default 'first'
Where there are duplicate values: - first
: prioritize the first occurrence(s) - last
: prioritize the last occurrence(s) - all
: do not drop any duplicates, even it means selecting more than n
items.
DataFrame .. note:: This function cannot be used with all column types. For example, when specifying columns with object
or category
dtypes, TypeError
is raised.
The first n
rows ordered by the given columns in descending order. notna
notna() -> bigframes.dataframe.DataFrame
Detect existing (non-missing) values.
Return a boolean same-sized object indicating if the values are not NA. Non-missing values get mapped to True. Characters such as empty strings ''
or numpy.inf
are not considered NA values. NA values get mapped to False values.
NDFrame
Mask of bool values for each element that indicates whether an element is not an NA value. notnull
notnull() -> bigframes.dataframe.DataFrame
Detect existing (non-missing) values.
Return a boolean same-sized object indicating if the values are not NA. Non-missing values get mapped to True. Characters such as empty strings ''
or numpy.inf
are not considered NA values. NA values get mapped to False values.
NDFrame
Mask of bool values for each element that indicates whether an element is not an NA value. nsmallest
nsmallest(
n: int,
columns: typing.Union[typing.Hashable, typing.Sequence[typing.Hashable]],
keep: str = "first",
) -> bigframes.dataframe.DataFrame
Return the first n
rows ordered by columns
in ascending order.
Return the first n
rows with the smallest values in columns
, in ascending order. The columns that are not specified are returned as well, but not used for ordering.
This method is equivalent to df.sort_values(columns, ascending=True).head(n)
, but more performant.
n
int
Number of rows to return.
columns
label or list of labels
Column label(s) to order by.
keep
{'first', 'last', 'all'}, default 'first'
Where there are duplicate values: - first
: prioritize the first occurrence(s) - last
: prioritize the last occurrence(s) - all
: do not drop any duplicates, even it means selecting more than n
items.
DataFrame .. note:: This function cannot be used with all column types. For example, when specifying columns with object
or category
dtypes, TypeError
is raised.
The first n
rows ordered by the given columns in ascending order. nunique
nunique() -> bigframes.series.Series
Count number of distinct elements in specified axis.
pct_changepct_change(periods: int = 1) -> bigframes.dataframe.DataFrame
Fractional change between the current and a prior element.
Computes the fractional change from the immediately previous row by default. This is useful in comparing the fraction of change in a time series of elements.
Note: Despite the name of this method, it calculates fractional change (also known as per unit change or relative change) and not percentage change. If you need the percentage change, multiply these values by 100. Parameter Name Descriptionperiods
int, default 1
Periods to shift for forming percent change.
Returns Type DescriptionSeries or DataFrame
The same type as the calling object. pivot
pivot(
*,
columns: typing.Union[typing.Hashable, typing.Sequence[typing.Hashable]],
index: typing.Optional[
typing.Union[typing.Hashable, typing.Sequence[typing.Hashable]]
] = None,
values: typing.Optional[
typing.Union[typing.Hashable, typing.Sequence[typing.Hashable]]
] = None
) -> bigframes.dataframe.DataFrame
Return reshaped DataFrame organized by given index / column values.
Reshape data (produce a "pivot" table) based on column values. Uses unique values from specified index
/ columns
to form axes of the resulting DataFrame. This function does not support data aggregation, multiple values will result in a MultiIndex in the columns.
columns
str or object or a list of str
Column to use to make new frame's columns.
index
str or object or a list of str, optional
Column to use to make new frame's index. If not given, uses existing index.
values
str, object or a list of the previous, optional
Column(s) to use for populating new frame's values. If not specified, all remaining columns will be used and the result will have hierarchically indexed columns.
powpow(other: int | bigframes.series.Series, axis: str | int = "columns") -> DataFrame
Get Exponential power of dataframe and other, element-wise (binary operator **
).
Equivalent to dataframe ** other
, but with support to substitute a fill_value for missing data in one of the inputs. With reverse version, rpow
.
Among flexible wrappers (add
, sub
, mul
, div
, mod
, pow
) to arithmetic operators: +
, -
, *
, /
, //
, %
, **
.
>>> import bigframes.pandas as bpd
>>> bpd.options.display.progress_bar = None
>>> df = bpd.DataFrame({
... 'A': [1, 2, 3],
... 'B': [4, 5, 6],
... })
You can use method name:
>>> df['A'].pow(df['B'])
0 1
1 32
2 729
dtype: Int64
You can also use arithmetic operator **
:
>>> df['A'] ** (df['B'])
0 1
1 32
2 729
dtype: Int64
Parameters Name Description other
float, int, or Series
Any single or multiple element data structure, or list-like object.
axis
{0 or 'index', 1 or 'columns'}
Whether to compare by the index (0 or 'index') or columns. (1 or 'columns'). For Series input, axis to match Series index on.
Returns Type DescriptionDataFrame
DataFrame result of the arithmetic operation. prod
prod(
axis: typing.Union[str, int] = 0, *, numeric_only: bool = False
) -> bigframes.series.Series
Return the product of the values over the requested axis.
Parameters Name Descriptionaßxis
{index (0), columns (1)}
Axis for the function to be applied on. For Series this parameter is unused and defaults to 0.
numeric_only
bool. default False
Include only float, int, boolean columns.
productproduct(
axis: typing.Union[str, int] = 0, *, numeric_only: bool = False
) -> bigframes.series.Series
Return the product of the values over the requested axis.
Parameters Name Descriptionaßxis
{index (0), columns (1)}
Axis for the function to be applied on. For Series this parameter is unused and defaults to 0.
numeric_only
bool. default False
Include only float, int, boolean columns.
raddradd(
other: float | int | bigframes.series.Series | DataFrame,
axis: str | int = "columns",
) -> DataFrame
Get addition of DataFrame and other, element-wise (binary operator +
).
Equivalent to dataframe + other
. With reverse version, radd
.
Among flexible wrappers (add
, sub
, mul
, div
, mod
, pow
) to arithmetic operators: +
, -
, *
, /
, //
, %
, **
.
>>> import bigframes.pandas as bpd
>>> bpd.options.display.progress_bar = None
>>> df = bpd.DataFrame({
... 'A': [1, 2, 3],
... 'B': [4, 5, 6],
... })
You can use method name:
>>> df['A'].add(df['B'])
0 5
1 7
2 9
dtype: Int64
You can also use arithmetic operator +
:
>>> df['A'] + (df['B'])
0 5
1 7
2 9
dtype: Int64
Parameters Name Description other
float, int, or Series
Any single or multiple element data structure, or list-like object.
axis
{0 or 'index', 1 or 'columns'}
Whether to compare by the index (0 or 'index') or columns. (1 or 'columns'). For Series input, axis to match Series index on.
Returns Type DescriptionDataFrame
DataFrame result of the arithmetic operation. rank
rank(
axis=0,
method: str = "average",
numeric_only=False,
na_option: str = "keep",
ascending=True,
) -> bigframes.dataframe.DataFrame
Compute numerical data ranks (1 through n) along axis.
By default, equal values are assigned a rank that is the average of the ranks of those values.
Parameters Name Descriptionmethod
{'average', 'min', 'max', 'first', 'dense'}, default 'average'
How to rank the group of records that have the same value (i.e. ties): average
: average rank of the group, min
: lowest rank in the group max: highest rank in the group,
first: ranks assigned in order they appear in the array,
dense`: like 'min', but rank always increases by 1 between groups.
numeric_only
bool, default False
For DataFrame objects, rank only numeric columns if set to True.
na_option
{'keep', 'top', 'bottom'}, default 'keep'
How to rank NaN values: keep
: assign NaN rank to NaN values, , top
: assign lowest rank to NaN values, bottom
: assign highest rank to NaN values.
ascending
bool, default True
Whether or not the elements should be ranked in ascending order.
Returns Type Descriptionsame type as caller
Return a Series or DataFrame with data ranks as values. rdiv
rdiv(
other: float | int | bigframes.series.Series | DataFrame,
axis: str | int = "columns",
) -> DataFrame
Get floating division of DataFrame and other, element-wise (binary operator /
).
Equivalent to other / dataframe
. With reverse version, truediv
.
Among flexible wrappers (add
, sub
, mul
, div
, mod
, pow
) to arithmetic operators: +
, -
, *
, /
, //
, %
, **
.
>>> import bigframes.pandas as bpd
>>> bpd.options.display.progress_bar = None
>>> df = bpd.DataFrame({
... 'A': [1, 2, 3],
... 'B': [4, 5, 6],
... })
>>> df['A'].rtruediv(df['B'])
0 4.0
1 2.5
2 2.0
dtype: Float64
It's equivalent to using arithmetic operator: /
:
>>> df['B'] / (df['A'])
0 4.0
1 2.5
2 2.0
dtype: Float64
Parameters Name Description other
float, int, or Series
Any single or multiple element data structure, or list-like object.
axis
{0 or 'index', 1 or 'columns'}
Whether to compare by the index (0 or 'index') or columns. (1 or 'columns'). For Series input, axis to match Series index on.
reindexreindex(
labels=None,
*,
index=None,
columns=None,
axis: typing.Optional[typing.Union[str, int]] = None,
validate: typing.Optional[bool] = None
)
Conform DataFrame to new index with optional filling logic.
Places NA in locations having no value in the previous index. A new object is produced.
Parameters Name Descriptionlabels
array-like, optional
New labels / index to conform the axis specified by 'axis' to.
index
array-like, optional
New labels for the index. Preferably an Index object to avoid duplicating data.
columns
array-like, optional
New labels for the columns. Preferably an Index object to avoid duplicating data.
axis
int or str, optional
Axis to target. Can be either the axis name ('index', 'columns') or number (0, 1).
Returns Type DescriptionDataFrame
DataFrame with changed index. reindex_like
reindex_like(
other: bigframes.dataframe.DataFrame, *, validate: typing.Optional[bool] = None
)
Return an object with matching indices as other object.
Conform the object to the same index on all axes. Optional filling logic, placing Null in locations having no value in the previous index.
Parameter Name Descriptionother
Object of the same data type
Its row and column indices are used to define the new indices of this object.
Returns Type DescriptionSeries or DataFrame
Same type as caller, but with changed indices on each axis. rename
rename(
*, columns: typing.Mapping[typing.Hashable, typing.Hashable]
) -> bigframes.dataframe.DataFrame
Rename columns.
Dict values must be unique (1-to-1). Labels not contained in a dict will be left as-is. Extra labels listed don't throw an error.
Parameter Name Descriptioncolumns
Mapping
Dict-like from old column labels to new column labels.
Exceptions Type DescriptionKeyError
If any of the labels is not found. Returns Type Description bigframes.dataframe.DataFrame
DataFrame with the renamed axis labels. rename_axis
rename_axis(
mapper: typing.Union[typing.Hashable, typing.Sequence[typing.Hashable]], **kwargs
) -> bigframes.dataframe.DataFrame
Set the name of the axis for the index.
Note: Currently only accepts a single string parameter (the new name of the index). Returns Type Descriptionbigframes.dataframe.DataFrame
DataFrame with the new index name reorder_levels
reorder_levels(order: LevelsType, axis: int | str = 0)
Rearrange index levels using input order. May not drop or duplicate levels.
Parameters Name Descriptionorder
list of int or list of str
List representing new level order. Reference level by number (position) or by key (label).
axis
{0 or 'index', 1 or 'columns'}, default 0
Where to reorder levels.
Returns Type DescriptionDataFrame
DataFrame of rearranged index. reset_index
reset_index(*, drop: bool = False) -> bigframes.dataframe.DataFrame
Reset the index.
Reset the index of the DataFrame, and use the default one instead.
Parameter Name Descriptiondrop
bool, default False
Do not try to insert index into dataframe columns. This resets the index to the default integer index.
Returns Type Descriptionbigframes.dataframe.DataFrame
DataFrame with the new index. rfloordiv
rfloordiv(
other: float | int | bigframes.series.Series | DataFrame,
axis: str | int = "columns",
) -> DataFrame
Get integer division of DataFrame and other, element-wise (binary operator //
).
Equivalent to other // dataframe
. With reverse version, rfloordiv
.
Among flexible wrappers (add
, sub
, mul
, div
, mod
, pow
) to arithmetic operators: +
, -
, *
, /
, //
, %
, **
.
>>> import bigframes.pandas as bpd
>>> bpd.options.display.progress_bar = None
>>> df = bpd.DataFrame({
... 'A': [1, 2, 3],
... 'B': [4, 5, 6],
... })
>>> df['A'].rfloordiv(df['B'])
0 4
1 2
2 2
dtype: Int64
It's equivalent to using arithmetic operator: //
:
>>> df['B'] // (df['A'])
0 4
1 2
2 2
dtype: Int64
Parameters Name Description other
float, int, or Series
Any single or multiple element data structure, or list-like object.
axis
{0 or 'index', 1 or 'columns'}
Whether to compare by the index (0 or 'index') or columns. (1 or 'columns'). For Series input, axis to match Series index on.
Returns Type DescriptionDataFrame
DataFrame result of the arithmetic operation. rmod
rmod(
other: int | bigframes.series.Series | DataFrame, axis: str | int = "columns"
) -> DataFrame
Get modulo of DataFrame and other, element-wise (binary operator %
).
Equivalent to other % dataframe
. With reverse version, mod
.
Among flexible wrappers (add
, sub
, mul
, div
, mod
, pow
) to arithmetic operators: +
, -
, *
, /
, //
, %
, **
.
>>> import bigframes.pandas as bpd
>>> bpd.options.display.progress_bar = None
>>> df = bpd.DataFrame({
... 'A': [1, 2, 3],
... 'B': [4, 5, 6],
... })
>>> df['A'].rmod(df['B'])
0 0
1 1
2 0
dtype: Int64
It's equivalent to using arithmetic operator: %
:
>>> df['B'] % (df['A'])
0 0
1 1
2 0
dtype: Int64
Parameters Name Description other
float, int, or Series
Any single or multiple element data structure, or list-like object.
axis
{0 or 'index', 1 or 'columns'}
Whether to compare by the index (0 or 'index') or columns. (1 or 'columns'). For Series input, axis to match Series index on.
Returns Type DescriptionDataFrame
DataFrame result of the arithmetic operation. rmul
rmul(
other: float | int | bigframes.series.Series | DataFrame,
axis: str | int = "columns",
) -> DataFrame
Get multiplication of DataFrame and other, element-wise (binary operator *
).
Equivalent to dataframe * other
. With reverse version, rmul
.
Among flexible wrappers (add
, sub
, mul
, div
, mod
, pow
) to arithmetic operators: +
, -
, *
, /
, //
, %
, **
.
>>> import bigframes.pandas as bpd
>>> bpd.options.display.progress_bar = None
>>> df = bpd.DataFrame({
... 'A': [1, 2, 3],
... 'B': [4, 5, 6],
... })
You can use method name:
>>> df['A'].mul(df['B'])
0 4
1 10
2 18
dtype: Int64
You can also use arithmetic operator *
:
>>> df['A'] * (df['B'])
0 4
1 10
2 18
dtype: Int64
Parameters Name Description other
float, int, or Series
Any single or multiple element data structure, or list-like object.
axis
{0 or 'index', 1 or 'columns'}
Whether to compare by the index (0 or 'index') or columns. (1 or 'columns'). For Series input, axis to match Series index on.
Returns Type DescriptionDataFrame
DataFrame result of the arithmetic operation. rolling
rolling(window: int, min_periods=None) -> bigframes.core.window.Window
Provide rolling window calculations.
Parameters Name Descriptionwindow
int, timedelta, str, offset, or BaseIndexer subclass
Size of the moving window. If an integer, the fixed number of observations used for each window. If a timedelta, str, or offset, the time period of each window. Each window will be a variable sized based on the observations included in the time-period. This is only valid for datetime-like indexes. To learn more about the offsets & frequency strings, please see this link https://pandas.pydata.org/pandas-docs/stable/user_guide/timeseries.html#offset-aliases
__. If a BaseIndexer subclass, the window boundaries based on the defined get_window_bounds
method. Additional rolling keyword arguments, namely min_periods
, center
, closed
and step
will be passed to get_window_bounds
.
min_periods
int, default None
Minimum number of observations in window required to have a value; otherwise, result is np.nan
. For a window that is specified by an offset, min_periods
will default to 1. For a window that is specified by an integer, min_periods
will default to the size of the window.
rpow(
other: int | bigframes.series.Series, axis: str | int = "columns"
) -> DataFrame
Get Exponential power of dataframe and other, element-wise (binary operator rpow
).
Equivalent to other ** dataframe
, but with support to substitute a fill_value for missing data in one of the inputs. With reverse version, pow
.
Among flexible wrappers (add
, sub
, mul
, div
, mod
, pow
) to arithmetic operators: +
, -
, *
, /
, //
, %
, **
.
>>> import bigframes.pandas as bpd
>>> bpd.options.display.progress_bar = None
>>> df = bpd.DataFrame({
... 'A': [1, 2, 3],
... 'B': [4, 5, 6],
... })
>>> df['A'].rpow(df['B'])
0 4
1 25
2 216
dtype: Int64
It's equivalent to using arithmetic operator: **
:
>>> df['B'] ** (df['A'])
0 4
1 25
2 216
dtype: Int64
Parameters Name Description other
float, int, or Series
Any single or multiple element data structure, or list-like object.
axis
{0 or 'index', 1 or 'columns'}
Whether to compare by the index (0 or 'index') or columns. (1 or 'columns'). For Series input, axis to match Series index on.
Returns Type DescriptionDataFrame
DataFrame result of the arithmetic operation. rsub
rsub(
other: float | int | bigframes.series.Series | DataFrame,
axis: str | int = "columns",
) -> DataFrame
Get subtraction of DataFrame and other, element-wise (binary operator -
).
Equivalent to other - dataframe
. With reverse version, sub
.
Among flexible wrappers (add
, sub
, mul
, div
, mod
, pow
) to arithmetic operators: +
, -
, *
, /
, //
, %
, **
.
>>> import bigframes.pandas as bpd
>>> bpd.options.display.progress_bar = None
>>> df = bpd.DataFrame({
... 'A': [1, 2, 3],
... 'B': [4, 5, 6],
... })
>>> df['A'].rsub(df['B'])
0 3
1 3
2 3
dtype: Int64
It's equivalent to using arithmetic operator: -
:
>>> df['B'] - (df['A'])
0 3
1 3
2 3
dtype: Int64
Parameters Name Description other
float, int, or Series
Any single or multiple element data structure, or list-like object.
axis
{0 or 'index', 1 or 'columns'}
Whether to compare by the index (0 or 'index') or columns. (1 or 'columns'). For Series input, axis to match Series index on.
Returns Type DescriptionDataFrame
DataFrame result of the arithmetic operation. rtruediv
rtruediv(
other: float | int | bigframes.series.Series | DataFrame,
axis: str | int = "columns",
) -> DataFrame
Get floating division of DataFrame and other, element-wise (binary operator /
).
Equivalent to other / dataframe
. With reverse version, truediv
.
Among flexible wrappers (add
, sub
, mul
, div
, mod
, pow
) to arithmetic operators: +
, -
, *
, /
, //
, %
, **
.
>>> import bigframes.pandas as bpd
>>> bpd.options.display.progress_bar = None
>>> df = bpd.DataFrame({
... 'A': [1, 2, 3],
... 'B': [4, 5, 6],
... })
>>> df['A'].rtruediv(df['B'])
0 4.0
1 2.5
2 2.0
dtype: Float64
It's equivalent to using arithmetic operator: /
:
>>> df['B'] / (df['A'])
0 4.0
1 2.5
2 2.0
dtype: Float64
Parameters Name Description other
float, int, or Series
Any single or multiple element data structure, or list-like object.
axis
{0 or 'index', 1 or 'columns'}
Whether to compare by the index (0 or 'index') or columns. (1 or 'columns'). For Series input, axis to match Series index on.
samplesample(
n: typing.Optional[int] = None,
frac: typing.Optional[float] = None,
*,
random_state: typing.Optional[int] = None
) -> bigframes.dataframe.DataFrame
Return a random sample of items from an axis of object.
You can use random_state
for reproducibility.
n
Optional[int], default None
Number of items from axis to return. Cannot be used with frac
. Default = 1 if frac
= None.
frac
Optional[float], default None
Fraction of axis items to return. Cannot be used with n
.
random_state
Optional[int], default None
Seed for random number generator.
set_indexset_index(
keys: typing.Union[typing.Hashable, typing.Sequence[typing.Hashable]],
append: bool = False,
drop: bool = True,
) -> bigframes.dataframe.DataFrame
Set the DataFrame index using existing columns.
Set the DataFrame index (row labels) using one existing column. The index can replace the existing index.
Returns Type DescriptionDataFrame
Changed row labels. shift
shift(periods: int = 1) -> bigframes.dataframe.DataFrame
Shift index by desired number of periods.
Shifts the index without realigning the data.
Returns Type DescriptionNDFrame
Copy of input object, shifted. skew
skew(*, numeric_only: bool = False)
Return unbiased skew over requested axis.
Normalized by N-1.
Parameter Name Descriptionnumeric_only
bool, default False
Include only float, int, boolean columns.
sort_indexsort_index(
ascending: bool = True, na_position: typing.Literal["first", "last"] = "last"
) -> bigframes.dataframe.DataFrame
Sort object by labels (along an axis).
sort_valuessort_values(
by: str | typing.Sequence[str],
*,
ascending: bool | typing.Sequence[bool] = True,
kind: str = "quicksort",
na_position: typing.Literal["first", "last"] = "last"
) -> DataFrame
Sort by the values along row axis.
Parameters Name Descriptionby
str or Sequence[str]
Name or list of names to sort by.
ascending
bool or Sequence[bool], default True
Sort ascending vs. descending. Specify list for multiple sort orders. If this is a list of bools, must match the length of the by.
kind
str, default 'quicksort'
Choice of sorting algorithm. Accepts 'quicksort', 'mergesort', 'heapsort', 'stable'. Ignored except when determining whether to sort stably. 'mergesort' or 'stable' will result in stable reorder.
na_position
{'first', 'last'}, default last
{'first', 'last'}
, default 'last' Puts NaNs at the beginning if first
; last
puts NaNs at the end.
stack(level: typing.Union[str, int, typing.Sequence[typing.Union[str, int]]] = -1)
Stack the prescribed level(s) from columns to index.
Return a reshaped DataFrame or Series having a multi-level index with one or more new inner-most levels compared to the current DataFrame. The new inner-most levels are created by pivoting the columns of the current dataframe:
DataFrame or Series
Stacked dataframe or series. std
std(
axis: typing.Union[str, int] = 0, *, numeric_only: bool = False
) -> bigframes.series.Series
Return sample standard deviation over requested axis.
Normalized by N-1 by default.
Parameter Name Descriptionnumeric_only
bool. default False
Default False. Include only float, int, boolean columns.
subsub(
other: float | int | bigframes.series.Series | DataFrame,
axis: str | int = "columns",
) -> DataFrame
Get subtraction of DataFrame and other, element-wise (binary operator -
).
Equivalent to dataframe - other
. With reverse version, rsub
.
Among flexible wrappers (add
, sub
, mul
, div
, mod
, pow
) to arithmetic operators: +
, -
, *
, /
, //
, %
, **
.
>>> import bigframes.pandas as bpd
>>> bpd.options.display.progress_bar = None
>>> df = bpd.DataFrame({
... 'A': [1, 2, 3],
... 'B': [4, 5, 6],
... })
You can use method name:
>>> df['A'].sub(df['B'])
0 -3
1 -3
2 -3
dtype: Int64
You can also use arithmetic operator -
:
>>> df['A'] - (df['B'])
0 -3
1 -3
2 -3
dtype: Int64
Parameters Name Description other
float, int, or Series
Any single or multiple element data structure, or list-like object.
axis
{0 or 'index', 1 or 'columns'}
Whether to compare by the index (0 or 'index') or columns. (1 or 'columns'). For Series input, axis to match Series index on.
Returns Type DescriptionDataFrame
DataFrame result of the arithmetic operation. subtract
subtract(
other: float | int | bigframes.series.Series | DataFrame,
axis: str | int = "columns",
) -> DataFrame
Get subtraction of DataFrame and other, element-wise (binary operator -
).
Equivalent to dataframe - other
. With reverse version, rsub
.
Among flexible wrappers (add
, sub
, mul
, div
, mod
, pow
) to arithmetic operators: +
, -
, *
, /
, //
, %
, **
.
>>> import bigframes.pandas as bpd
>>> bpd.options.display.progress_bar = None
>>> df = bpd.DataFrame({
... 'A': [1, 2, 3],
... 'B': [4, 5, 6],
... })
You can use method name:
>>> df['A'].sub(df['B'])
0 -3
1 -3
2 -3
dtype: Int64
You can also use arithmetic operator -
:
>>> df['A'] - (df['B'])
0 -3
1 -3
2 -3
dtype: Int64
Parameters Name Description other
float, int, or Series
Any single or multiple element data structure, or list-like object.
axis
{0 or 'index', 1 or 'columns'}
Whether to compare by the index (0 or 'index') or columns. (1 or 'columns'). For Series input, axis to match Series index on.
Returns Type DescriptionDataFrame
DataFrame result of the arithmetic operation. sum
sum(
axis: typing.Union[str, int] = 0, *, numeric_only: bool = False
) -> bigframes.series.Series
Return the sum of the values over the requested axis.
This is equivalent to the method numpy.sum
.
axis
{index (0), columns (1)}
Axis for the function to be applied on. For Series this parameter is unused and defaults to 0.
numeric_only
bool. default False
Default False. Include only float, int, boolean columns.
swaplevelswaplevel(i: int = -2, j: int = -1, axis: int | str = 0)
Swap levels i and j in a MultiIndex
.
Default is to swap the two innermost levels of the index.
Parameters Name Descriptioni
j: int or str
j: Levels of the indices to be swapped. Can pass level name as string.
axis
{0 or 'index', 1 or 'columns'}, default 0
The axis to swap levels on. 0 or 'index' for row-wise, 1 or 'columns' for column-wise.
Returns Type DescriptionDataFrame
DataFrame with levels swapped in MultiIndex. tail
tail(n: int = 5) -> bigframes.dataframe.DataFrame
Return the last n
rows.
This function returns last n
rows from the object based on position. It is useful for quickly verifying data, for example, after sorting or appending rows.
For negative values of n
, this function returns all rows except the first |n|
rows, equivalent to df[|n|:]
.
If n is larger than the number of rows, this function returns all rows.
Parameter Name Descriptionn
int, default 5
Number of rows to select.
to_csvto_csv(
path_or_buf: str, sep=",", *, header: bool = True, index: bool = True
) -> None
Write object to a comma-separated values (csv) file on Cloud Storage.
Parameters Name Descriptionpath_or_buf
str
A destination URI of Cloud Storage files(s) to store the extracted dataframe in format of gs://<bucket_name>/<object_name_or_glob>
. If the data size is more than 1GB, you must use a wildcard to export the data into multiple files and the size of the files varies. None, file-like objects or local file paths not yet supported.
index
bool, default True
If True, write row names (index).
Returns Type DescriptionNone
String output not yet supported. to_dict
to_dict(orient: Literal['dict', 'list', 'series', 'split', 'tight', 'records', 'index'] = 'dict', into: type[dict] = <class 'dict'>, **kwargs) -> dict | list[dict]
Convert the DataFrame to a dictionary.
The type of the key-value pairs can be customized with the parameters (see below).
Examples:
>>> import bigframes.pandas as bpd
>>> bpd.options.display.progress_bar = None
>>> df = bpd.DataFrame({'col1': [1, 2], 'col2': [3, 4]})
>>> df.to_dict()
{'col1': {0: 1, 1: 2}, 'col2': {0: 3, 1: 4}}
You can specify the return orientation.
>>> df.to_dict('series')
{'col1': 0 1
1 2
Name: col1, dtype: Int64,
'col2': 0 3
1 4
Name: col2, dtype: Int64}
>>> df.to_dict('split')
{'index': [0, 1], 'columns': ['col1', 'col2'], 'data': [[1, 3], [2, 4]]}
>>> df.to_dict("tight")
{'index': [0, 1],
'columns': ['col1', 'col2'],
'data': [[1, 3], [2, 4]],
'index_names': [None],
'column_names': [None]}
Parameters Name Description orient
str {'dict', 'list', 'series', 'split', 'tight', 'records', 'index'}
Determines the type of the values of the dictionary. 'dict' (default) : dict like {column -> {index -> value}}. 'list' : dict like {column -> [values]}. 'series' : dict like {column -> Series(values)}. split' : dict like {'index' -> [index], 'columns' -> [columns], 'data' -> [values]}. 'tight' : dict like {'index' -> [index], 'columns' -> [columns], 'data' -> [values], 'index_names' -> [index.names], 'column_names' -> [column.names]}. 'records' : list like [{column -> value}, ... , {column -> value}]. 'index' : dict like {index -> {column -> value}}.
into
class, default dict
The collections.abc.Mapping subclass used for all Mappings in the return value. Can be the actual class or an empty instance of the mapping type you want. If you want a collections.defaultdict, you must pass it initialized.
index
bool, default True
Whether to include the index item (and index_names item if orient
is 'tight') in the returned dictionary. Can only be False
when orient
is 'split' or 'tight'.
dict or list of dict
Return a collections.abc.Mapping object representing the DataFrame. The resulting transformation depends on the orient
parameter. to_excel
to_excel(excel_writer, sheet_name: str = "Sheet1", **kwargs) -> None
Write DataFrame to an Excel sheet.
To write a single DataFrame to an Excel .xlsx file it is only necessary to specify a target file name. To write to multiple sheets it is necessary to create an ExcelWriter
object with a target file name, and specify a sheet in the file to write to.
Multiple sheets may be written to by specifying unique sheet_name
. With all data written to the file it is necessary to save the changes. Note that creating an ExcelWriter
object with a file name that already exists will result in the contents of the existing file being erased.
Examples:
>>> import bigframes.pandas as bpd
>>> import tempfile
>>> bpd.options.display.progress_bar = None
>>> df = bpd.DataFrame({'col1': [1, 2], 'col2': [3, 4]})
>>> df.to_excel(tempfile.TemporaryFile())
Parameters Name Description excel_writer
path-like, file-like, or ExcelWriter object
File path or existing ExcelWriter.
sheet_name
str, default 'Sheet1'
Name of sheet which will contain DataFrame.
to_gbqto_gbq(
destination_table: str,
*,
if_exists: typing.Optional[typing.Literal["fail", "replace", "append"]] = "fail",
index: bool = True,
ordering_id: typing.Optional[str] = None
) -> None
Write a DataFrame to a BigQuery table.
Examples:
>>> import bigframes.pandas as bpd
>>> bpd.options.display.progress_bar = None
>>> df = bpd.DataFrame({'col1': [1, 2], 'col2': [3, 4]})
>>> # destination_table = PROJECT_ID + "." + DATASET_ID + "." + TABLE_NAME
>>> df.to_gbq("bigframes-dev.birds.test-numbers", if_exists="replace")
Parameters Name Description destination_table
str
Name of table to be written, in the form dataset.tablename
or project.dataset.tablename
.
if_exists
str, default 'fail'
Behavior when the destination table exists. Value can be one of: 'fail'
If table exists raise pandas_gbq.gbq.TableCreationError. 'replace'
If table exists, drop it, recreate it, and insert data. 'append'
If table exists, insert data. Create if does not exist.
index
bool. default True
whether write row names (index) or not.
ordering_id
Optional[str], default None
If set, write the ordering of the DataFrame as a column in the result table with this name.
to_jsonto_json(
path_or_buf: str,
orient: typing.Literal[
"split", "records", "index", "columns", "values", "table"
] = "columns",
*,
lines: bool = False,
index: bool = True
) -> None
Convert the object to a JSON string, written to Cloud Storage.
Note NaN's and None will be converted to null and datetime objects will be converted to UNIX timestamps.
Note: Onlyorient='records'
and lines=True
is supported so far. Parameters Name Description path_or_buf
str
A destination URI of Cloud Storage files(s) to store the extracted dataframe in format of gs://<bucket_name>/<object_name_or_glob>
. Must contain a wildcard *
character. If the data size is more than 1GB, you must use a wildcard to export the data into multiple files and the size of the files varies. None, file-like objects or local file paths not yet supported.
orient
{split
, records
, index
, columns
, values
, table
}, default 'columns
Indication of expected JSON string format. * Series: - default is 'index' - allowed values are: {{'split', 'records', 'index', 'table'}}. * DataFrame: - default is 'columns' - allowed values are: {{'split', 'records', 'index', 'columns', 'values', 'table'}}. * The format of the JSON string: - 'split' : dict like {{'index' -> [index], 'columns' -> [columns], 'data' -> [values]}} - 'records' : list like [{{column -> value}}, ... , {{column -> value}}] - 'index' : dict like {{index -> {{column -> value}}}} - 'columns' : dict like {{column -> {{index -> value}}}} - 'values' : just the values array - 'table' : dict like {{'schema': {{schema}}, 'data': {{data}}}} Describing the data, where data component is like orient='records'
.
index
bool, default True
If True, write row names (index).
lines
bool, default False
If 'orient' is 'records' write out line-delimited json format. Will throw ValueError if incorrect 'orient' since others are not list-like.
Returns Type DescriptionNone
String output not yet supported. to_latex
to_latex(
buf=None,
columns: Sequence | None = None,
header: bool | Sequence[str] = True,
index: bool = True,
**kwargs
) -> str | None
Render object to a LaTeX tabular, longtable, or nested table.
Requires \usepackage{{booktabs}}
. The output can be copy/pasted into a main LaTeX document or read from an external file with \input{{table.tex}}
.
Examples:
>>> import bigframes.pandas as bpd
>>> bpd.options.display.progress_bar = None
>>> df = bpd.DataFrame({'col1': [1, 2], 'col2': [3, 4]})
>>> print(df.to_latex())
\begin{tabular}{lrr}
\toprule
& col1 & col2 \\
\midrule
0 & 1 & 3 \\
1 & 2 & 4 \\
\bottomrule
\end{tabular}
<BLANKLINE>
Parameters Name Description buf
str, Path or StringIO-like, optional, default None
Buffer to write to. If None, the output is returned as a string.
columns
list of label, optional
The subset of columns to write. Writes all columns by default.
header
bool or list of str, default True
Write out the column names. If a list of strings is given, it is assumed to be aliases for the column names.
index
bool, default True
Write row names (index).
to_markdownto_markdown(buf=None, mode: str = "wt", index: bool = True, **kwargs) -> str | None
Print DataFrame in Markdown-friendly format.
Examples:
>>> import bigframes.pandas as bpd
>>> bpd.options.display.progress_bar = None
>>> df = bpd.DataFrame({'col1': [1, 2], 'col2': [3, 4]})
>>> print(df.to_markdown())
| | col1 | col2 |
|---:|-------:|-------:|
| 0 | 1 | 3 |
| 1 | 2 | 4 |
Parameters Name Description buf
str, Path or StringIO-like, optional, default None
Buffer to write to. If None, the output is returned as a string.
mode
str, optional
Mode in which file is opened.
index
bool, optional, default True
Add index (row) labels.
to_numpyto_numpy(dtype=None, copy=False, na_value=None, **kwargs) -> numpy.ndarray
Convert the DataFrame to a NumPy array.
Examples:
>>> import bigframes.pandas as bpd
>>> bpd.options.display.progress_bar = None
>>> df = bpd.DataFrame({'col1': [1, 2], 'col2': [3, 4]})
>>> df.to_numpy()
array([[1, 3],
[2, 4]], dtype=object)
Parameters Name Description dtype
None
The dtype to pass to numpy.asarray()
.
copy
bool, default None
Whether to ensure that the returned value is not a view on another array.
na_value
Any, default None
The value to use for missing values. The default value depends on dtype and the dtypes of the DataFrame columns.
Returns Type Descriptionnumpy.ndarray
The converted NumPy array. to_orc
to_orc(path=None, **kwargs) -> bytes | None
Write a DataFrame to the ORC format.
Examples:
>>> import bigframes.pandas as bpd
>>> bpd.options.display.progress_bar = None
>>> df = bpd.DataFrame({'col1': [1, 2], 'col2': [3, 4]})
>>> import tempfile
>>> df.to_orc(tempfile.TemporaryFile())
Parameter Name Description path
str, file-like object or None, default None
If a string, it will be used as Root Directory path when writing a partitioned dataset. By file-like object, we refer to objects with a write() method, such as a file handle (e.g. via builtin open function). If path is None, a bytes object is returned.
to_pandasto_pandas(
max_download_size: typing.Optional[int] = None,
sampling_method: typing.Optional[str] = None,
random_state: typing.Optional[int] = None,
) -> pandas.core.frame.DataFrame
Write DataFrame to pandas DataFrame.
Parameters Name Descriptionmax_download_size
int, default None
Download size threshold in MB. If max_download_size is exceeded when downloading data (e.g., to_pandas()), the data will be downsampled if bigframes.options.sampling.enable_downsampling is True, otherwise, an error will be raised. If set to a value other than None, this will supersede the global config.
sampling_method
str, default None
Downsampling algorithms to be chosen from, the choices are: "head": This algorithm returns a portion of the data from the beginning. It is fast and requires minimal computations to perform the downsampling; "uniform": This algorithm returns uniform random samples of the data. If set to a value other than None, this will supersede the global config.
random_state
int, default None
The seed for the uniform downsampling algorithm. If provided, the uniform method may take longer to execute and require more computation. If set to a value other than None, this will supersede the global config.
Returns Type Descriptionpandas.DataFrame
A pandas DataFrame with all rows and columns of this DataFrame if the data_sampling_threshold_mb is not exceeded; otherwise, a pandas DataFrame with downsampled rows and all columns of this DataFrame. to_pandas_batches
to_pandas_batches() -> typing.Iterable[pandas.core.frame.DataFrame]
Stream DataFrame results to an iterable of pandas DataFrame
to_parquetto_parquet(
path: str,
*,
compression: typing.Optional[typing.Literal["snappy", "gzip"]] = "snappy",
index: bool = True
) -> None
Write a DataFrame to the binary Parquet format.
This function writes the dataframe as a parquet file <https://parquet.apache.org/>
_ to Cloud Storage.
Examples:
>>> import bigframes.pandas as bpd
>>> bpd.options.display.progress_bar = None
>>> df = bpd.DataFrame({'col1': [1, 2], 'col2': [3, 4]})
>>> gcs_bucket = "gs://bigframes-dev-testing/sample_parquet*.parquet"
>>> df.to_parquet(path=gcs_bucket)
Parameters Name Description path
str
Destination URI(s) of Cloud Storage files(s) to store the extracted dataframe in format of gs://<bucket_name>/<object_name_or_glob>
. If the data size is more than 1GB, you must use a wildcard to export the data into multiple files and the size of the files varies.
compression
str, default 'snappy'
Name of the compression to use. Use None
for no compression. Supported options: 'gzip'
, 'snappy'
.
index
bool, default True
If True
, include the dataframe's index(es) in the file output. If False
, they will not be written to the file.
to_pickle(path, **kwargs) -> None
Pickle (serialize) object to file.
Examples:
>>> import bigframes.pandas as bpd
>>> bpd.options.display.progress_bar = None
>>> df = bpd.DataFrame({'col1': [1, 2], 'col2': [3, 4]})
>>> gcs_bucket = "gs://bigframes-dev-testing/sample_pickle_gcs.pkl"
>>> df.to_pickle(path=gcs_bucket)
Parameter Name Description path
str
File path where the pickled object will be stored.
to_recordsto_records(
index: bool = True, column_dtypes=None, index_dtypes=None
) -> numpy.recarray
Convert DataFrame to a NumPy record array.
Index will be included as the first field of the record array if requested.
Examples:
>>> import bigframes.pandas as bpd
>>> bpd.options.display.progress_bar = None
>>> df = bpd.DataFrame({'col1': [1, 2], 'col2': [3, 4]})
>>> df.to_records()
rec.array([(0, 1, 3), (1, 2, 4)],
dtype=[('index', 'O'), ('col1', 'O'), ('col2', 'O')])
Parameters Name Description index
bool, default True
Include index in resulting record array, stored in 'index' field or using the index label, if set.
column_dtypes
str, type, dict, default None
If a string or type, the data type to store all columns. If a dictionary, a mapping of column names and indices (zero-indexed) to specific data types.
index_dtypes
str, type, dict, default None
If a string or type, the data type to store all index levels. If a dictionary, a mapping of index level names and indices (zero-indexed) to specific data types. This mapping is applied only if index=True
.
np.recarray
NumPy ndarray with the DataFrame labels as fields and each row of the DataFrame as entries. to_string
to_string(
buf=None,
columns: Sequence[str] | None = None,
col_space=None,
header: bool | Sequence[str] = True,
index: bool = True,
na_rep: str = "NaN",
formatters=None,
float_format=None,
sparsify: bool | None = None,
index_names: bool = True,
justify: str | None = None,
max_rows: int | None = None,
max_cols: int | None = None,
show_dimensions: bool = False,
decimal: str = ".",
line_width: int | None = None,
min_rows: int | None = None,
max_colwidth: int | None = None,
encoding: str | None = None,
) -> str | None
Render a DataFrame to a console-friendly tabular output.
Examples:
>>> import bigframes.pandas as bpd
>>> bpd.options.display.progress_bar = None
>>> df = bpd.DataFrame({'col1': [1, 2], 'col2': [3, 4]})
>>> print(df.to_string())
col1 col2
0 1 3
1 2 4
Parameters Name Description buf
str, Path or StringIO-like, optional, default None
Buffer to write to. If None, the output is returned as a string.
columns
sequence, optional, default None
The subset of columns to write. Writes all columns by default.
col_space
int, list or dict of int, optional
The minimum width of each column.
header
bool or sequence, optional
Write out the column names. If a list of strings is given, it is assumed to be aliases for the column names.
index
bool, optional, default True
Whether to print index (row) labels.
na_rep
str, optional, default 'NaN'
String representation of NAN to use.
formatters
list, tuple or dict of one-param. functions, optional
Formatter functions to apply to columns' elements by position or name. The result of each function must be a unicode string. List/tuple must be of length equal to the number of columns.
float_format
one-parameter function, optional, default None
Formatter function to apply to columns' elements if they are floats. The result of this function must be a unicode string.
sparsify
bool, optional, default True
Set to False for a DataFrame with a hierarchical index to print every multiindex key at each row.
index_names
bool, optional, default True
Prints the names of the indexes.
justify
str, default None
How to justify the column labels. If None uses the option from the print configuration (controlled by set_option), 'right' out of the box. Valid values are, 'left', 'right', 'center', 'justify', 'justify-all', 'start', 'end', 'inherit', 'match-parent', 'initial', 'unset'.
max_rows
int, optional
Maximum number of rows to display in the console.
min_rows
int, optional
The number of rows to display in the console in a truncated repr (when number of rows is above max_rows
).
max_cols
int, optional
Maximum number of columns to display in the console.
show_dimensions
bool, default False
Display DataFrame dimensions (number of rows by number of columns).
decimal
str, default '.'
Character recognized as decimal separator, e.g. ',' in Europe.
line_width
int, optional
Width to wrap a line in characters.
max_colwidth
int, optional
Max width to truncate each column in characters. By default, no limit.
encoding
str, default "utf-8"
Set character encoding.
Returns Type Descriptionstr or None
If buf is None, returns the result as a string. Otherwise returns None. truediv
truediv(
other: float | int | bigframes.series.Series | DataFrame,
axis: str | int = "columns",
) -> DataFrame
Get floating division of DataFrame and other, element-wise (binary operator /
).
Equivalent to dataframe / other
. With reverse version, rtruediv
.
Among flexible wrappers (add
, sub
, mul
, div
, mod
, pow
) to arithmetic operators: +
, -
, *
, /
, //
, %
, **
.
>>> import bigframes.pandas as bpd
>>> bpd.options.display.progress_bar = None
>>> df = bpd.DataFrame({
... 'A': [1, 2, 3],
... 'B': [4, 5, 6],
... })
You can use method name:
>>> df['A'].truediv(df['B'])
0 0.25
1 0.4
2 0.5
dtype: Float64
You can also use arithmetic operator /
:
>>> df['A'] / (df['B'])
0 0.25
1 0.4
2 0.5
dtype: Float64
Parameters Name Description other
float, int, or Series
Any single or multiple element data structure, or list-like object.
axis
{0 or 'index', 1 or 'columns'}
Whether to compare by the index (0 or 'index') or columns. (1 or 'columns'). For Series input, axis to match Series index on.
Returns Type DescriptionDataFrame
DataFrame result of the arithmetic operation. unstack
unstack(
level: typing.Union[str, int, typing.Sequence[typing.Union[str, int]]] = -1
)
Pivot a level of the (necessarily hierarchical) index labels.
Returns a DataFrame having a new level of column labels whose inner-most level consists of the pivoted index labels.
If the index is not a MultiIndex, the output will be a Series (the analogue of stack when the columns are not a MultiIndex).
updateupdate(other, join: str = "left", overwrite=True, filter_func=None)
Modify in place using non-NA values from another DataFrame.
Aligns on indices. There is no return value.
Examples:
>>> import bigframes.pandas as bpd
>>> bpd.options.display.progress_bar = None
>>> df = bpd.DataFrame({'A': [1, 2, 3],
... 'B': [400, 500, 600]})
>>> new_df = bpd.DataFrame({'B': [4, 5, 6],
... 'C': [7, 8, 9]})
>>> df.update(new_df)
>>> df
A B
0 1 4
1 2 5
2 3 6
<BLANKLINE>
[3 rows x 2 columns]
Parameters Name Description other
DataFrame, or object coercible into a DataFrame
Should have at least one matching index/column label with the original DataFrame. If a Series is passed, its name attribute must be set, and that will be used as the column name to align with the original DataFrame.
join
{'left'}, default 'left'
Only left join is implemented, keeping the index and columns of the original object.
overwrite
bool, default True
How to handle non-NA values for overlapping keys: True: overwrite original DataFrame's values with values from other
. False: only update values that are NA in the original DataFrame.
filter_func
callable(1d-array) -> bool 1d-array, optional
Can choose to replace values other than NA. Return True for values that should be updated.
Returns Type DescriptionNone
This method directly changes calling object. value_counts
value_counts(
subset: typing.Optional[
typing.Union[typing.Hashable, typing.Sequence[typing.Hashable]]
] = None,
normalize: bool = False,
sort: bool = True,
ascending: bool = False,
dropna: bool = True,
)
Return a Series containing counts of unique rows in the DataFrame.
Parameters Name Descriptionsubset
label or list of labels, optional
Columns to use when counting unique combinations.
normalize
bool, default False
Return proportions rather than frequencies.
sort
bool, default True
Sort by frequencies.
ascending
bool, default False
Sort in ascending order.
dropna
bool, default True
Don’t include counts of rows that contain NA values.
Returns Type DescriptionSeries
Series containing counts of unique rows in the DataFrame var
var(
axis: typing.Union[str, int] = 0, *, numeric_only: bool = False
) -> bigframes.series.Series
Return unbiased variance over requested axis.
Normalized by N-1 by default.
Parameters Name Descriptionaxis
{index (0), columns (1)}
Axis for the function to be applied on. For Series this parameter is unused and defaults to 0.
numeric_only
bool. default False
Default False. Include only float, int, boolean columns.
Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see the Google Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.
Last updated 2025-08-12 UTC.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2025-08-12 UTC."],[],[]]
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4