Home - News ( United States | United Kingdom | Italy | Germany ) - Football scores

Showing content from https://spark.apache.org/docs/latest/api/R/reference/repartition.html below:

Repartition — repartition • SparkR

The following options for repartition are possible:

• 1. Return a new SparkDataFrame that has exactly numPartitions.

• 2. Return a new SparkDataFrame hash partitioned by the given columns into numPartitions.

• 3. Return a new SparkDataFrame hash partitioned by the given column(s), using spark.sql.shuffle.partitions as number of partitions.

repartition(x, ...)

# S4 method for class 'SparkDataFrame'
repartition(x, numPartitions = NULL, col = NULL, ...)

x

a SparkDataFrame.

...

additional column(s) to be used in the partitioning.

numPartitions

the number of partitions to use.

col

the column by which the partitioning will be performed.

repartition since 1.4.0

coalesce, repartitionByRange

Other SparkDataFrame functions: SparkDataFrame-class, agg(), alias(), arrange(), as.data.frame(), attach,SparkDataFrame-method, broadcast(), cache(), checkpoint(), coalesce(), collect(), colnames(), coltypes(), createOrReplaceTempView(), crossJoin(), cube(), dapplyCollect(), dapply(), describe(), dim(), distinct(), dropDuplicates(), dropna(), drop(), dtypes(), exceptAll(), except(), explain(), filter(), first(), gapplyCollect(), gapply(), getNumPartitions(), group_by(), head(), hint(), histogram(), insertInto(), intersectAll(), intersect(), isLocal(), isStreaming(), join(), limit(), localCheckpoint(), merge(), mutate(), ncol(), nrow(), persist(), printSchema(), randomSplit(), rbind(), rename(), repartitionByRange(), rollup(), sample(), saveAsTable(), schema(), selectExpr(), select(), showDF(), show(), storageLevel(), str(), subset(), summary(), take(), toJSON(), unionAll(), unionByName(), union(), unpersist(), unpivot(), withColumn(), withWatermark(), with(), write.df(), write.jdbc(), write.json(), write.orc(), write.parquet(), write.stream(), write.text()

if (FALSE) { # \dontrun{
sparkR.session()
path <- "path/to/file.json"
df <- read.json(path)
newDF <- repartition(df, 2L)
newDF <- repartition(df, numPartitions = 2L)
newDF <- repartition(df, col = df$"col1", df$"col2")
newDF <- repartition(df, 3L, col = df$"col1", df$"col2")
} # }

RetroSearch is an open source project built by @garambo | Open a GitHub Issue

Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo

HTML: 3.2 | Encoding: UTF-8 | Version: 0.7.4