RetroSearch Browse

Home - News ( United States | United Kingdom | Italy | Germany ) - Football scores

Showing content from https://docs.rs/datafusion/latest/datafusion/logical_expr/enum.LogicalPlan.html below:

LogicalPlan in datafusion::logical_expr - Rust

pub enum LogicalPlan {
Show 25 variants    Projection(Projection),
    Filter(Filter),
    Window(Window),
    Aggregate(Aggregate),
    Sort(Sort),
    Join(Join),
    Repartition(Repartition),
    Union(Union),
    TableScan(TableScan),
    EmptyRelation(EmptyRelation),
    Subquery(Subquery),
    SubqueryAlias(SubqueryAlias),
    Limit(Limit),
    Statement(Statement),
    Values(Values),
    Explain(Explain),
    Analyze(Analyze),
    Extension(Extension),
    Distinct(Distinct),
    Dml(DmlStatement),
    Ddl(DdlStatement),
    Copy(CopyTo),
    DescribeTable(DescribeTable),
    Unnest(Unnest),
    RecursiveQuery(RecursiveQuery),
}

Expand description

A LogicalPlan is a node in a tree of relational operators (such as Projection or Filter).

Represents transforming an input relation (table) to an output relation (table) with a potentially different schema. Plans form a dataflow tree where data flows from leaves up to the root to produce the query result.

LogicalPlans can be created by the SQL query planner, the DataFrame API, or programmatically (for example custom query languages).

§See also:

Expr: For the expressions that are evaluated by the plan
LogicalPlanBuilder: For building LogicalPlans
tree_node: To inspect and rewrite LogicalPlans

§Examples §Creating a LogicalPlan from SQL:

See SessionContext::sql

§Creating a LogicalPlan from the DataFrame API:

See DataFrame::logical_plan

§Creating a LogicalPlan programmatically:

See LogicalPlanBuilder

§Visiting and Rewriting LogicalPlans

Using the tree_node API, you can recursively walk all nodes in a LogicalPlan. For example, to find all column references in a plan:

let plan = table_scan(Some("employee"), &employee_schema(), None)?
 .filter(col("salary").gt(lit(1000)))?
 .project(vec![col("name")])?
 .build()?;

let mut expressions = HashSet::new();
plan.apply(|node| {
  node.apply_expressions(|expr| {
   expressions.insert(expr.clone());
   Ok(TreeNodeRecursion::Continue) })?;
  Ok(TreeNodeRecursion::Continue) }).unwrap();

assert_eq!(expressions.len(), 2);
println!("Found expressions: {:?}", expressions);
let salary = Expr::Column(Column::new(Some("employee"), "salary"));
assert!(expressions.contains(&salary.gt(lit(1000))));
let name = Expr::Column(Column::new(Some("employee"), "name"));
assert!(expressions.contains(&name));

You can also rewrite plans using the tree_node API. For example, to replace the filter predicate in a plan:

use datafusion_common::tree_node::Transformed;
let plan = table_scan(Some("employee"), &employee_schema(), None)?
 .filter(col("salary").gt(lit(1000)))?
 .project(vec![col("name")])?
 .build()?;

let transformed_result = plan.transform(|node| {
  if let LogicalPlan::Filter(mut filter) = node {
    filter.predicate = Expr::Column(Column::new(Some("employee"), "salary")).lt(lit(2000));
    let new_plan = LogicalPlan::Filter(filter);
    return Ok(Transformed::yes(new_plan)); }
  Ok(Transformed::no(node))
}).unwrap();

assert!(transformed_result.transformed);
let rewritten_plan = transformed_result.data;

assert_eq!(rewritten_plan.display_indent().to_string(),
"Projection: employee.name\
\n  Filter: employee.salary < Int32(2000)\
\n    TableScan: employee");

Evaluates an arbitrary list of expressions (essentially a SELECT with an expression list) on its input.

Filters rows from its input that do not match an expression (essentially a WHERE clause with a predicate expression).

Semantically, <predicate> is evaluated for each row of the input; If the value of <predicate> is true, the input row is passed to the output. If the value of <predicate> is false (or null), the row is discarded.

Windows input based on a set of window spec and window function (e.g. SUM or RANK). This is used to implement SQL window functions, and the OVER clause.

See Window for more details

Aggregates its input based on a set of grouping and aggregate expressions (e.g. SUM). This is used to implement SQL aggregates and GROUP BY.

See Aggregate for more details

Sorts its input according to a list of sort expressions. This is used to implement SQL ORDER BY

Join two logical plans on one or more join columns. This is used to implement SQL JOIN

Repartitions the input based on a partitioning scheme. This is used to add parallelism and is sometimes referred to as an “exchange” operator in other systems

Union multiple inputs with the same schema into a single output stream. This is used to implement SQL UNION [ALL] and INTERSECT [ALL].

Produces rows from a TableSource, used to implement SQL FROM tables or views.

Produces no rows: An empty relation with an empty schema that produces 0 or 1 row. This is used to implement SQL SELECT that has no values in the FROM clause.

Produces the output of running another query. This is used to implement SQL subqueries

Aliased relation provides, or changes, the name of a relation.

Skip some number of rows, and then fetch some number of rows.

A DataFusion Statement such as SET VARIABLE or START TRANSACTION

Values expression. See Postgres VALUES documentation for more details. This is used to implement SQL such as VALUES (1, 2), (3, 4)

Produces a relation with string representations of various parts of the plan. This is used to implement SQL EXPLAIN.

Runs the input, and prints annotated physical plan as a string with execution metric. This is used to implement SQL EXPLAIN ANALYZE.

Extension operator defined outside of DataFusion. This is used to extend DataFusion with custom relational operations that

Remove duplicate rows from the input. This is used to implement SQL SELECT DISTINCT ....

Data Manipulation Language (DML): Insert / Update / Delete

Data Definition Language (DDL): CREATE / DROP TABLES / VIEWS / SCHEMAS

COPY TO for writing plan results to files

Describe the schema of the table. This is used to implement the SQL DESCRIBE command from MySQL.

Unnest a column that contains a nested list type such as an ARRAY. This is used to implement SQL UNNEST

A variadic query (e.g. “Recursive CTEs”)

Source § Source

Get a reference to the logical plan’s schema