Applies to: Databricks Runtime
Invalidates and refreshes all the cached data (and the associated metadata) in Apache Spark cache for all Datasets that contains the given data source path. Path matching is by prefix, that is, /
would invalidate everything that is cached.
See REFRESH (MATERIALIZED VIEW or STREAMING TABLE) for refreshing the data in streaming tables and materialized views.
SyntaxâSee Disk cache vs. Spark cache for the differences between disk caching and the Apache Spark cache.
Parametersâresource_path
The path of the resource that is to be refreshed.
SQL
-- The Path is resolved using the datasource's File Index.
> CREATE TABLE test(ID INT) using parquet;
> INSERT INTO test SELECT 1000;
> CACHE TABLE test;
> INSERT INTO test SELECT 100;
> REFRESH "hdfs://path/to/table";
Related statementsâ
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4