note
This article covers Databricks Connect for Databricks Runtime 13.3 LTS and above.
This article describes how to install Databricks Connect for Scala. See What is Databricks Connect?. For the Python version of this article, see Install Databricks Connect for Python.
RequirementsâYour target Databricks workspace and cluster must meet the requirements for Compute configuration for Databricks Connect.
You have the Java Development Kit (JDK) installed on your development machine. Databricks recommends that the version of your JDK installation matches the JDK version on your Databricks cluster. To find the JDK version of the Databricks Runtime on your cluster, refer to the System environment section of the Databricks Runtime release notes or the version compatibility table.
note
Choosing a JDK install that is below or above the JDK version on your cluster might produce unexpected results, or your code might not run at all.
You have Scala installed on your development machine. Databricks recommends that the version of your Scala installation matches the Scala version on your Databricks cluster. To find the Scala version of the Databricks Runtime version of your cluster, refer to the System environment section of the Databricks Runtime release notes or the version compatibility table.
If you are using user-defined functions (UDFs), the local Scala and Java versions must match the Scala and Java versions of the Databricks Runtime version of the cluster. To find the Scala and Java versions of the Databricks Runtime version of your cluster, refer to the System environment section of the Databricks Runtime release notes or the version compatibility table.
A Scala build tool on your development machine, such as sbt.
The following table shows compatible Databricks Connect, JDK, and Scala versions. Databricks Connect version numbers correspond to Databricks Runtime version numbers.
Add a reference to the Databricks Connect clientâTo set up the Databricks Connect client, first add a reference to the client. In your Scala project's build file such as build.sbt for sbt, pom.xml for Maven, or build.gradle for Gradle, add the following reference to the Databricks Connect client. Replace the version number with the version of the Databricks Connect library that matches the Databricks Runtime version on your cluster. You can find the Databricks Connect library version numbers in the Maven central repository.
libraryDependencies += "com.databricks" % "databricks-connect" % "14.0.0"
Or for Databricks Connect 17.0 and above:
libraryDependencies += "com.databricks" %% "databricks-connect" % "17.0.0"
XML
<dependency>
<groupId>com.databricks</groupId>
<artifactId>databricks-connect</artifactId>
<version>14.0.0</version>
</dependency>
Or for Databricks Connect 17.0 and above:
XML
<dependency>
<groupId>com.databricks</groupId>
<artifactId>databricks-connect_2.13</artifactId>
<version>17.0.0</version>
</dependency>
implementation 'com.databricks.databricks-connect:14.0.0'
Or for Databricks Connect 17.0 and above:
implementation 'com.databricks.databricks-connect_2.13:17.0.0'
Next, configure properties to establish a connection between Databricks Connect and your remote Databricks cluster. These properties include settings to authenticate Databricks Connect with your cluster. See Compute configuration for Databricks Connect.
For Databricks Connect for Databricks Runtime 13.3 LTS and above, for Scala, Databricks Connect includes the Databricks SDK for Java. This SDK implements the Databricks client unified authentication standard, a consolidated and consistent architectural and programmatic approach to authentication. This approach makes setting up and automating authentication with Databricks more centralized and predictable. It enables you to configure Databricks authentication once and then use that configuration across multiple Databricks tools and SDKs without further authentication configuration changes.
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4