Home - News ( United States | United Kingdom | Italy | Germany ) - Football scores

Showing content from https://github.com/housepower/spark-clickhouse-connector below:

ClickHouse/spark-clickhouse-connector: Spark ClickHouse Connector build on DataSourceV2 API

Build on Apache Spark DataSourceV2 API.

See the documentation for how to use this connector.

• Java 8 or 17
• Scala 2.12 or 2.13
• Apache Spark 3.3 or 3.4 or 3.5

Notes:

1. As of 0.5.0, this connector switches from ClickHouse raw gRPC Client to ClickHouse Official Java Client, which brings HTTP protocol support, extends the range of supported versions of ClickHouse Server.
2. Since 0.6.0, HTTP becomes the default protocol.
3. Since 0.7.0, gRPC is deprecated and not recommended, it may be removed in the future.
4. Since 0.8.0, gRPC is removed.
5. Since 0.8.0, the project groupId is rename from com.github.housepower to com.clickhouse.spark, and the class xenon.clickhouse.ClickHouseCatalog is renamed to com.clickhouse.spark.ClickHouseCatalog

Build w/o test

./gradlew clean build -x test

The project leverage Testcontainers and Docker Compose to do integration tests, you should install Docker and Docker Compose before running test, and check more details on Testcontainers document if you'd like to run test with remote Docker daemon.

Run all test

./gradlew clean test

Run all test w/ Spark 3.3 and Scala 2.13

./gradlew clean test -Dspark_binary_version=3.3 -Dscala_binary_version=2.13

Run single test

./gradlew test --tests=ConvertDistToLocalWriteSuite

Test against custom ClickHouse image

CLICKHOUSE_IMAGE=custom-org/clickhouse-server:custom-tag ./gradlew test

RetroSearch is an open source project built by @garambo | Open a GitHub Issue

Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo

HTML: 3.2 | Encoding: UTF-8 | Version: 0.7.4