Search code examples
dataframeapache-sparkclickhouse

How can I write spark Dataframe to clickhouse

val df = spark.read.parquet(path)
val IP ="190.176.35.145"
val port = "9000" 
val table = "table1"
val user = "defalut"
val password = "default"

I don't know how to write df directly into clickhouse, and I am not finding any similar answers.

Solution

  • Writing to the clickhouse database is similar to writing any other database through JDBC. Just make sure to import the ClickHouseDriver class to your code. The username and password are passed into the ckProperties object. The write command is as follows, you can replace the database name in the string:

    import ru.yandex.clickhouse._

val jdbcUrl = "jdbc:clickhouse://190.176.35.145:9000/your_database_name"

val ckProperties = new Properties()

df.write.mode("append").option("driver", "ru.yandex.clickhouse.ClickHouseDriver").jdbc(jdbcUrl, table = "table1", ckProperties)