Search code examples
scalaapache-sparkcassandraspark-cassandra-connector

How to insert rows into cassandra if they don't exist using spark- cassandra driver?

I want to write to cassandra from a data frame and I want to exclude the rows if a particular row is already existing (i.e Primary key- though upserts happen I don't want to change the other columns) using spark-cassandra connector. Is there a way we can do that?

Thanks.!

Solution

  • You can use the ifNotExists WriteConf option which was introduced in this pr.

    It works like so:

    val writeConf = WriteConf(ifNotExists = true)
rdd.saveToCassandra(keyspaceName, tableName, writeConf = writeConf)