Search code examples
mysqlapache-sparkapache-spark-sqlrdd

How to put data from Spark RDD to Mysql Table

I have to move data from Spark RDD to Mysql Table. Can someone help me on this?

Solution

  • An example with may possibilities, but here is one style to get you going:

    import org.apache.spark.sql.SaveMode

// Write out and create the table as well to mysql via DF Writer API 

val RDD = spark.sparkContext.parallelize(Seq(
    ("A2", "X", "done"),
    ("A2", "Y", "done"),
    ("C2", "Y", "done"),
    ("B2", "Y", "done"),
    ("Z2", "Z", "done")
  ))

val jdbcDF = RDD.toDF("Company", "Type", "Status")


// Saving data to a JDBC source - creates the table as well 

jdbcDF.write
   .format("jdbc")
   .option("url", "jdbc:mysql://db4free.net:3306/mySQLDB")
   .option("dbtable", "tabname")
   .option("user", "xxx")
   .option("password", "yyy")
 //.save()
   .mode(SaveMode.Append)

    You may need to set driver as I ran this under Databricks.