Write dataframe to Teradata in Spark

Question

I have values in dataframe , and I have created a table structure in Teradata. My requirement is to load dataframe to Teradata. But I am getting error:

I have tried following code :

df.write.format("jdbc")
  .option("driver","com.teradata.jdbc.TeraDriver")
  .option("url","organization.td.intranet")
  .option("dbtable",s"select * from td_s_zm_brainsdb.emp")
  .option("user","userid")
  .option("password","password")
  .mode("append")
  .save()

I got an error :

java.lang.NullPointerException at org.apache.spark.sql.execution.datasources.jdbc.JdbcRelationProvider.createRelation(JdbcRelationProvider.scala:93) at org.apache.spark.sql.execution.datasources.DataSource.write(DataSource.scala:518) at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:215) ... 48 elided

I changed url option to make it similar to jdbc url, and ran following command:

df.write.format("jdbc")
  .option("driver","com.teradata.jdbc.TeraDriver")
  .option("url","jdbc:teradata//organization.td.intranet,CHARSET=UTF8,TMODE=ANSI,user=G01159039")
  .option("dbtable",s"select * from td_s_zm_brainsdb.emp")
  .option("user","userid")
  .option("password","password")
  .mode("append")
  .save()

Still i am getting error:

java.lang.NullPointerException at org.apache.spark.sql.execution.datasources.jdbc.JdbcRelationProvider.createRelation(JdbcRelationProvider.scala:93) at org.apache.spark.sql.execution.datasources.DataSource.write(DataSource.scala:518) at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:215) ... 48 elided

I have included following jars:

with --jars option 
tdgssconfig-16.10.00.03.jar
terajdbc4-16.10.00.03.jar
teradata-connector-1.2.1.jar

Version of Teradata 15 Spark version 2

Answer 1

The JDBC Url should be in the following form :

val jdbcUrl = s"jdbc:teradata://${jdbcHostname}/database=${jdbcDatabase},user=${jdbcUsername},password=${jdbcPassword}"

It was causing an exception, because I didn't supply username and password.