Search code examples
apache-sparkjtds

spark driver not found


I am trying to write dataframe to sqlserver using spark. I am using the method write for dataframewriter to write to sql server. Using DriverManager.getConnection I am able to get connection of sqlserver and able to write but when using jdbc method and passing uri I am getting "No suitable driver found". I have passed the jtds jar in the --jars in spark-shell. Spark version : 1.4


Solution

  • The issue is that spark is not finding driver jar file. So download jar and place in all worker nodes of spark cluster on the same path and add this path to SPARK_CLASSPATH in spark-env.sh file as follow

    SPARK_CLASSPATH=/home/mysql-connector-java-5.1.6.jar
    

    hope it will help