I am newbie to Cassandra database and I am trying to save my Spark dataframe to the Cassandra DB.
While creating the table I am getting the exception. "SyntaxException: no viable alternative at input"
.
val sparkContext = spark.sparkContext
//Set the Log file level
sparkContext.setLogLevel("WARN")
//Connect Spark to Cassandra and execute CQL statements from Spark applications
val connector = CassandraConnector(sparkContext.getConf)
connector.withSessionDo(session =>
{
session.execute("DROP KEYSPACE IF EXISTS my_keyspace")
session.execute("CREATE KEYSPACE my_keyspace WITH replication = {'class':'SimpleStrategy', 'replication_factor':1}")
session.execute("USE my_keyspace")
session.execute("CREATE TABLE mytable('Inbound_Order_No' varchar,'Material' varchar,'Container_net_weight' double,'Shipping_Line' varchar,'Container_No' varchar,'Month' int,'Day' int,'Year' int,'Job_Run_Date' timestamp, PRIMARY KEY(Inbound_Order_No,Container_No))")
df.write
.format("org.apache.spark.sql.cassandra")
.mode("overwrite")
.option("confirm.truncate", "true")
.option("spark.cassandra.connection.host", "localhost")
.option("spark.cassandra.connection.port", "9042")
.option("keyspace", "my_keyspace")
.option("table", "mytable")
.save()
}
)
I am unable to trace the error, hence seeking the help.
Please note:I am doing this work in windows system and everything is setup locally. I have also shared my spark code if you find any other error then please do share with me.
session.execute("CREATE TABLE mytable(\"Inbound_Order_No\" varchar,\"Material\" varchar,\"Container_net_weight\" double,\"Shipping_Line\" varchar,\"Container_No\" varchar,\"Month\" int,\"Day\" int,\"Year\" int,\"Job_Run_Date\" timestamp, PRIMARY KEY(\"Inbound_Order_No\",\"Container_No\"))")
Double quote is used for case sensitive columns and not single quote.
session.execute("CREATE TABLE mytable(Inbound_Order_No varchar,Material varchar,Container_net_weight double,Shipping_Line varchar,Container_No varchar,Month int,Day int,Year int,Job_Run_Date timestamp, PRIMARY KEY(Inbound_Order_No,Container_No))")
If you want column names in lower case the use above query.. Cassandra will create column names in lower case by default (If not enclosed in double quotes)
As requested in comment command to run in cqlsh:
CREATE TABLE mytable("Inbound_Order_No" varchar,"Material" varchar,"Container_net_weight" double,"Shipping_Line" varchar,"Container_No" varchar,"Month" int,"Day" int,"Year" int,"Job_Run_Date" timestamp, PRIMARY KEY("Inbound_Order_No","Container_No"))