Search code examples
jsonscalaapache-sparkioerror

Spark java.io.IOException: No input paths specified in job

I have the following error:

java.io.IOException: No input paths specified in job

When I execute the following line:

val df = sqlContext.read.json("file://home/test.json")

Any ideas?

Solution

  • If you want to load data from local please use 

    sqlContext.read.json("file:///home/shankar/test.json")

    or just

    sqlContext.read.json("/home/shankar/test.json")

    will work

    If you want to load the file from HDFS use

    sqlContext.read.json("hdfs:///user/shankar/test.json")

    Hope this helps!