Below is my Scala code:
import scala.io.Source
case class TempData(day:Int , DayOfYear:Int , month:Int , year:Int ,
precip:Double , snow:Double , tave:Double, tmax:Double, tmin:Double)
object TempData {
def main(args:Array[String]) : Unit ={
val source = Source.fromFile("C:///DataResearch/SparkScala/MN212142_9392.csv")
val lines = source.getLines().drop(1)
val data= lines.map{ line =>
val p = line.split(",")
TempData(p(0).toInt , p(1).toInt, p(2).toInt, p(4).toInt
, p(5).toDouble, p(6).toDouble , p(7).toDouble, p(8).toDouble,p(9).toDouble)
}.toArray
source.close()
data.take(5).foreach(println)
}
}
Below is the error:
C:\Java\jdk1.8.0_144\bin\java "-javaagent:C:\Program Files\JetBrains\IntelliJ IDEA Community Edition 2017.3.3\lib\idea_rt.jar=2298:C:\Program Files\JetBrains\IntelliJ IDEA Community Edition 2017.3.3\bin" -Dfile.encoding=UTF-8 -classpath C:\Java\jdk1.8.0_144\jre\lib\charsets.jar;C:\Java\jdk1.8.0_144\jre\lib\deploy.jar;C:\Java\jdk1.8.0_144\jre\lib\ext\access-bridge-64.jar;C:\Java\jdk1.8.0_144\jre\lib\ext\cldrdata.jar;C:\Java\jdk1.8.0_144\jre\lib\ext\dnsns.jar;C:\Java\jdk1.8.0_144\jre\lib\ext\jaccess.jar;C:\Java\jdk1.8.0_144\jre\lib\ext\jfxrt.jar;C:\Java\jdk1.8.0_144\jre\lib\ext\localedata.jar;C:\Java\jdk1.8.0_144\jre\lib\ext\nashorn.jar;C:\Java\jdk1.8.0_144\jre\lib\ext\sunec.jar;C:\Java\jdk1.8.0_144\jre\lib\ext\sunjce_provider.jar;C:\Java\jdk1.8.0_144\jre\lib\ext\sunmscapi.jar;C:\Java\jdk1.8.0_144\jre\lib\ext\sunpkcs11.jar;C:\Java\jdk1.8.0_144\jre\lib\ext\zipfs.jar;C:\Java\jdk1.8.0_144\jre\lib\javaws.jar;C:\Java\jdk1.8.0_144\jre\lib\jce.jar;C:\Java\jdk1.8.0_144\jre\lib\jfr.jar;C:\Java\jdk1.8.0_144\jre\lib\jfxswt.jar;C:\Java\jdk1.8.0_144\jre\lib\jsse.jar;C:\Java\jdk1.8.0_144\jre\lib\management-agent.jar;C:\Java\jdk1.8.0_144\jre\lib\plugin.jar;C:\Java\jdk1.8.0_144\jre\lib\resources.jar;C:\Java\jdk1.8.0_144\jre\lib\rt.jar;C:\Users\Dell\IdeaProjects\ScalaSpark\target\scala-2.11\classes;C:\Users\Dell\.ivy2\cache\org.scala-lang\scala-library\jars\scala-library-2.11.12.jar TempData
Exception in thread "main" java.io.FileNotFoundException: C:\DataResearch\SparkScala\MN212142_9392.csv (The system cannot find the file specified)
at java.io.FileInputStream.open0(Native Method)
at java.io.FileInputStream.open(FileInputStream.java:195)
at java.io.FileInputStream.<init>(FileInputStream.java:138)
at scala.io.Source$.fromFile(Source.scala:91)
at scala.io.Source$.fromFile(Source.scala:76)
at scala.io.Source$.fromFile(Source.scala:54)
at TempData$.main(TempData.scala:7)
at TempData.main(TempData.scala)
Process finished with exit code 1
My file(MN212142_9392.csv) is in C drive inside folder DataResearch and SparkScala. I have tried all possible changes but nothing of any help.
Try to use File
class instead of Source
.
new File("path")
It should solve your problem :)
Or use class Files
:)
Below you can find links to documentation:
https://docs.oracle.com/javase/7/docs/api/java/io/File.html
https://docs.oracle.com/javase/7/docs/api/java/nio/file/Files.html
Double check your path. In my opinion better option is to use a relative path instead of absolute. Put this file in a project and try to use the relative path. Your application can't find your file :)
Edit:
In created a simple file TempData.scala which contains:
import scala.io.Source
object TempData {
def main(args:Array[String]) : Unit ={
val source = Source.fromFile("test.csv")
val lines = source.getLines()
}
}
I created file test.csv
I used scalac TempData.scala
and after that scala TempData
. Everything is okay, because you use methods in correct way. You have a problem with path. So my propose is try to use file that you put in the project and if everything is okay check your previous path :)