Search code examples
windowsscalaapache-spark

How to access local files in Spark on Windows?

I am using Spark on Windows. I know in *nix, accessing local file code likes this: 

val textFile = sc.textFile("file:///usr/local/spark/README.md")

But how can I access a local file on Windows? I have tried following methods: 

val logFile = "C:\spark-1.3.1-bin-hadoop2.4\README.md"
val logFile = "file\\C:\spark-1.3.1-bin-hadoop2.4\README.md"

But all can't work.

Solution

  • Unfortunately in windows you have to escape "\".

    Try:

    "C:\\spark-1.3.1-bin-hadoop2.4\\README.md"