I am trying to connect R with a local instance of Spark using Rstudio. However, I get the error message shown. What am I missing? I am using windows 10. I am following the tutorial on rstudio.
library(sparklyr)
spark_install(version = "1.6.1")
library(sparklyr)
library(dplyr)
sc <- spark_connect(master = "local")
The error message I am getting:
Error in start_shell(scon, list(), jars, packages) :
Failed to launch Spark shell. Ports file does not exist.
Path: C:\Users\username\AppData\Local\rstudio\spark\Cache\spark-1.6.1- bin-hadoop2.6\bin\spark-submit.cmd
Parameters: --packages "com.databricks:spark-csv_2.11:1.3.0,com.amazonaws:aws-java-sdk-pom:1.10.34" --jars "C:\Users\user\Documents\R\win-library\3.3\sparklyr\java\rspark_utils.jar" sparkr-shell C:\Users\aaaa\AppData\Local\Temp\Rtmpw9PBxS\filea5860886dd1.out
'cmd' is not recognized as an internal or external command,
operable program or batch file.
Adding C:\Windows\system32 to the System Variables Path solved it. Before, it was added to User variables Path only which resulted in the error message.