Search code examples
rapache-sparksparkrsparklyr

How to stop Spark without exiting RStudio when using sparklyr?

I can easily stop Spark when I'm using SparkR with sparkR.stop() as in the code block below:

library(SparkR)
sparkR.session(master = "local")
sparkR.conf(SPARK_HOME)
sparkR.stop()
detach("package:SparkR", unload=TRUE)

How can i do the same when I'm running Sparklyr?

library(sparklyr)
library(dplyr)
sc <- spark_connect(master = "local")
????? what do I enter here ???
detach("package:dplyr", unload=TRUE)
detach("package:sparklyr", unload=TRUE)
Solution

  • Try to use spark_disconnect()

    sc <- spark_connect(master = "local") # open the connection
spark_disconnect(sc) # close the connection