{Py4JJavaError}An error occurred while calling o339.save

Question

I trained Logistic regression model in pyspark but couldn't save the model.

Model = LogisticRegression(featuresCol='TF-IDF', labelCol='labels', maxIter=10)`
lr_model = Model.fit(train_data)`
type(lrmodel) /*<class pyspark.ml.classification.LogisticRegressionModel'>
lr_model.write().overwrite().save("MODEL_LR")

I'm facing error that

{Py4JJavaError}An error occurred while calling o384.save. ExitCodeException exitCode=-1073741515: at org.apache.hadoop.util.Shell.runCommand(Shell.java:1007) at org.apache.hadoop.util.Shell.run(Shell.java:900) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1212) at org.apache.hadoop.util.Shell.execCommand(Shell.java:1306) at org.apache.hadoop.util.Shell.execCommand(Shell.java:1306) at org.apache.hadoop.util.Shell.execCommand(Shell.java:1288) at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:978) at org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:660) at org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:700) at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:672) at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:672) at org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:699) at

org.apache.hadoop.fs.RawLocalFileSystem....

I also tried lr_model.save("MODEL_LR") but same error throws.

Answer 1

Issue resolved by following steps:

1. install microsoft visual c++ 2010 from this https://www.microsoft.com/en-au/download/details.aspx?id=26999
2. set Environmental variables, once pyspark environment setup completed.
3. when using IDE, use import findspark findspark.init()