apache-sparkkubernetesjupyter-notebookmlopsmlrun
Issue with MLRun Spark service start and impact all Jupyter notebooks
I reconfigured Spark infrastructure in K8s (as part of MLRun/Iguazio platform) and after that, I got a lot of issues in level of services:
- Spark service (with information
Failed) - All jupyter notebooks (with information
Failed dependencies)
and also general error/message:
Some services have not been successfully deployed. Check the services status as shown below.
See the print screen 
I changed only amount of RAM (1-30 GB RAM), vCPU (1-14) and Replicas (3).
Did you get the similar issue and how to avoid the situation?
Solution
It was human mistake, the solution was easy and the key problem was in Spark service configuration (I configured extremely small vCPU values and it generated timeouts for Spark service):
- I used setting vCPU in the range 1-14 but I used default units
millicpu(not cpu) - After setup correct units
cpuand restart of Spark service, everything was fine.
Wrong setting 
Correct setting 
- Spark dataframe not adding columns with null values
- Azure Data Factory trigger Databricks cluster termination
- Spark Streaming - Refresh Static Data
- How to use Jts in spark in java
- Casting RDD to a different type (from float64 to double)
- symbolic reference class is not accessible: class sun.util.calendar.ZoneInfo, from interface spark.sql.catalyst.util.SparkDateTimeUtils
- Removing NULL items from PySpark arrays
- What is the fastest way to calculate a daily balance with compound interest in Pandas or Spark?
- Spark dataframe write method writing many small files
- How to calculate the size of dataframe in bytes in Spark?
- Spark Row object instantiated differently from overloaded prototypes?
- How to copy and convert parquet files to csv
- How to tune spark executor number, cores and executor memory?
- Spark/pyspark on same version but "py4j.Py4JException: Constructor org.apache.spark.api.python.PythonFunction does not exist"
- How to disable all logging info to the spark console from .net application
- How to resolve the following AWS Glue error while writing to Redshift using Spark: "ORA-01722: invalid number"?
- how to avoid row number in read_sql output
- Spark getItem shortcut
- Spark: fill spec value between flag values
- How to read Parquet file from S3 without spark? Java
- spark dataframe sum of column based on condition
- Check whether boolean column contains only True values
- reading csv file with header and tail into apache arrow
- Pyspark - how to initialize common DataFrameReader options separately?
- Open, High, Low, Close, Volume in PySpark using tick data
- Dataframe.write() produces csv file on single node jobs cluster, but not on 2+1 nodes cluster
- Internals of worker/executor usage during coalesce/repartition
- How to run Multi threaded jobs in apache spark using scala or python?
- Including null values in an Apache Spark Join
- Why is my PySpark row_number column messed up when applying a schema?