Spark AQE Post-Shuffle partitions coalesce don't work as expected, and even make data skew in so...

Spark ignores Iceberg Nessie catalog...

Convert date to ISO week date in Spark...

Pyspark.ml - Error when loading model and Pipeline...

Why are spark3 dynamic partitions slow to write to hive...

Start of the week on Monday in Spark...

java.lang.NullPointerException while reading specific sheet from xlsx using org.zuinnote.spark.offic...

Adaptive Query Execution and Shuffle Partitions...

Spark 3 KryoSerializer issue - Unable to find class: org.apache.spark.util.collection.OpenHashMap...

Why would finding an aggregate of a partition column in Spark 3 take very long time?...

Spark: DF.as[Type] fails to compile...

Spark can't connect to DB with built-in connection providers...

Date from week date format: 2022-W02-1 (ISO 8601)...

Spark 3.0 is much slower to read json files than Spark 2.4...

Create a lookup column in pyspark...

Scala: Parse timestamp using spark 3.1.2...

How to save spark dataset in encrypted format?...

SPARK 3 - Populate value with value from previous rows (lookup)...

AnalysisException when loading a PipelineModel with Spark 3...

Does Apache Spark 3 support GPU usage for Spark RDDs?...

spark struct streaming writeStream output no data but no error...

org.apache.spark.shuffle.FetchFailedException: Connection from server1/xxx.xxx.x.xxx:7337 closed...

Spark 3.0 streaming metrics in Prometheus...

Spark 3.0.1 tasks are failing when using zstd compression codec...

Does Spark 3.0.1 support custom Aggregators on window functions?...

How to access Spark DataFrame data in GPU from ML Libraries such as PyTorch or Tensorflow...

How to create a map column to count occurrences without udaf...

find set of keys in Scala map where values overlap...

How to port UDAF to Aggregator?...

Pyspark SelectExp() not working for first() and last()...