Search code examples
dataframescalaapache-sparkamazon-s3parquet

Getting FileNotFoundException exception while writing data into S3 bucket

While inserting into S3 bucket getting below error

Caused by: java.io.FileNotFoundException: File s3n://data/_temporary/0/_temporary/attempt_20230704174844_0087_m_000005_0 does not exist.

df.write.format("parquet").partitionBy("year")
             .mode(SaveMode.Append)
             .save("s3n://data/")
Solution

  • It is referring the S3 file as s3n://, AFAIK the correct S3 URI is just s3://