Search code examples
javaamazon-s3parquet

Read file parquet file from s3 in java


I am new to Java & spark sql. I wanted to know if there is a way to read a file of type Spark Parquet format saved as JSON in S3 in my Java code?


Solution

  • 
    val spark = SparkSession.builder()
          .master("local[1]")
          .getOrCreate()
    
    spark.read().json("s3a://your/json/path.json")
    
    spark.write().parquet("parquet/file/path.parquet")