Search code examples
amazon-web-servicesamazon-s3parquet

Write to S3 in Parquet format in Python (or Typescript) without spark


I want to write data to S3 in Parquet format from inside AWS Lambda job, without using Spark. Either Python or Typescript should work. If the function cane to append to the existing Parquet file that would be even more awesome. Thank you!


Solution

  • Use PyArrow library or the AWS SDK for JavaScript in Typescript to write down data to S3 in Parquet format from inside an Lambda function.