Search code examples
apache-kafkaparquetapache-kafka-connect

Kafka-connect file sink connector write in parquet file format

I am looking for kafka-connect connector that will write from kafka to local file system in parquet file format. I don't want to use the hdfs or s3 sink connector for the same.

Solution

  • format.class=ParquetFormat only exists in the mentioned connectors

    You can use file:// prefix to write to local disk, or use a project like MinIO to reproduce self-hosted S3