Search code examples
apache-flinknfsext4distributed-filesystem

Can Flink output be sinked to a NFS or GPFS file system?


Flink has a RollingFileSink which per the documentation works with HDFS (Hadoop). Are there similar classes which can be used to sink data to file systems such as NFS, ext4 or GPFS?

https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/connectors/filesystem_sink.html


Solution

  • The RollingSink class works with all Hadoop-compatible file systems (it simply uses Hadoop's FileSystem abstraction).

    The list of compatible file systems includes POSIX-compliant file systems (file://, which covers NFS, ext4, ...) and GPFS as well.