Use NiFi to write into AWS S3 Bucket with MFA enabled

Question

Is it possible to use the NiFi processor PutS3Object to write data into a S3 Bucket, while MFA is active?

And if not, what would be an alternative considering writing data to S3 from hive/HDFS?

Answer 1

As of version 1.8.0, I do not believe this is supported by the Apache NiFi *S3Object processors (or the AWSCredentialsProviderControllerService). I recommend opening a feature request ticket on the Jira site. You may be able to accomplish this immediately by using one of the following options:

• Use ExecuteStreamCommand or ExecuteProcess to call a terminal command or shell script using the AWS S3 CLI
• Use ExecuteScript to run a custom Python (/Ruby/Groovy/etc.) script which uses the AWS S3 SDK directly (example code)
• Write a custom processor which uses the AWS S3 SDK directly