The AWS docs makes it look as if you can specify DataFormatConversionConfiguration
for a AWS::KinesisFirehose::DeliveryStream
in cloudformation, but is not documentation on where the property is supposed to go. Tried adding it under Properties
, ExtendedS3DestinationConfiguration
, ProcessingConfiguration
, and one of the Processors
. Each time, CF complains with-
The following resource(s) failed to update: [EventFirehose]. 12:24:32 UTC-0500
UPDATE_FAILED AWS::KinesisFirehose::DeliveryStream EventFirehose Encountered unsupported property DataFormatConversionConfiguration
There own docs say-
If you want Kinesis Data Firehose to convert the format of your input data from JSON to Parquet or ORC, specify the optional DataFormatConversionConfiguration element in ExtendedS3DestinationConfiguration or in ExtendedS3DestinationUpdate.
What am I doing wrong?
As per SDK documentation, it should be inside ExtendedS3DestinationConfiguration
or ExtendedS3DestinationUpdate
. However, the cloudformation is currently NOT supporting this property as per this docs. This is a very common discrepancy between cloudformation and other AWS services. One similar issue is mentioned here AWS ECS: Severe bug in ECS Service Cloudformation template (which got resolved recently).
For the time being, you can update via SDK or wait for some time for cloudformation to catch up.
If you want Kinesis Data Firehose to convert the format of your input data from JSON to Parquet or ORC, specify the optional DataFormatConversionConfiguration element in ExtendedS3DestinationConfiguration or in ExtendedS3DestinationUpdate
Edit
As of June 2019, the DataFormatConversionConfiguration
property is in available in CloudFormation. See change log: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/ReleaseHistory.html