Currently I am writing a test library to test the configuration settings. I would like to set only few parameters of firehose like SizeInMBs and IntervalInSeconds. All other parameters will remain same. Is there a simple way to do it?
I wrote the following method
def set_firehose_buffering_hints(self, size_mb, interval_sec):
response = self._firehose_client.describe_delivery_stream(DeliveryStreamName=self.firehose)
lambdaarn = (response['DeliveryStreamDescription']
['Destinations'][0]['ExtendedS3DestinationDescription']
['ProcessingConfiguration']['Processors'][0]['Parameters'][0]['ParameterValue'])
response = self._firehose_client.update_destination(DeliveryStreamName=self.firehose,
CurrentDeliveryStreamVersionId=response['DeliveryStreamDescription']['VersionId'],
DestinationId=response['DeliveryStreamDescription']['Destinations'][0]['DestinationId'],
ExtendedS3DestinationUpdate={
"BufferingHints": {
"IntervalInSeconds": interval_sec,
"SizeInMBs": size_mb
},
'ProcessingConfiguration': {
'Processors': [{
'Type': 'Lambda',
'Parameters': [
{
'ParameterName': 'LambdaArn',
'ParameterValue': lambdaarn
},
{
'ParameterName': 'BufferIntervalInSeconds',
'ParameterValue': str(interval_sec)
},
{
'ParameterName': 'BufferSizeInMBs',
'ParameterValue': str(size_mb)
}]
}]
}})