Search code examples
amazon-web-serviceshiveamazon-dynamodbemramazon-emr

Amazon DynamoDB: warning "Configured write throughput of the dynamodb table JobDetails is less than the cluster map capacity"

I'm using Apache Hive on Amazon EMR to batch download some data from Amazon DynamoDB to Amazon S3. I have received the following warning:

WARNING: Configured write throughput of the dynamodb table JobDetails is less than the cluster map capacity. ClusterMapCapacity: 20 WriteThroughput: 5

WARNING: Writes to this table might result in a write outage on the table.

I think WriteThroughput is related to the configuration of my DynamoDB table.

What does this warning mean? Is the ClusterMapCapacity related to my EMR cluster?

Is this warning a problem I should address, or I can just ignore it?

Solution

  • Yes, you should address this problem.

    You can have up to 20 map jobs writing to the table simultaneously, while table accepts only 5 write requests per second.

    So, you need to have at least 20 write capacity on table (or even more if map can write multiple times per second) to avoid throttling (throttling means that data can be lost).