How to calculate Dynamodb item size? Getting validationerror 400KB Boto

Question

ValidationException: ValidationException: 400 Bad Request {u'message': u'Item size has exceeded the maximum allowed size', u'__type': u'com.amazon.coral.validate#ValidationException'}

The item object I have, has size of 92004 Bytes

>>> iii
<boto.dynamodb2.items.Item object at 0x7f7922c97190>
>>> iiip = iii.prepare_full() # it is now in dynamodb format e.g. "Item":{"time":{"N":"300"}, "user":{"S":"self"}} 

>>> len(json.dumps(iiip))
92004
>>>

The size I get 92004 is less than 400KB, Why do I see the above mentioned error when saving the item?

Any pointers?

EDIT:

I played around with different sizes of data,

>>> i00['Resources'] = "A" * 66848; len(json.dumps(i00))
68481
>>> i = Item(ct.table, data=i00); i.save()
True
>>> i.delete()
True
>>> i00['Resources'] = "A" * 66849; len(json.dumps(i00))
68482
>>> i = Item(ct.table, data=i00); i.save()
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/var/www/virtualenv/ken/local/lib/python2.7/site-packages/boto/dynamodb2/items.py", line 455, in save
    returned = self.table._put_item(final_data, expects=expects)
  File "/var/www/virtualenv/ken/local/lib/python2.7/site-packages/boto/dynamodb2/table.py", line 835, in _put_item
    self.connection.put_item(self.table_name, item_data, **kwargs)
  File "/var/www/virtualenv/ken/local/lib/python2.7/site-packages/boto/dynamodb2/layer1.py", line 1510, in put_item
    body=json.dumps(params))
  File "/var/www/virtualenv/ken/local/lib/python2.7/site-packages/boto/dynamodb2/layer1.py", line 2842, in make_request
    retry_handler=self._retry_handler)
  File "/var/www/virtualenv/ken/local/lib/python2.7/site-packages/boto/connection.py", line 954, in _mexe
    status = retry_handler(response, i, next_sleep)
  File "/var/www/virtualenv/ken/local/lib/python2.7/site-packages/boto/dynamodb2/layer1.py", line 2882, in _retry_handler
    response.status, response.reason, data)
ValidationException: ValidationException: 400 Bad Request
{u'message': u'Item size has exceeded the maximum allowed size', u'__type': u'com.amazon.coral.validate#ValidationException'}

In other words, the size of cloudtrail data has to be less than 68482 bytes. I wonder why they claim it to be 400KB. Clearly, I am missing something.

Answer 1

Answering my own question since it might help someone with the same problem. I contacted aws technical support and here is the explanation:

I had 5 indexes on my dynamodb table, since the data is replicated for each index; the total data = 68481 * (5 + 1) = 410886 which is close to 400KB.

I feel this is missing from Dynamodb documentation and it'd be nice if amazon adds it.

So, to summarize, the total data (item size) that ends up being saved in dynamodb table is Actual data * (number of indexes + 1).