Search code examples
amazon-dynamodbamazon-dynamodb-streams

Get only updated fields along with identifiers in DynamoDB stream


Is there a way to get ONLY updated fields in an object along with some other attributes (ie identifiers for that object, be it primary key or whatever) On amazon documentation: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Streams.html I see we have four flavors: KEYS ONLY, NEW IMAGE, OLD IMAGE, NEW AND OLD IMAGES The only possible way I can see right now to fetch all updated attributes along with the identifier is using NEW_AND_OLD_IMAGES and then cross-checking each attribute in NEW IMAGE vs OLD IMAGE. Is there a better way going about this?


Solution

  • No, there is no way to do what you are hoping for. You could imagine a fifth StreamViewType which only stores the key and the updated attributes, but unfortunately DynamoDB Streams does not support it.

    Interestingly, though I doubt it helps you, ReturnValues does support the mode you were wishing for - seeUPDATED_NEW in UpdateItem's documentation. But alas it's not supported by DynamoDB streams.

    If you are willing to entertain alternatives to DynamoDB, there is the open-source Scylla Alternator which is compatible with DynamoDB and DynamoDB Streams, and its implementation of Streams is based on Scylla's CDC ("change-data capture"), which can natively capture deltas exactly like you hoped for. These deltas are not currently exposed in Scylla's DynamoDB Streams-compatible API because it's not part of that API - but such a feature can be added fairly easily.