Search code examples
amazon-web-servicesamazon-dynamodbamazon-sns

DynamoDB master and child tables as Publisher/ Subscriber in Amazon SNS

I have two DynamoDB tables where one table is having records of Publishers (master table) and the other table is having records of Subscribers (child table).

I want to build a system where a change in the Publisher side leads to all Subscribers being notified. I have checked the documentation of AWS SNS, subscriber could be Lambda or SQS etc.

Is there any way I can make my child records as subscribers in DynamoDB?

So that whenever there is change in publisher records all the child records should be notified immediately.

Solution

  • To expand on luk2302 completely valid comment:

    There are a few misconceptions here:

    • There is no direct integration between DynamoDB and SNS, so subscribing to individual items in a table doesn't work
    • You can't manage the Subscribers for SNS outside of SNS, they need to be subscribed within SNS

    The easiest way to implement the Publisher/Subscriber Pattern would indeed be SNS, but you can't just write to a DynamoDB table and integrate it. If you want a DynamoDB based solution, I'd probably start like this.

    There will be two tables:

    • Publishers with a composite Primary Key of PK and SK and DynamoDB streams enabled
    • Subscribers with a composite Primary Key of PK and SK

    Also there is a Lambda function called StreamProcessor that has the Publisher table's stream as an input.

    Publishers looks like this

    PK SK Message
    P#<id> M#<id0> Some Message
    P#<id> M#<id1> Some Message
    P#<id> M#<id2> Some Message
    P#<id> M#<id3> Some Message

    P#<id> with <id> being the id of the Publisher and the M#<id0> being a message ID.

    Subscribers looks like this:

    PK SK Contact
    P#<id> S#<id0> test1@mail.com
    P#<id> S#<id1> test2@mail.com
    P#<id> S#<id2> test3@mail.com
    P#<id> S#<id3> test4@mail.com

    P#<id> with <id> being the id of the Publisher and the S#<id0> being a subscription ID.

    Now whenever there is a change in the Publisher table (e.g. a new message gets added to a publisher), the StreamProcessor Lambda will be called. The StreamProcessor can then do something like this:

    1. Extract the Publisher ID from the update record
    2. Query the Subscribers table with PK=P#<id> to get a list of subscribers
    3. For each result:
    4. Determine the mode of contact
    5. Send a message to the subscriber