Amazon DMS not fully replicating Deleted CDC records

Question

I've followed all the instruction to the dot in the official Amazon User Guide of setting up an DMS task here

My particular use-case, i.e. getting data from Amazon RDS postgres to s3, is for initial-load + ongoing change.

The particular instruction in the Guide are under 'Working with AWS-managed PostgreSQL databases as DMS sources'

Here's what the cdc records look like for a new record, where we update email and then delete it:

Op, id, name, email, address_line_1, address_line_2

I, 1, 'Sal', 'one@email.com', 'some street', 'some place'

U, 1, 'Sal', 'another@email.com', 'some street', 'some place'

D, 1,,,,

Here's what according to the documentation they should look likehere

D, 1, 'Sal', 'another@email.com', 'some street', 'some place'

So basically, rather than getting the before image of the row upon delete, we are only getting the primary key + Op

I've tried out everything I can think of at this point - but I've had trouble accessing replication store in postgres to see if deletes aren't being recorded in WAL or if they're just not being migrated.

Any help will be appreciated

Answer 1

I found the answer finally - putting it here in case anyone is in a similar rut - The replicate identity of the tables need to be FULL:

Here's the query:

ALTER TABLE table_name REPLICA IDENTITY FULL