I have a .NET Windows Application which stores/retrieves data to/from a SQL Server. I also have a clone of this application running in cloud (created using AWS services). The app. in cloud uses DynamoDB. Now, I need to move the data from SQL Server to DynamoDB is two phases:
1) Bulk loading data from SQL Server to DynamoDB initially
2) Continuously moving data from SQL Server to DynamoDB e.g., when a new user is added in the .NET Windows Application, the user added in SQL Server should also be created in DynamoDB
Also, there is a third phase (vice-versa):
3) When a user is created in the cloud app., the user added in DynamoDB should also be created in the SQL Server.
Following are the few things I checked:
AWS Data Migration Service - I felt this is to bulk migrate data initially i.e., phase 1 only. Can we do continuous data migration using this?
I heard I can add the new database updates in SQS and it will be picked up and migrated in the other database
I need some advise on design techniques to achieve this.
As you already know, the first part can be done easily but you cannot use it for other 2 phases.
Below are the options you can try on: 1- write a small function in the .net code itself to do both phases 2 and 3 at any data trigger you need copy.
2- Create a stored proc, with an trigger event in both dbs to copy the newly entered data vice versa.
3- SQS can be used but it will be costly operation and will take more time for implementation as you need continuous feed.
So choose the one which is fitting around your needs and architecture requirements.
thanks