I'm trying to pass the data between my Azure-files NFS to AWS EFS. In my Azure file NFS I have 11TB of data, and the AWS EFS is empty.
I tried to pass the data between the Azure-files NFS to EFS the following way:
In my ec2 server I have two mounts - one for Azure-files NFS (in /mnt/shared), and one for EFS (in /mnt/efs).
I ran the following command to copy the files:
rsync -avu /mnt/shared/ /mnt/efs/ &
After few hours I saw that it stopped (after passing 200 gb). After reading answers online, I tried new command:
rsync -auq --whole-file /mnt/shared/ /mnt/efs/ &
But again it stopped in the middle (I don't know why).
Any idea why it stopped, and what is the best and easiest way to pass the data?
• I would suggest you use AWS Datasync for transferring large amounts of data from Azure NFS to AWS EFS from the linux EC2 instance as simplifies, automates, and accelerates moving and replicating data between on-premises storage systems, and between AWS storage services. DataSync can copy data between Network File System (NFS), Server Message Block (SMB) file servers, self-managed object storage, AWS Snowcone, Amazon S3 buckets, Amazon EFS file systems, and FSx for Windows File Server file systems.
To transfer files between two different file systems using DataSync, kindly follow the below documentation link for configuring the same: -
https://docs.aws.amazon.com/efs/latest/ug/gs-step-four-sync-files.html
Also, you can consider using your Amazon EFS file system to work with AWS Transfer Family as described in the below AWS documentation link: -
https://docs.aws.amazon.com/efs/latest/ug/using-aws-transfer-integration.html