Search code examples
azureamazon-s3azure-blob-storageazcopy

Any best alternative for Azcopy for data movement?

I have 100s of TB data to move from S3 to blob storage. Is there any best alternative of Azcopy? Because Azcopy use high bandwidth and have high CPU usage. I don't want to use It. In Azcopy v10 still these issues are coming after applying the required parameters. Can someone help me in this regard, I did R&D on It but not found any alternate.

Solution

  • I agree with @S RATH.

    For big data moving, Data Factory is the best alternative of Azcopy. It has the better Copy performance : enter image description here

    Data Factory support Amazon S3 and Blob Storage as the connector.

    With Copy active, You could create the Amazon S3 as the source dataset and Blob Storage as Sink dataset.

    Ref these tutorials:

    1. Copy data from Amazon Simple Storage Service by using Azure Data Factory: This article outlines how to copy data from Amazon Simple Storage Service (Amazon S3). To learn about Azure Data Factory, read the introductory article.
    2. Copy and transform data in Azure Blob storage by using Azure Data Factory: This article outlines how to use the Copy activity in Azure Data Factory to copy data from and to Azure Blob storage. It also describes how to use the Data Flow activity to transform data in Azure Blob storage. To learn about Azure Data Factory, read the introductory article.

    Data Factory also provide many ways to improve the data copy performance, ref: Copy activity performance and scalability guide

    I thinks it will help you save much time, as we know, time is money.