Search code examples
azureazure-data-factoryetl

ADF Copy Files From VM To Storage Account No more connections can be made to this remote computer at this time

I have a copy activity in a pipeline. The activity is coping files from a VM to a Storage Account. The source directory has over a million files in it with a total size of about 1.5TB. I am trying to take chunks of files at a time not the entire directory. I am using a table with start and end quarter dates. The first step in the pipeline reads from this date table to get the series of dates to process. Then in the copy activity I use those dates to filter by the date modified in the source directory. This works for the first few iterations but then I get the following error.

Operation on target CopyFromVMToStorageAcct failed: ErrorCode=FileUnexcepetedIOError,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Error occurred when trying to access the file in Folder ***, File filter: ''.,Source=Microsoft.DataTransfer.Common,''Type=System.IO.IOException,Message=No more connections can be made to this remote computer at this time because there are already as many connections as the computer can accept.
,Source=mscorlib,'

I need to figure out how to make sure that the copy activity is actually closing the connection to the source computer before moving to the next date range. Or I need to have the source computer allow all the additional connections.

I have seen other people edit their registry setting to increase the 'CachedLogonsCount'. The default value is 10 and I have bumped mine up to 15 but that didn't help at all.

Solution

  • No more connections can be made to this remote computer at this time because there are already as many connections as the computer can accept.

    The error is saying that there are multiple connections are getting created and it is reaching the limit of VM connections.

    To resolve It you can try below resolutions:

    1. Optimize your copy activity to limit the concurrent connections with maxConcurrentConnections Option. The maximum number of connections that can be made simultaneously to the data store while an activity is running. Only when you wish to restrict concurrent connections should you specify a value. enter image description here For above example it will restrict the concurrent connections with your specified value.
    2. Try by updating the following registry settings as suggested in this Document:
    [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Lanmanserver\Parameters]  
MaxMpxCt = 20 (Default = 10 commands)