Is there any asynchronous way to connect to an SFTP server?

Question

I am working on an API that receives messages and uploads them to an SFTP server in a .json format. However, the connection time is taking too long, and the SFTP Connection takes over 1 second, which is too much. I want to know if there is any way to connect to the server and send the file without waiting for so long.

I have thought about saving the files, answering the client, and then uploading the file with this method:

public async Task Upload(string localPath, string remotePath)
{
    await Task.Run(() =>
    {
        // upload code here
    });
}

However, if an exception occurs during the upload, the API might have already responded with an OK to the client, making it impossible to notify the client that the upload failed.

Here's the existing code that I am currently using to upload files:

public void Upload(string localPath, string remotePath)
{
    try
    {
        using (var sftp = new SftpClient(...))
            {
                sftp.Connect(); // Very slow
                sftp.UploadFile(...);
                sftp.Disconnect();
    }
    catch (Exception ex)
    {
        _logger.LogError(ex, "Error Upload SFTPHandler");
    }
}

Can someone suggest a solution to connect to the SFTP server and send the file without waiting for so long?

Answer 1

However, if an exception occurs during the upload, the API might have already responded with an OK to the client, making it impossible to notify the client that the upload failed.

This is true for all "fire-and-forget" work, which is why it's often the wrong solution.

If you absolutely can't wait more than 1 second (?? doesn't sound too unreasonable to me), then your best option is a basic distributed architecture, as described on my blog. The basic distributed architecture has two parts, with an optional third part (which it sounds like you would want):

1. A durable queue. This is necessary to prevent the work from being lost.
2. A background process that processes the queue.
3. Client-side polling or notification to inform the client when the work completes.