Sporadic "File is being used by another process" when trying to open a file after copying to a network share

Question

In our application we have an ASP.NET web server and an API server for taking on various jobs. In this particular workflow the user uploads a file that the Web server copies to a network share for the Job server to consume. The Job server is notified of this through a new row in an MSSQL table which it scans every second. Sometime this year during integration testing this flow began randomly failing with:

System.IO.IOException: The process cannot access the file '\\a\\b\\c.xlsx' because it is being used by another process."

Locally DEVs and QA never had this issue, but I assume it's because their shared folders are placed on their respective work machines. When the shared folder is moved from the local file system to a network share, the same exception begins to appear.

I am able to reproduce on a random occasion with this short application:

static void Main(string[] args)
{
    const string DestinationDirectory = "\\\\10.122.xxx.xxx\\Testing";

    var inputPath = args[0];
    var filename = Path.GetFileName(inputPath);
    var destinationPath = Path.Combine(DestinationDirectory, filename);

    Console.WriteLine(filename);

    Console.WriteLine("Copying the source file...");
    File.Copy(inputPath, destinationPath, true);

    Console.WriteLine("Opening the destination file...");
    try
    {
        using var _ = File.Open(destinationPath, FileMode.Open, FileAccess.Read);
        Console.WriteLine("Success!");
    }
    catch (Exception ex)
    {
        Console.WriteLine(ex);
    }
}

If I run it with this batch script I hit the exception roughly 1 in 10 times.

@echo off

for /l %%x in (1, 1, 10) do (
    FileShareDemo.exe "C:\a\b\c\myfile"
)

myfile
Copying the source file...
Opening the destination file...
System.IO.IOException: The process cannot access the file '\\10.122.xxx.xxx\Testing\myfile' because it is being used by another process.
   at Microsoft.Win32.SafeHandles.SafeFileHandle.CreateFile(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options)
   at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
   at System.IO.Strategies.OSFileStreamStrategy..ctor(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
   at System.IO.Strategies.FileStreamHelpers.ChooseStrategyCore(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
   at System.IO.File.Open(String path, FileMode mode, FileAccess access)
   at FileShareDemo.Program.Main(String[] args)

• From my testing, it seems that the file gets locked for a tiny amount of time, hence the sporadic nature.
• Issue present on .NET 6 and .NET 8.
• It could well be a Windows related issue, but running this batch script:

@echo off

set file=myfile
set source="C:\a\b\c\%file%"
set destination="\\10.122.xxx.xxx\Testing\%file%"

for /l %%x in (1, 1, 10) do (
    echo Copying...
    copy %source% %destination%
    
    echo Reading...
    type %destination%
    REM powershell -command "Get-Content %destination% -Encoding byte -TotalCount 2"
)

which performs the same commands, does not trigger the error, so I'm led to believe it's .NET related.

I have tried to use the Handle utility from Microsoft to find all file associated handles before and after file open, but the application is too slow to register them.

Answer 1

It seems that something on Windows locks new files for the tiniest moment, and the solution to that, is to try accessing the file again after a short period. We put a 5 second delay, and the issue has been gone since.