Multi-threaded loop through collection with C#

Question

I have seen lots of examples on how to do this (read through lots of threads on this site), but I'm having a very hard time getting something that works without locking up the UI thread AND allows me to control the number of threads executing at a time.

I have this method, which loops through the collection and calls an async method to process the items, but it locks up the UI...

    async Task ProcessItems()
    {
        using (var semaphore = new SemaphoreSlim(5))
        {
            var tasks = MyCollection.Select(async (MyItem) =>
            {
                await semaphore.WaitAsync();
                try
                {
                    await ProcessItem(MyItem);
                }
                finally
                {
                    semaphore.Release();
                }
            }).ToArray();

            await Task.WhenAll(tasks);
        }
    }

The 'ProcessItem' method calls another method, which makes some API calls over the internet to upload some files.

This has been working, but I have no control over how many threads are executed at a time...

    foreach (MyItemClass MyItem in MyCollection)
    {
        MyItemClass tmp = logItem;
        Thread thread = new Thread(() => ProcessItem(tmp));
        thread.Start();
    }

EDIT: Here's the ProcessItem method...

    public async Task<string> ProcessItem(MyItemClass MyItem)
    {
        MyItem.Status = "Transferring";

        HelperClass.UploadFileOrFolder(
            siteUrl,
            MyItem.DocumentLibrary,
            MyItem.RootFolder,
            srcRootPath,
            MyItem.SourcePath);

        MyItem.Status = "Transferred";

        return "Transferred";
    }

And here is the UploadFileorFolder method, which is using the SharePoint.Client.File.SaveBinaryDirect method to upload a file to SharePoint...

    public static void UploadFileOrFolder(string siteUrl, string libraryName, string rootFolder, string path, string file)
    {
        ClientContext ctx = GetSPContext(siteUrl);

        using (fs = new FileStream(file, FileMode.Open, FileAccess.Read, FileShare.Read, bufferSize: 4096, useAsync: true))
        {
            Microsoft.SharePoint.Client.File.SaveBinaryDirect(ctx, targetFileUrl, fs, true);
        }
    }

Answer 1

Your ProcessItem method is marked as async, and it returns a Task, which indicates it's supposed to be asynchronous, but it is not in fact asynchronous. it will execute entirely synchronously. async merely allows the use of await in a method, and does nothing to actually make a method asynchronous. You are currently getting a compiler warning informing you of this fact.

If the API you're using exposes an actually asynchronous method to upload the file you're trying to upload, you should be using that instead of the synchronous version. If it doesn't, then while it's undesirable, you'll need to use Task.Run (or an equivalent) to run the synchronous method in a thread pool thread asynchronously.