Search code examples
c#exceptionweb-scrapingtry-catchparallel.foreach

Continue after an exception in c#


I'm trying to get the doctypes of approx. 3k links. But I get an exception always when it hits the 700-900 mark line.

How can I continue at the point where the exception occurred(so I'm not obliged to start from zero once again)? Is that even possible?

Here is the code that I used:

     try
        {
            Parallel.ForEach(linkList, link => 
            {
                stopwatch.Restart();
                Console.Write($"Downloading page {index++} of {linkList.Count}...");
                documents.Add(LoadPage(link));
                Console.Write($" in {stopwatch.Elapsed.TotalMilliseconds} ms");
                Console.WriteLine();
            });

            return documents;
        }
        catch (Exception e)
        {
            ???
        }

Solution

  • Try wrapping the internal code in the try-catch

            Parallel.ForEach(linkList, link => 
            {
                try
                {
                    stopwatch.Restart();
                    Console.Write($"Downloading page {index++} of {linkList.Count}...");
                    documents.Add(LoadPage(link));
                    Console.Write($" in {stopwatch.Elapsed.TotalMilliseconds} ms");
                    Console.WriteLine();
                }
                catch (Exception e)
                {
                    ???
                }
            });
    
            return documents;
    

    EDIT:

    You may also want to look at thread-safe collections that C# has to offer as normal collections are not thread-safe