Multiple enumeration of IEnumerable - StackOverflowException

Question

An interesting problem with IEnumerable and Linq.

static void Main(string[] args)
{
    var subject = "#Accountancy #premier #Agile #Apache #automation #Automation #banking #Banking #bankIngs #AutoMation";
    var hashtags = subject.Split('#').Select(hashtag => hashtag.Trim().ToUpper()).Distinct();

    var plurals = hashtags.Where((hashtag) =>
    {
        return hashtags.Contains($"{hashtag.ToUpper()}S");
    }).Select(h => $"{h.ToUpper()}S");      //.ToList(); (1) - will not break

    //filter hashtags
    hashtags = hashtags.Except(plurals);    //.ToList(); (2) - will not break

    //if iterate, would break with:
    //System.StackOverflowException was unhandled Message: An unhandled exception of type 'System.StackOverflowException' occurred in mscorlib.dll
    foreach (var hashtag in hashtags)
    {
        Console.WriteLine(hashtag);
    }

    Console.Read();
}

Curious on how would one explain why overflow exception occurs?

Answer 1

Walk through it step by step.

1. Plurals is every word in hashtags which also has the same word ending in s
2. Hashtags is every word except plurals.

To execute 2, you must execute 1. However, hashtags is continually changing, so plurals is trying to execute not on the original collection, but on the result of 2 (which again, relies on the result of 1).

Your query will attempt to be:

hashtags = hashtags.Except(plurals);

Replacing plurals

hashtags = hashtags.Except(
            hashtags.Where(hashtag => { return hashtags.Contains($"{hashtag.ToUpper()}S"); })
                    .Select(h => $"{h.ToUpper()}S")
           );

But hashtags is hashtags.Except(plurals);

hashtags.Except(
            hashtags.Except(plurals).Where(hashtag => { return hashtags.Contains($"{hashtag.ToUpper()}S"); })
                    .Select(h => $"{h.ToUpper()}S")
           );

And then we need to replace plurals again.. and so on.

Your fixes (adding .ToList()) is the logical way to fix it.