Search code examples
c#algorithmlinqienumerableenumerator

"Unzip" IEnumerable dynamically in C# or best alternative


Lets assume you have a function that returns a lazily-enumerated object:

struct AnimalCount
{
    int Chickens;
    int Goats;
}

IEnumerable<AnimalCount> FarmsInEachPen()
{
    ....
    yield new AnimalCount(x, y);
    ....
}

You also have two functions that consume two separate IEnumerables, for example:

ConsumeChicken(IEnumerable<int>);
ConsumeGoat(IEnumerable<int>);

How can you call ConsumeChicken and ConsumeGoat without a) converting FarmsInEachPen() ToList() beforehand because it might have two zillion records, b) no multi-threading.

Basically:

ConsumeChicken(FarmsInEachPen().Select(x => x.Chickens));
ConsumeGoats(FarmsInEachPen().Select(x => x.Goats));

But without forcing the double enumeration.

I can solve it with multithread, but it gets unnecessarily complicated with a buffer queue for each list.

So I'm looking for a way to split the AnimalCount enumerator into two int enumerators without fully evaluating AnimalCount. There is no problem running ConsumeGoat and ConsumeChicken together in lock-step.

I can feel the solution just out of my grasp but I'm not quite there. I'm thinking along the lines of a helper function that returns an IEnumerable being fed into ConsumeChicken and each time the iterator is used, it internally calls ConsumeGoat, thus executing the two functions in lock-step. Except, of course, I don't want to call ConsumeGoat more than once..


Solution

  • I figured it out, thanks in large part due to the path that @Lee put me on.

    You need to share a single enumerator between the two zips, and use an adapter function to project the correct element into the sequence.

    private static IEnumerable<object> ConsumeChickens(IEnumerable<int> xList)
    {
        foreach (var x in xList)
        {
            Console.WriteLine("X: " + x);
            yield return null;
        }
    }
    
    private static IEnumerable<object> ConsumeGoats(IEnumerable<int> yList)
    {
        foreach (var y in yList)
        {
            Console.WriteLine("Y: " + y);
            yield return null;
        }
    }
    
    private static IEnumerable<int> SelectHelper(IEnumerator<AnimalCount> enumerator, int i)
    {
        bool c = i != 0 || enumerator.MoveNext();
        while (c)
        {
            if (i == 0)
            {
                yield return enumerator.Current.Chickens;
                c = enumerator.MoveNext();
            }
            else
            {
                yield return enumerator.Current.Goats;
            }
        }
    }
    
    private static void Main(string[] args)
    {
        var enumerator = GetAnimals().GetEnumerator();
    
        var chickensList = ConsumeChickens(SelectHelper(enumerator, 0));
        var goatsList = ConsumeGoats(SelectHelper(enumerator, 1));
    
        var temp = chickensList.Zip(goatsList, (i, i1) => (object) null);
        temp.ToList();
    
        Console.WriteLine("Total iterations: " + iterations);
    }