I need to write a simple script that loads data from multiple files and merges it somehow. However, given the fact that the files might be quite huge I'd like to load data partially. To do so I decided to use yield. And according to examples I found I could use following construction for single generator:
$generator = $someClass->load(); //load method uses yield so it returns generator object
foreach($generator as $i) {
// do something
}
But what if I want to use two generators at once?
$generatorA = $someClass1->load(); //load method uses yield so it returns generator object
$generatorB = $someClass2->load(); //load method uses yield so it returns generator object
foreach($generatorA as $i) {
// how can I access to resultSet from generatorB here?
}
Generators in PHP implement the Iterator interface, so you can merge / combine multiple Generators like you can combine multiple Iterators.
If you want to iterate over both generators one after the other (merge A + B), then you can make use of the AppendIterator.
$aAndB = new AppendIterator();
$aAndB->append($generatorA);
$aAndB->append($generatorB);
foreach ($aAndB as $i) {
...
If you want to iterate over both generator at once, you can make use of MultipleIterator.
$both = new MultipleIterator();
$both->attachIterator($generatorA);
$both->attachIterator($generatorB);
foreach ($both as list($valueA, $valueB)) {
...
Example for those two incl. examples and caveats are in this blog-post of mine as well:
This is a caveat useful to understand when passing Iterators along that are Generators, and as it may happen when composing them.
As calling a generator function already executes to the first yield (or return), it is an iterator that can not rewind and throw if they would be rewound:
PHP Fatal error: Uncaught Exception: Cannot rewind a generator that was already run (PHP 8.2)
PHP Fatal error: Uncaught exception 'Exception' with message 'Cannot rewind a generator that was already run' (PHP 5.6)
PHP Fatal error: Uncaught Exception: Cannot traverse an already closed generator (PHP 8.2)
PHP Fatal error: Uncaught exception 'Exception' with message 'Cannot traverse an already closed generator' (PHP 5.6)
In Nikic's iter library you can find an implementation of a rewindable generator that works by invoking the generator function with its arguments again.
When decorating or composing Generators, you may want to handle this alternatively by rendering the rewind() method of the Iterator protocol void.
PHP has a standard implementation for that with the NoRewindIterator. Wrapping the Generator within then allows to re-iterate over the generator without throwing.
This can have the benefit to hide the throwing behaviour and make a Generator behave more expected with whole the Iterator protocol.
$genFunc = static function () {
yield 'k' => 'v';
};
$iter = new NoRewindIterator($genFunc());
foreach (new LimitIterator($iter, 0, 1) as $k => $v) {
var_dump("[ $k => $v ]");
}
foreach (new LimitIterator($iter, 0, 1) as $k => $v) {
var_dump("[ $k => $v ]");
}
At very rare places if the abstraction still leaks, there is also CachingIterator but I don't have a practical example at hand, only remembering a scenario where getting the count of an overall collection in advance, but then having segments to pull and then yield, so a chain of generators from a generator that could also be empty, by optimistically lazy fetching and the collection then could be smaller or larger as by the initial count.