Search code examples
javascripttypescriptecmascript-6iteratorgenerator

Reusing a Generator in Multiple Functions/Methods


I am trying to create a file reader object (from readFileSync) and serve the lines from a generator function. My intention is to pass this gnerator object to multiple functions and sequentialy parse a file. However, after using the generator in a single function, the state of the generator shift from suspended to closed. I come from a Python background and this is a very much possible operation in Python. Would like to know what I am doing wrong here. Following is the code I used:

Generator function definition (I am using readFileSync and it is not async, please disregard that for the time being as I am trying to get the generator working):

function* getFileGen(path: string){
  const fileContent = fs
  .readFileSync(path, {
    encoding: "utf-8",
    flag: "r",
  })
  .split("\n");

  while(true){
      const thisLine = fileContent.shift();
      if(!thisLine){
        break;
      }
      yield thisLine; 
  }
}

The two functions in which I would like to use the generator in:

function getFirstFew(stream: Generator){
  let i = 0;
  for(let v of stream){
    console.log(v);
    if(i > 1){
      break;
    }
    i++;
  }
}

function getNextFew(stream: Generator){
  let i = 0;
  for(let v of stream){
    console.log(v);
    if(i > 7){
      break;
    }
    i++;
  }

And finally create a generator and pass it sequentially to two functions that would print a number of lines:

const myStream = getFileGen('path/to/file');

getFirstFew(myStream);
getNextFew(myStream);

The first function executes correctly and prints 3 lines; however by the time the generator is passed to the getNextFew function, it has already closed.


Solution

  • From the docs:

    In for...of loops, abrupt iteration termination can be caused by break, throw or return. In these cases, the iterator is closed.

    And, specifically:

    Do not reuse generators

    Generators should not be re-used, even if the for...of loop is terminated early, for example via the break keyword. Upon exiting a loop, the generator is closed and trying to iterate over it again does not yield any further results.

    Emphasis mine.

    I'll admit though, I'm not very strong with JS, so I can't recommend a comparable workaround. You may need to use a list or another strict structure that you have more control over.

    You may be able to implement a tee function comparable to Python's tee that creates a copy of the iterator, then iterate one of the copies.