Search code examples
pythonlambdageneratoryield

Can a python lambda/fn yield on behalf of an arbitrary caller?


UPDATE: example now lists desired results (boldfaced below)

I find myself writing lots of functions that search through some data, where I want to let the caller specify behaviours when matches are found: they might print something out or add it to one of their data structures, but it's also highly desirable to be able to optionally return found data for further transmission, storage or processing.

Example

def find_stuff(visitor):    # library search function
    for x in (1, 2, 3, 4, 5, 6):
        visitor(x)

First client usage:

def my_visitor(x):   # client visitor functions (also often use lambdas)
    if x > 3:
        yield x / 2   #>>> WANT TO DO SOMETHING LIKE THIS <<<#

results = find_stuff(my_visitor)   # client usage

results should yield 4/2, 5/2, then 6/2... i.e. 2, 2, 3.

Second client usage:

def print_repr_visitor(x):
    print repr(x)

find_stuff(print_repr_visitor)     # alternative usage

should print 1 2 3 4 5 6 (separate lines) but yield nothing

But, the yield doesn't create a generator in "results" (at least with python 2.6.6 which I'm stuck with).


What I've tried

I've been hacking this up, often like this...

def find_stuff(visitor):
    for x in (1, 2, 3, 4, 5):
        val = visitor(x)
        if val is not None:
             yield val

...or sometimes, when the list of visitor parameters is a pain to type out too many times...

def find_stuff(visitor):
    for x in (1, 2, 3, 4, 5):
        val = visitor(x)
        if val == 'yield':
            yield x
        elif val is not None:
             yield val

The Issues / Question

These "solutions" are not only clumsy - needing explicit built-in support from the "find" routine - they remove sentinel values from the set of results the visitor can yield back to the top-level caller...

    Are there better alternatives in terms of concision, intuitiveness, flexibility, elegance etc?

Thanks!


Solution

  • In Python 3, you can use yield from to yield items from a subgenerator:

    def find_stuff(visitor):
        for x in (1, 2, 3, 4, 5):
            yield from visitor(x)
    

    In Python 2, you have to loop over the subgenerator. This takes more code and doesn't handle a few edge cases, but it's usually good enough:

    def find_stuff(visitor):
        for x in (1, 2, 3, 4, 5):
            for item in visitor(x):
                yield item
    

    The edge cases are things like trying to send values or throw exceptions into the subgenerator. If you're not using coroutine functionality, you probably don't need to worry about them.