How can I generate unique itertools chains?

Question

For example, what's the itertools.chain() equivalent of:

set.union({1,2,3},{3,4,2,5},{1,6,2,7})

(obviously that returns a generator, rather than a set)

Answer 1

There isn't anything in itertools which will do this for you directly.

In order to avoid yielding duplicate items, you'll need to keep track of what you've already yielded, and the obvious way to do so is with a set. Here's a simple wrapper around itertools.chain() which does that:

from itertools import chain

def uniq_chain(*args, **kwargs):
    seen = set()
    for x in chain(*args, **kwargs):
        if x in seen:
            continue
        seen.add(x)
        yield x

... and here it is in action:

>>> list(uniq_chain(range(0, 20, 5), range(0, 20, 3), range(0, 20, 2)))
[0, 5, 10, 15, 3, 6, 9, 12, 18, 2, 4, 8, 14, 16]

Alternatively, if you prefer to compose a solution from smaller building blocks (which is a more flexible and "itertoolsy" way to do it), you could write a general purpose uniq() function and combine it with chain():

def uniq(iterable):
    seen = set()
    for x in iterable:
        if x in seen:
            continue
        seen.add(x)
        yield x

In action:

>>> list(uniq(chain(range(0, 20, 5), range(0, 20, 3), range(0, 20, 2))))
[0, 5, 10, 15, 3, 6, 9, 12, 18, 2, 4, 8, 14, 16]