Uneven chunking in python

Question

Given a list of chunk sizes, how would you partition an iterable into variable-length chunks? I'm trying to coax itertools.islice without success yet.

for chunk_size in chunk_list: 
   foo(iter, chunk_size)

Answer 1

You need to make an iter object of your iterable so you can call islice on it with a particular size, and pick up where you left off on the next iteration. This is a perfect use for a generator function:

def uneven_chunker(iterable, chunk_list):
    group_maker = iter(iterable)
    for chunk_size in chunk_list:
        yield itertools.islice(group_maker, chunk_size)

Example:

>>> iterable = 'the quick brown fox jumps over the lazy dog'
>>> chunk_size = [1, 2, 3, 4, 5, 6]
>>> for item in uneven_chunker(iterable, chunk_size):
...     print ''.join(item)
...
t
he
 qu
ick
brown
 fox j