Search code examples
pythonarraysnumpynumpy-ndarray

Partition array into N chunks with Numpy

There is this How do you split a list into evenly sized chunks? for splitting an array into chunks. Is there anyway to do this more efficiently for giant arrays using Numpy?

Solution

  • Try numpy.array_split.

    From the documentation:

    >>> x = np.arange(8.0)
>>> np.array_split(x, 3)
    [array([ 0.,  1.,  2.]), array([ 3.,  4.,  5.]), array([ 6.,  7.])]

    Identical to numpy.split, but won't raise an exception if the groups aren't equal length.

    If number of chunks > len(array) you get blank arrays nested inside, to address that - if your split array is saved in a, then you can remove empty arrays by:

    [x for x in a if x.size > 0]

    Just save that back in a if you wish.