Search code examples
numpyarray-broadcasting

Simple numpy broadcasting


I'm sure this question has an answer already somewhere, but I'm having trouble finding it since there are so many broadcasting questions, so perhaps someone can either point to it or answer directly.

I want to apply a function to each element of a NumPy array.

I have:

def test1(x):
    return np.sum(x)
def test2(y):
    return test1([5.0, y])
test2([1.0,2.0,3.0])
# I want the result to be [6.0, 7.0, 8.0]
# But I get TypeError: unsupported operand type(s) for +: 'float' and 'list'

How can I do this?

In Julia it would be:

test2.([1.0,2.0,3.0])

Thanks


Solution

  • Here's the full error message - which for some reason you decided wasn't relevant!

    In [99]: def test1(x):
        ...:     return np.sum(x)
        ...: def test2(y):
        ...:     return test1([5.0, y])
        ...: test2([1.0,2.0,3.0])
    /usr/local/lib/python3.8/dist-packages/numpy/core/fromnumeric.py:86: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.
      return ufunc.reduce(obj, axis, dtype, out, **passkwargs)
    Traceback (most recent call last):
      Input In [99] in <cell line: 5>
        test2([1.0,2.0,3.0])
      Input In [99] in test2
        return test1([5.0, y])
      Input In [99] in test1
        return np.sum(x)
      File <__array_function__ internals>:180 in sum
      File /usr/local/lib/python3.8/dist-packages/numpy/core/fromnumeric.py:2298 in sum
        return _wrapreduction(a, np.add, 'sum', axis, dtype, out, keepdims=keepdims,
      File /usr/local/lib/python3.8/dist-packages/numpy/core/fromnumeric.py:86 in _wrapreduction
        return ufunc.reduce(obj, axis, dtype, out, **passkwargs)
    TypeError: unsupported operand type(s) for +: 'float' and 'list'
    

    Let's add a debugging print to test1:

    In [100]: def test1(x):
         ...:     print(x)
         ...:     return np.sum(x)
         ...: test2([1.0,2.0,3.0])
    [5.0, [1.0, 2.0, 3.0]]
    ....
    

    Do you understand why x is this list - with a number and a list?

    np.sum is a numpy function, and like most, it converts the input into an array first:

    In [101]: np.array([5.0, [1.0, 2.0, 3.0]])
    <ipython-input-101-79819e7b65b8>:1: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.
      np.array([5.0, [1.0, 2.0, 3.0]])
    Out[101]: array([5.0, list([1.0, 2.0, 3.0])], dtype=object)
    

    If we pass an array instead, we still get the ragged array warning, but it runs:

    In [102]: test2(np.array([1,2,3]))
    [5.0, array([1, 2, 3])]
    /usr/local/lib/python3.8/dist-packages/numpy/core/fromnumeric.py:86: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.
      return ufunc.reduce(obj, axis, dtype, out, **passkwargs)
    Out[102]: array([6., 7., 8.])
    

    np.sum can operate on the 5.0 and np.array([1,2,3])

    If we use the python sum instead, the number and array do add - without that intermediate step of creating a ragged array:

    In [104]: sum([5, np.array([1,2,3])])
    Out[104]: array([6, 7, 8])
    

    Or the full function sequence:

    In [105]: def test1(x):
         ...:     return sum(x)
         ...: def test2(y):
         ...:     return test1([5.0, y])
         ...: test2(np.array([1.0,2.0,3.0]))
    Out[105]: array([6., 7., 8.])
    

    Another version:

    In [111]: def test1(x):
         ...:     return np.add(*x)
         ...: def test2(y):
         ...:     return test1([5.0, y])
         ...: test2([1.0,2.0,3.0])
    Out[111]: array([6., 7., 8.])
    

    np.sum takes an array(-like), and sums all the terms. np.add takes two arguments, and adds them (with broadcasting). *x is used to unpack the test2 list. np.add will convert the list y to array as needed.

    Stripped of the function calls, what you are trying to do is:

    In [112]: 5 + np.array([1,2,3])
    Out[112]: array([6, 7, 8])