Search code examples
pythonarraysnumpytypestruthiness

Truth value of numpy array with one falsey element seems to depend on dtype


import numpy as np
a = np.array([0])
b = np.array([None])
c = np.array([''])
d = np.array([' '])

Why should we have this inconsistency:

>>> bool(a)
False
>>> bool(b)
False
>>> bool(c)
True
>>> bool(d)
False

Solution

  • For arrays with one element, the array's truth value is determined by the truth value of that element.

    The main point to make is that np.array(['']) is not an array containing one empty Python string. This array is created to hold strings of exactly one byte each and NumPy pads strings that are too short with the null character. This means that the array is equal to np.array(['\0']).

    In this regard, NumPy is being consistent with Python which evaluates bool('\0') as True.

    In fact, the only strings which are False in NumPy arrays are strings which do not contain any non-whitespace characters ('\0' is not a whitespace character).

    Details of this Boolean evaluation are presented below.


    Navigating NumPy's labyrinthine source code is not always easy, but we can find the code governing how values in different datatypes are mapped to Boolean values in the arraytypes.c.src file. This will explain how bool(a), bool(b), bool(c) and bool(d) are determined.

    Before we get to the code in that file, we can see that calling bool() on a NumPy array invokes the internal _array_nonzero() function. If the array is empty, we get False. If there are two or more elements we get an error. But if the array has exactly one element, we hit the line:

    return PyArray_DESCR(mp)->f->nonzero(PyArray_DATA(mp), mp);
    

    Now, PyArray_DESCR is a struct holding various properties for the array. f is a pointer to another struct PyArray_ArrFuncs that holds the array's nonzero function. In other words, NumPy is going to call upon the array's own special nonzero function to check the Boolean value of that one element.

    Determining whether an element is nonzero or not is obviously going to depend on the datatype of the element. The code implementing the type-specific nonzero functions can be found in the "nonzero" section of the arraytypes.c.src file.

    As we'd expect, floats, integers and complex numbers are False if they're equal with zero. This explains bool(a). In the case of object arrays, None is similarly going to be evaluated as False because NumPy just calls the PyObject_IsTrue function. This explains bool(b).

    To understand the results of bool(c) and bool(d), we see that the nonzero function for string type arrays is mapped to the STRING_nonzero function:

    static npy_bool
    STRING_nonzero (char *ip, PyArrayObject *ap)
    {
        int len = PyArray_DESCR(ap)->elsize; // size of dtype (not string length)
        int i;
        npy_bool nonz = NPY_FALSE;
    
        for (i = 0; i < len; i++) {
            if (!Py_STRING_ISSPACE(*ip)) {   // if it isn't whitespace, it's True
                nonz = NPY_TRUE;
                break;
            }
            ip++;
        }
        return nonz;
    }
    

    (The unicode case is more or less the same idea.)

    So in arrays with a string or unicode datatype, a string is only False if it contains only whitespace characters:

    >>> bool(np.array([' ']))
    False
    

    In the case of array c in the question, there is a really a null character \0 padding the seemingly-empty string:

    >>> np.array(['']) == np.array(['\0'])
    array([ True], dtype=bool)
    

    The STRING_nonzero function sees this non-whitespace character and so bool(c) is True.

    As noted at the start of this answer, this is consistent with Python's evaluation of strings containing a single null character: bool('\0') is also True.


    Update: Wim has fixed the behaviour detailed above in NumPy's master branch by making strings which contain only null characters, or a mix of only whitespace and null characters, evaluate to False. This means that NumPy 1.10+ will see that bool(np.array([''])) is False, which is much more in line with Python's treatment of "empty" strings.