Search code examples
pythonnumpyscipyipythonsparse-matrix

scipy.sparse __add__ method being called when adding to a regular numpy ndarray?

I'm calculating the dot product between a scipy.sparse matrix (CSC) and a numpy ndarray vector:

>>> print type(np_vector), np_vector.shape
<type 'numpy.ndarray'> (200,)
>>> print type(sp_matrix), sparse.isspmatrix(sp_matrix), sp_matrix.shape
<class 'scipy.sparse.csc.csc_matrix'> True (200, 200)
>>> dot_vector = dot(np_vector, sp_matrix)

The result seems to be a new ndarray vector as I was expecting:

>>> print type(dot_vector), dot_vector.shape
<type 'numpy.ndarray'> (200,)

But when I then try to add a scalar to that vector I receive the exception:

>>> scalar = 3.0
>>> print dot_vector + scalar 
C:\Python27\lib\site-packages\scipy\sparse\compressed.pyc in __add__(self, other)
    173                 return self.copy()
    174             else: # Now we would add this scalar to every element.
--> 175                 raise NotImplementedError('adding a nonzero scalar to a '
    176                                           'sparse matrix is not supported')
    177         elif isspmatrix(other):

NotImplementedError: adding a nonzero scalar to a sparse matrix is not supported

As if the result dot_vector was a sparse matrix again.

Specifically, it seems as if I have an ndarray but the sparse matrix __add__ is called for the + operator.

This is the method I would expect to be called:

>>> print dot_vector.__add__
<method-wrapper '__add__' of numpy.ndarray object at 0x05250690>

Am I missing something here or does this really look weird?
What determines which method is called for the + operator?
I am running this code in an IPython Notebook (ipython notebook --pylab inline). Could it be that IPython --pylab or the notebook kernel somehow screw things up?

Thanks for any help!

Solution

  • What your call to np.dot is doing is not very different from what you get if you do the following:

    >>> np.dot([1, 2, 3], 4)
array([ 4,  8, 12])

    Because np.dot doesn't know about sparse matrices, the return in your case is again the product of each element of the vector with your original sparse matrix. This is probably performed calling the __rmul__ method of the sparse matrix, so what you get is a 200 item array, each of which is a sparse matrix itself. When you try to add a scalar to that vector, it gets broadcasted, and when trying to add the scalar to each matrix, the error pops up.

    The correct way of doing this would be to call the .dot method of the sparse matrix. To premultiply by a row vector:

    >>> aa = sps.csc_matrix(np.arange(9).reshape(3, 3))
>>> bb = np.arange(3)
>>> aa.T.dot(bb)
array([15, 18, 21])

    And to postmultiply by a column vector:

    >>> aa.dot(bb)
array([ 5, 14, 23])

    And this are of course completely equivalent to what you would get operating with arrays:

    >>> aaa = np.arange(9).reshape(3,3)
>>> aaa.dot(bb)
array([ 5, 14, 23])
>>> bb.dot(aaa)
array([15, 18, 21])