Search code examples
pythonnumpytheano

Theano: Operate on nonzero elements of sparse matrix

I'm trying to take the exp of nonzero elements in a sparse theano variable. I have the current code:

A = T.matrix("Some matrix with many zeros")
A_sparse = theano.sparse.csc_from_dense(A)

I'm trying to do something that's equivalent to the following numpy syntax:

mask = (A_sparse != 0)
A_sparse[mask] = np.exp(A_sparse[mask])

but Theano doesn't support != masks yet. (And (A_sparse > 0) | (A_sparse < 0) doesn't seem to work either.)

How can I achieve this?

Solution

  • The support for sparse matrices in Theano is incomplete, so some things are tricky to achieve. You can use theano.sparse.structured_exp(A_sparse) in that particular case, but I try to answer your question more generally below.

    Comparison

    In Theano one would normally use the comparison operators described here: http://deeplearning.net/software/theano/library/tensor/basic.html

    For example, instead of A != 0, one would write T.neq(A, 0). With sparse matrices one has to use the comparison operators in theano.sparse. Both operators have to be sparse matrices, and the result is also a sparse matrix:

    mask = theano.sparse.neq(A_sparse, theano.sparse.sp_zeros_like(A_sparse))

    Modifying a Subtensor

    In order to modify part of a matrix, one can use theano.tensor.set_subtensor. With dense matrices this would work:

    indices = mask.nonzero()
A = T.set_subtensor(A[indices], T.exp(A[indices]))

    Notice that Theano doesn't have a separated boolean type—the mask is zeros and ones—so nonzero() has to be called first to take the indices of the nonzero elements. Furthermore, this is not implemented for sparse matrices.

    Operating on Nonzero Sparse Elements

    Theano provides sparse operations that are said to be structured and operate only on the nonzero elements. See: http://deeplearning.net/software/theano/tutorial/sparse.html#structured-operation

    More precisely, they operate on the data attribute of a sparse matrix, independent of the indices of the elements. Such operations are straightforward to implement. Note that the structured operations will operate on all the values in the data array, also those that are explicitly set to zero.