Search code examples
theano

Why do we need Theano reshape?

I don't understand why do we need tensor.reshape() function in Theano. It is said in the documentation:

Returns a view of this tensor that has been reshaped as in numpy.reshape.

As far as I understood, theano.tensor.var.TensorVariable is some entity that is used for computation graphs creation. And it is absolutely independent of shapes. For instance when you create your function you can pass there matrix 2x2 or matrix 100x200. As I thought reshape somehow restricts this variety. But it is not. Suppose the following example:

X = tensor.matrix('X')
X_resh = X.reshape((3, 3))
Y = X_resh ** 2
f = theano.function([X_resh], Y)

print(f(numpy.array([[1, 2], [3, 4]])))

As I understood, it should give an error since I passed matrix 2x2 not 3x3, but it computes element-wise squares perfectly.

So what is the shape of the theano tensor variable and where should we use it?

Solution

  • There is an error in the provided code though Theano fails to point this out.

    Instead of

    f = theano.function([X_resh], Y)

    you should really use

    f = theano.function([X], Y)

    Using the original code you are actually providing the tensor after the reshape so the reshape command never gets executed. This can be seen by adding

    theano.printing.debugprint(f)

    which prints

    Elemwise{sqr,no_inplace} [id A] ''   0
 |<TensorType(float64, matrix)> [id B]

    Note that there is no reshape operation in this compiled execution graph.

    If one changes the code so that X is used as the input instead of X_resh then Theano throws an error including the message

    ValueError: total size of new array must be unchanged Apply node that caused the error: Reshape{2}(X, TensorConstant{(2L,) of 3})

    This is expected because one cannot reshape a tensor with shape (2, 2) (i.e. 4 elements) into a tensor with shape (3, 3) (i.e. 9 elements).

    To address the broader question, we can use symbolic expressions in the target shape and those expressions can be functions of the input tensor's symbolic shape. Here's some examples:

    import numpy
import theano
import theano.tensor

X = theano.tensor.matrix('X')
X_vector = X.reshape((X.shape[0] * X.shape[1],))
X_row = X.reshape((1, X.shape[0] * X.shape[1]))
X_column = X.reshape((X.shape[0] * X.shape[1], 1))
X_3d = X.reshape((-1, X.shape[0], X.shape[1]))
f = theano.function([X], [X_vector, X_row, X_column, X_3d])

for output in f(numpy.array([[1, 2], [3, 4]])):
    print output.shape, output