CNTK Python API: access layers after loading model

Question

I can not access layers after loading the model.

I created the model as follows:

def create_model(vocab_dim, hidden_dim):

    input_seq_axis1 = Axis('inputAxis1')
    input_sequence_before = sequence.input_variable(shape=vocab_dim, sequence_axis=input_seq_axis1, is_sparse = use_sparse)
    input_sequence_after = sequence.input_variable(shape=vocab_dim, sequence_axis=input_seq_axis1, is_sparse = use_sparse)
    e=Sequential([
        C.layers.Embedding(hidden_dim),
        Stabilizer()
        ],name='Embedding')
    a = Sequential([
        e,  
        C.layers.Recurrence(C.layers.LSTM(hidden_dim//2),name='ForwardRecurrence'),
        ],name='ForwardLayer')
    b = Sequential([
        e,  
        C.layers.Recurrence(C.layers.LSTM(hidden_dim//2),go_backwards=True),
       ],name='BackwardLayer')
    latent_vector = C.splice(a(input_sequence_before), b(input_sequence_after))

    bias = C.layers.Parameter(shape = (vocab_dim, 1), init = 0, name='Bias')
    weights = C.layers.Parameter(shape = (vocab_dim, hidden_dim), init = C.initializer.glorot_uniform(), name='Weights')
    z = C.times_transpose(weights, latent_vector,name='Transpose') + bias
    z = C.reshape(z, shape = (vocab_dim))

    return z

Then I load the model:

def load_my_model(vocab_dim, hidden_dim):

    z=load_model("models/lm_epoch0.dnn")
    input_sequence_before = z.arguments[0]
    input_sequence_after = z.arguments[1]
    a=z.ForwardLayer
    b=z.BackwardLayer
    latent_vector = C.splice(a(input_sequence_before), b(input_sequence_after))

I get an error: TypeError("argument ForwardRecurrence's type SequenceOver[inputAxis1][Tensor[100]] is incompatible with the type SequenceOver[inputAxis1][SparseTensor[50000]] of the passed Variable",)

It looks like the layer referenced by name (z.ForwardLayer) represents the function from the layer immediate input. How can I calculate "latent_vector" (I need this variable to create cross entropy and loss functions to continue training) ?

Answer 1

Based on the error, the dimensions of your input seq is too big (5000) compared to what your ForwardLayer expects (100).

When you select the node ForwardLayer via z.ForwardLayer, you are only select that very specific node/layer, but not the layers/nodes/rest of the computation graphs that is connected to it.

You should do a = C.combine([z.ForwardLayer.owner]) and you should be fine.