CNTK: cloning a single layer

Question

My current goal is to clone a single layer from an already trained model.

The first problem is that the clone() method clones the entire graph from the supplied node which is not what I want.

So I tried cloning it manually (in this case a Dense layer) by retrieving its weights from the node as follows:

node = C.logging.graph.find_by_name(model, 'node')
C.layers.Dense(node.shape, init=node.W.value, init_bias=node.b.value)

Unfortunately this does not work since I get the following shady error:

TypeError: in method 'random_initializer_with_rank', argument 1 of type 'CNTK::ParameterInitializer const &'

Answer 1

The clone() method does not necessarily clone the entire graph. It allows you to "cut out" a piece of graph, via the substitutions argument. The substitutions argument specifies the input nodes of the part of the graph you want to clone; basically where you want to cut it.

For example, to clone a middle layer of a stack, identify

• its root, let's call it layer_root
• its input(s). Let's say there is one input node, you store it as layer_input

Then you should be able to clone just this part according to this following code sketch:

substitutions = {
    layer_input : C.placeholder(name='cloned_layer_input')
}
cloned_layer = layer_root.clone(clone_method, substitutions)

The substitutions will cause clone() to stop cloning once it hits layer_input, and in the clone, replace it with the placeholder.

The result will be a callable, like any layers of the layers lib (like C.Dense()) or any function defined with @C.Function, which is I believe what you are looking for.