Access Weight and Bias with nn::sequential

Question

If I define std::vector<torch::nn::Linear> linear_layers; and fill this vector with some torch::nn::Linear objects, then I can access the weight and bias values by linear_layers[k].weight and linear_layers[k].bias. Same feature is available with other layer types, e.g., torch::nn::Conv2d.

If create my network using nn::sequential and then push back either of Linear or Conv2d I cannot access the weight and bias directly. Now, my question is how can I access the weight and bias values of each layer when I have used nn::sequential?

Thanks, Afshin

Answer 1

Here is the soultion: [see the link https://discuss.pytorch.org/t/common-class-of-linear-conv-etc/39987/8 ]

include

using namespace torch; using namespace torch::nn;

int main() { auto net = Sequential(Conv2d(1 /input channels/, 1 /output channels/, 2 /kernel size/), Conv2d(1, 1, 2));

for (auto& p : net->named_parameters()) {

    NoGradGuard no_grad;

    // Access name.
    std::cout << p.key() << std::endl;

    // Access weigth and bias.
    p.value().zero_(); // set all zero
    std::cout << p.value() << std::endl;
}

return 0;
}

The layers of a sequential, have the following naming convention: ., e.g. see the console output

0.weight # name of the layer
(1,1,.,.) = 
  0  0
  0  0
[ Variable[CPUFloatType]{1,1,2,2} ]
0.bias
 0
[ Variable[CPUFloatType]{1} ]
1.weight
(1,1,.,.) = 
  0  0
  0  0
[ Variable[CPUFloatType]{1,1,2,2} ]
1.bias
 0
[ Variable[CPUFloatType]{1} ]