Is there a valid reason why nn.CDivTable throws error when backward is called?

Question

I've recently started playing with neural networks using Torch framework and Lua scripting language. I've got the basics with linear networks, so I tried something more complex but simple enough:

the idea is that I have 3 inputs, I have to choose first two, divide them, and forward the result to linear module. So, I've made this little script:

require "nn";
require "optim";

local N = 3;

local input = torch.Tensor{
    {1, 2, 3},
    {9, 20, 20},
    {9, 300, 1},
};

local output = torch.Tensor(N);
for i=1, N do
    output[i] = 1;
end

local ratioPerceptron = nn.Sequential();
ratioPerceptron:add(nn.Narrow(1, 1, 2));
ratioPerceptron:add(nn.CDivTable());
ratioPerceptron:add(nn.Reshape(N, 1));
ratioPerceptron:add(nn.Linear(1, 1));
ratioPerceptron:add(nn.Sigmoid());

local criterion = nn.BCECriterion();
local params, gradParams = ratioPerceptron:getParameters();
local optimState = {learningRate = 0.01};

local maxIteration = 100000;
for i=1, maxIteration do
    local function f(params)
        gradParams:zero();

        local outputs = ratioPerceptron:forward(input);
        local loss = criterion:forward(outputs, output);
        local dloss_doutputs = criterion:backward(outputs, output);
        ratioPerceptron:backward(input, dloss_doutputs);

        return loss, gradParams;
    end

    optim.sgd(f, params, optimState);
end

This fails when backward is called during training with error:

CDivTable.lua:21: both torch.LongStorage and (null) have no addition operator

But if I remove CDivTable from sequential module, and change nn.Reshape and nn.Linear to two-dimensional input (since we removed CDivTable which divides two-dim input to produce one-dim output) like this:

local ratioPerceptron = nn.Sequential();
ratioPerceptron:add(nn.Narrow(1, 1, 2));
ratioPerceptron:add(nn.Reshape(N, 2));
ratioPerceptron:add(nn.Linear(2, 1));
ratioPerceptron:add(nn.Sigmoid());

Training finishes without error... Is there any other way to divide two selected inputs and forward the result to the linear module?

Answer 1

The module CDivTable take a table as input and divides the elements of the first table by the ones of the second table. Here you feed your network with as single input, and not a table of two input. That is why you have an error with null I believe. Torch is unable to understand that your input (which consists in two vectors) should be considered as a table of two vectors. It only sees a tensor of size 2x3! Therefore you have to tell Torch to create a table from the input. Therefore you can use the module SplitTable(dim) that will split the input into tables along the dimension dim.

Insert this line ratioPerceptron:add(nn.SplitTable(1)) after the narrow module:

local ratioPerceptron = nn.Sequential();
ratioPerceptron:add(nn.Narrow(1, 1, 2));
ratioPerceptron:add(nn.SplitTable(1))
ratioPerceptron:add(nn.CDivTable());
ratioPerceptron:add(nn.Reshape(N, 1));
ratioPerceptron:add(nn.Linear(1, 1));
ratioPerceptron:add(nn.Sigmoid());

Besides, when you have such errors I suggest you looking at what is computed by your network by putting print statements: insert a line print(ratioPerceptron:forward(input)) before the line where you add a module that creates an error.