Mxnet element wise multiply

In MXNet, if I wanted to create a vector of weights that multiplied each input, i.e. to have w*x_i and then backprop over the weights w how would I do this?

I tried:

 y_hat = input
 w1 = mx.sym.Variable("w1")
 y_hat = mx.symbol.broadcast_mul(w1, y_hat)

Solution

You can cast the computation in terms of a dot product:

x = mx.nd.array([[1, 2, 3], [4, 5, 6]])
w = mx.nd.array([2,2,2])
mx.nd.dot(w, x.T)

will result in [ 12. 30.] as you wish.

Now just initialize w randomly, compute a loss between the output and your target output and then back propagate. You can use the new gluon interface for that (http://gluon.mxnet.io/).

Specifically, let's look at a minimal example adapted http://mxnet.io/tutorials/gluon/gluon.html and http://gluon.mxnet.io/P01-C05-autograd.html

Prepare the data

label = mx.nd.array([12,30])
x = mx.nd.array([[1, 2, 3], [4, 5, 6]])
w = random weights
w.attach_grad()

And train

with autograd.record():
    output = mx.nd.dot(w, x.T)
    loss = gluon.loss.L2Loss(output, label)
    loss.backward()

Don't forget updating the weights with the gradient you computed in the backward pass. The gradient will be available in w.grad. Run the training code together with the weight update in a loop as a single update likely won't suffice for convergence.