I have a CSV file with 4 columns. 3 inputs and one output. Already normalized. I can use nnet and neuralnet to train a network with 3 inputs, 3 hidden layers with 3 nodes each and one output. It works.
I would like to do the same with MXNET but the parameter for a "FullyConected" has to be hidden = 1 when doing a regression. Any other value just throws an Error message.
How do I build a network as the one in the title or in this image?
This is the code:
csvIn <- read.csv("normalized.csv")
require(mxnet)
inputData <- csvIn[,1:3]
outputData <- csvIn[,4]
lcinm <- data.matrix(inputData, rownames.force = "NA")
lcoutm <- data.matrix(outputData, rownames.force = "NA")
lcouta <- as.numeric(lcoutm)
data <- mx.symbol.Variable("data")
fc1 <- mx.symbol.FullyConnected(data, name="fc1", num_hidden=3)
act1 <- mx.symbol.Activation(fc1, name="sigm1", act_type="sigmoid")
fc2 <- mx.symbol.FullyConnected(act1, name="fc2", num_hidden=3)
act2 <- mx.symbol.Activation(fc2, name="sigm2", act_type="sigmoid")
fc3 <- mx.symbol.FullyConnected(act2, name="fc3", num_hidden=3)
softmax <- mx.symbol.LinearRegressionOutput(fc3, name="softmax")
mx.set.seed(0)
mxn <- mx.model.FeedForward.create(array.layout = "rowmajor", softmax, X = lcinm, y = lcouta, learning.rate=0.07, eval.metric=mx.metric.rmse)
This is the error message:
Start training with 1 devices
[08:54:33] C:/Users/qkou/mxnet/dmlc-core/include/dmlc/logging.h:235: [08:54:33] src/ndarray/ndarray.cc:231: Check failed: from.shape() == to->shape() operands shape mismatch
Error in exec$update.arg.arrays(arg.arrays, match.name, skip.null) :
[08:54:33] src/ndarray/ndarray.cc:231: Check failed: from.shape() == to->shape() operands shape mismatch
Input data (3 nodes)
> lcinm
INA INV INC
[1,] 0.327172792 0.1842063931 0.50227366
[2,] 0.328585645 0.1911366252 0.50394467
[3,] 0.329998499 0.1980668574 0.50557458
[4,] 0.333367019 0.1994041603 0.50606766
[5,] 0.338691205 0.2007416800 0.50656075
[6,] 0.344015391 0.2020789830 0.50705383
[7,] 0.345432095 0.2021049795 0.50698534
[8,] 0.346848798 0.2021309760 0.50691686
[9,] 0.348355970 0.2026784188 0.50617724
[10,] 0.349953611 0.2032256450 0.50542391
Output data (1 node)
> lcouta
[1] 0.6334235 0.6336314 0.6338394 0.6339434 0.6339434 0.6339434
[7] 0.6306156 0.6272879 0.6241681 0.6212562 0.6183444 0.6170965
Try the following:
data <- mx.symbol.Variable("data")
fc1 <- mx.symbol.FullyConnected(data, name="fc1", num_hidden=3)
act1 <- mx.symbol.Activation(fc1, name="sigm1", act_type="sigmoid")
fc2 <- mx.symbol.FullyConnected(act1, name="fc2", num_hidden=3)
act2 <- mx.symbol.Activation(fc2, name="sigm2", act_type="sigmoid")
fc3 <- mx.symbol.FullyConnected(act2, name="fc3", num_hidden=3)
act3 <- mx.symbol.Activation(fc3, name="sigm3", act_type="sigmoid")
fc4 <- mx.symbol.FullyConnected(act3, name="fc4", num_hidden=1)
linear_reg_ output <- mx.symbol.LinearRegressionOutput(fc4, name="output")
Here fc4 is actually the output of the whole network. We use LinearRegressionOutput as final output to enable the network to optimize for squared loss.You can also directly use fc4 as output and write you own loss function.