I want to fit a neural network model using caret package.There are 208 predictors all of which are important and cannot be discarded. The maximum value that I can give to the size parameter is 4 beyond which I get an error saying that there are too many weights.
> ctrl<-trainControl(method = 'cv',number = 5)
> my.grid <- expand.grid(.decay = 0.1, .size =5)
> nn.fit <- train(train_predictors,train_responses[["r2c1"]],method = "nnet",algorithm = 'backprop', tuneGrid = my.grid,trace=F, linout = TRUE,trControl = ctrl)
Something is wrong; all the RMSE metric values are missing:
RMSE Rsquared MAE
Min. : NA Min. : NA Min. : NA
1st Qu.: NA 1st Qu.: NA 1st Qu.: NA
Median : NA Median : NA Median : NA
Mean :NaN Mean :NaN Mean :NaN
3rd Qu.: NA 3rd Qu.: NA 3rd Qu.: NA
Max. : NA Max. : NA Max. : NA
NA's :1 NA's :1 NA's :1
Error: Stopping
In addition: Warning messages:
1: model fit failed for Fold1: decay=0.1, size=5 Error in nnet.default(x, y, w, ...) : too many (1051) weights
2: model fit failed for Fold2: decay=0.1, size=5 Error in nnet.default(x, y, w, ...) : too many (1051) weights
3: model fit failed for Fold3: decay=0.1, size=5 Error in nnet.default(x, y, w, ...) : too many (1051) weights
4: model fit failed for Fold4: decay=0.1, size=5 Error in nnet.default(x, y, w, ...) : too many (1051) weights
5: model fit failed for Fold5: decay=0.1, size=5 Error in nnet.default(x, y, w, ...) : too many (1051) weights
6: In nominalTrainWorkflow(x = x, y = y, wts = weights, info = trainInfo, :
There were missing values in resampled performance measures.
The model performs very badly with 4 neurons(size=4).What can I do to make the model work if I want to have more than 5 neurons?
There are other parameters in the tuning grid that you can specify for the nnet method. The available parameters for each method are available online but difficult to find. Here's my example using mxnet for adam nn:
mxnet_grid_A2 = expand.grid(layer1 = c(10, 12),
layer2 = c(4, 6),
layer3 = 2,
learningrate = c(0.001, 0.0001),
dropout = c(0, 0.2)
beta1 = .9,
beta2 = 0.999,
activation = 'relu')