Can we use Deep neural network for regression problems?

Question

I want to train the DNN using ReLUs instead of typical sigmoid units. I have successfully implemented below example for Classification problem (softmax layer),but i am trying to implement this for regression.My data-set is based on Load forecasting. enter link description here

Answer 1

Yes , you can use deep learning for regression, the main difference its the output layer, and the cost function. Imagine you have multiple hidden layers,and the last layer has an activation function f(x) = x instead of softmax, where x its the weighted sum across the last hidden layer.

For the cost function you will use the common mean squared error for example instead of the common cross-entropy(commonly used with softmax). Take a look at this neural network i created to perform regression(its written from scratch in python but you can use it as an idea to implement it on some framework) https://github.com/llealgt/bikeshare_regression_neural_network/blob/master/DLND%20Your%20first%20neural%20network.ipynb