Non-linear regression with neural networks

Question

How do you create an NN to solve for example: 2 * x ** 3 + 4 * x + 8 + random noise? I can do this easily with LinearRegression from sklearn, but I'd like to be able to achieve this for a multivariate sample where I have no idea wether the function is log/exp/poly/etc. Thank you! Ashkan

Answer 1

The nonlinearity in Neural Network can be achieved by simply having a layer with a nonlinear activation function, e.g. (relu). For example:

from tensorflow.keras import Sequential
from tensorflow.keras.layers import Dense

model = Sequential()
model.add(Dense(10, activation='relu', kernel_initializer='he_normal', input_shape=(n_features,)))
model.add(Dense(1))

model.compile(optimizer='adam', loss='mse')
model.fit(X_train, y_train, epochs=10, batch_size=16, verbose=0)

Just needed 3 more layers of these - thanks Tony!