I can't execute cross_val_score with keras.wrappers.scikit_learn.KerasRegressor

Question

# Python library

import pandas as pd
import numpy as np
import seaborn as sns
from tensorflow import keras
import matplotlib.pyplot as plt
from keras.wrappers.scikit_learn import KerasRegressor
from sklearn.model_selection import cross_val_score
from sklearn.model_selection import train_test_split

# 1) created from the data 

np.random.seed(0) 
m = 100 

X = np.linspace(0, 10, m).reshape(m,1) 
y = X + np.random.randn(m, 1)  

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)

scaler = MinMaxScaler()
X_train= scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)

# 2) Data visualization

print('dimensions de X:', X.shape)
print('dimensions de y:', y.shape)
plt.scatter(X,y)
plt.show()

# 3) Configuration of the Neural Network Layers

model = keras.Sequential()
model.add(keras.layers.Dense(100, activation='relu', input_dim=1))
model.add(keras.layers.Dropout(0.5))
model.add(keras.layers.Dense(100, activation='relu'))
model.add(keras.layers.Dropout(0.5))
model.add(keras.layers.Dense(1, activation='relu'))

# 4) Use the validation stick to train the model and display the learning curve   

Model = keras.Sequential([
keras.layers.Dense(4, activation='relu', input_dim=2),
keras.layers.Dense(2, activation='relu'),
keras.layers.Dense(1, activation='relu')])

opt = keras.optimizers.Adam()
Model.compile(opt, loss= 'mse')

Model = KerasRegressor(Model,batch_size=10,verbose=1, epochs=1000)

val_score = cross_val_score(Model, X_train, y_train, cv=10)
#plt.plot(val_score)

When I run the attached code normally it should work but for some reason it displays this error:

<ipython-input-6-041ea9940b77>:14: DeprecationWarning: KerasRegressor is deprecated, use Sci-Keras (https://github.com/adriangb/scikeras) instead. See https://www.adriangb.com/scikeras/stable/migration.html for help migrating.
  Model = KerasRegressor(Model,batch_size=10,verbose=1, epochs=1000)
/usr/local/lib/python3.8/dist-packages/sklearn/model_selection/_validation.py:372: FitFailedWarning: 
10 fits failed out of a total of 10.
The score on these train-test partitions for these parameters will be set to nan.
If these failures are not expected, you can try to debug them by setting error_score='raise'.

Below are more details about the failures:

10 fits failed with the following error:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/sklearn/model_selection/_validation.py", line 680, in _fit_and_score
    estimator.fit(X_train, y_train, **fit_params)
  File "/usr/local/lib/python3.8/dist-packages/keras/wrappers/scikit_learn.py", line 152, in fit
    self.model = self.build_fn(
  File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 67, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "/usr/local/lib/python3.8/dist-packages/keras/engine/base_layer.py", line 3100, in _split_out_first_arg
    raise ValueError(
ValueError: The first argument to `Layer.call` must always be passed.

warnings.warn(some_fits_failed_message, FitFailedWarning)

Answer 1

I don't know if the warning always triggers an error, but it's informing you that using the built-in wrapper is deprecated. The migrating guide is worth reading, but the basic steps are:

1. Install scikeras (See also: docs)

pip install scikeras[tensorflow]

2. Replace keras.wrappers with scikeras.wrappers:

# from keras.wrappers.scikit_learn import KerasRegressor
from scikeras.wrappers import KerasRegressor

Full example:

from tensorflow import keras
from sklearn.model_selection import cross_val_score
from sklearn.datasets import make_regression
from scikeras.wrappers import KerasRegressor

X, y = make_regression(n_samples=10_000)
input_shape = X.shape[1]
model = keras.Sequential([
    keras.layers.Dense(100, activation='relu', input_dim=input_shape),
    keras.layers.Dense(200, activation='relu'),
    keras.layers.Dense(200, activation='relu'),
    keras.layers.Dense(1, activation='linear')])
model.compile(keras.optimizers.Adam(), loss='mse')

model = KerasRegressor(model, batch_size=256, verbose=1, epochs=10)

val_score = cross_val_score(model, X, y, cv=5)