I have a dozen pre-trained Keras DNNs that I wish to add to an sklearn ensemble. The issue is that it seems I can not provide pre-trained models to KerasClassifier. KerasClassifier might only exist to create new, untrained models.
classifier_models = []
# The models variable contains a dict of pre-trained models.
for name, model in models:
try:
# This line raises an error if we use a pre-trained DNN.
model._estimator_type
classifier_models.append((name, model))
except:
# If we got a pre-trained Keras DNN, we must wrap it, then...
new_model = KerasClassifier(model=model)
# ...we set the _estimator_type and...
new_model._estimator_type = 'classifier'
# ...we continue the standard procedure.
classifier_models.append((name, new_model))
ERROR:
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_11744/990400553.py in <module>
3 try:
----> 4 model._estimator_type
5 classifier_models.append((name, model))
AttributeError: 'Sequential' object has no attribute '_estimator_type'
During handling of the above exception, another exception occurred:
AttributeError Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_11744/990400553.py in <module>
5 classifier_models.append((name, model))
6 except:
----> 7 new_model = KerasClassifier(model=model)
8 new_model._estimator_type = 'classifier'
9 classifier_models.append((name, new_model))
~\miniconda3\envs\epfl-ml\lib\site-packages\tensorflow\python\keras\wrappers\scikit_learn.py in __init__(self, build_fn, **sk_params)
75 self.build_fn = build_fn
76 self.sk_params = sk_params
---> 77 self.check_params(sk_params)
78
79 def check_params(self, params):
~\miniconda3\envs\epfl-ml\lib\site-packages\tensorflow\python\keras\wrappers\scikit_learn.py in check_params(self, params)
91 ]
92 if self.build_fn is None:
---> 93 legal_params_fns.append(self.__call__)
94 elif (not isinstance(self.build_fn, types.FunctionType) and
95 not isinstance(self.build_fn, types.MethodType)):
AttributeError: 'KerasClassifier' object has no attribute '__call__'
I do not wish to use KerasClassifier with a build function (example: KerasClassifier(build_fn=build_dnn()) because I already have a trained network and it would take a lot of time to re-train.
Apparently, the best way to go about this is using SciKeras which is an enhanced fork of that wrapper class that enables a lot more functionality including functional model types and multi-output plus you can use pre-built models. I'm currently figuring out the same thing you are but this looks promising.
It looks like the first variable model for the scikeras.wrappers.KerasClassifier wrapper can be either a function that builds a keras model or an already built one.
from the scikeras documentation for the KerasClassifier constructor parameters:
model:Union[None, Callable[…, tf.keras.Model], tf.keras.Model],default None
Used to build the Keras Model. When called, **must return a compiled instance of a Keras Model** to be used by fit, predict, etc. If None, you must implement _keras_build_fn.
Additionally:
Attributes model_tf.keras.Model
The instantiated and compiled Keras Model. For pre-built models, this will just be a reference to the passed Model instance.
https://www.adriangb.com/scikeras/stable/generated/scikeras.wrappers.KerasClassifier.html
So for example your wrapper would look like so:
wrapped_model = KerasClassifier(model=pre_built_model_here,optimizer='Adam',...ect)