I was trying to load the keras model which I saved during my training.So I went to keras documentation where I saw this.
Only topological loading (by_name=False) is supported when loading weights from the TensorFlow format. Note that topological loading differs slightly between TensorFlow and HDF5 formats for user-defined classes inheriting from tf.keras.Model: HDF5 loads based on a flattened list of weights, while the TensorFlow format loads based on the object-local names of attributes to which layers are assigned in the Model's constructor.
Could you please explain the above one?
For clarity purpose let's consider two cases.
Case 1: Simple model, and
Case 2: Complex model where user-defined classes inherited from tf.keras.Model were used.
When you save model weights (using model.save_weights) and then load weights (using model.load_weights), by default the load_weights method uses topological loading. This is same for Tensorflow saved_model ('tf') format as well as 'h5' format. For example,
loadedh5_model.load_weights('./MyModel_h5.h5')
# the line above is same as the line below (as second and third arguments are default)
#loadedh5_model.load_weights('./MyModel_h5.h5',by_name=False, skip_mismatch=False)
In case if you want to load weights of specific layers of a saved model, then you need to use by_name=True. There are use cases that requires this type of loading.
loadedh5_model.load_weights('./MyModel_h5.h5',by_name=True, skip_mismatch=False)
As of now only 'tf' format is only supported when user-defined classes inherited from tf.keras.Model were used in the model creation.
Only topological loading (by_name=False) is supported when loading weights from the TensorFlow format. Note that topological loading differs slightly between TensorFlow and HDF5 formats for user-defined classes inheriting from tf.keras.Model: HDF5 loads based on a flattened list of weights, while the TensorFlow format loads based on the object-local names of attributes to which layers are assigned in the Model's constructor.
The main reason is the way weights are in h5 format and tf format.
For example, consider Case 1 where HDF5 loads based on a flattened list of weights. The weights are loaded without any error. However, in Case 2, the model has user defined classes which requires different approach than just loading flattened weights. In order to take care of assigning weights of custom classes, 'tf' format load the weights based on the object-local names of attributes to which layers are assigned in the Model's constructor.
The following paragraph mentioned in keras website, further clarifies
When loading a weight file in TensorFlow format, returns the same status object as tf.train.Checkpoint.restore. When graph building, restore ops are run automatically as soon as the network is built (on first call for user-defined classes inheriting from Model, immediately if it is already built).
Another point to understand is keras Functional or Sequential models are static graphs of layers that can use flattened weights without any problem. Keras Subclassed model (as in our Case 2), is piece of Python code (a call method). There is no graph of layers. So as soon as the network is built with custom classes, restore ops are run to update status objects. Hope it helps.